| | --- |
| | license: other |
| | base_model: TheDrummer/Behemoth-R1-123B-v2 |
| | tags: |
| | - nvfp4 |
| | - modelopt |
| | - quantized |
| | - blackwell |
| | - b200 |
| | library_name: transformers |
| | --- |
| | |
| | # Behemoth-R1-V2 ModelOpt NVFP4 |
| |
|
| | NVFP4 quantized version of [TheDrummer/Behemoth-R1-123B-v2](https://huggingface.co/TheDrummer/Behemoth-R1-123B-v2) using NVIDIA Model Optimizer. |
| |
|
| | ## Quantization Details |
| |
|
| | | Property | Value | |
| | |----------|-------| |
| | | **Original Model** | TheDrummer/Behemoth-R1-123B-v2 | |
| | | **Quantization** | NVFP4 (FP4 weights, FP16 activations) | |
| | | **Method** | NVIDIA ModelOpt PTQ | |
| | | **Calibration Samples** | 512 | |
| | | **Max Sequence Length** | 4096 | |
| |
|
| | ## Hardware Requirements |
| |
|
| | - **Optimal**: NVIDIA Blackwell GPUs (B100, B200, RTX PRO 6000 Blackwell) |
| | - **Compatible**: Hopper/Ampere (will use weight-only mode) |
| |
|
| | ## Usage with vLLM |
| |
|
| | ```python |
| | from vllm import LLM, SamplingParams |
| | |
| | llm = LLM( |
| | model="TheHouseOfTheDude/Behemoth-R1-V2_ModelOpt-NVFP4", |
| | quantization="modelopt", |
| | trust_remote_code=True, |
| | ) |
| | |
| | sampling_params = SamplingParams(temperature=0.8, top_p=0.95, max_tokens=512) |
| | outputs = llm.generate(["Write a story about..."], sampling_params) |
| | print(outputs[0].outputs[0].text) |
| | ``` |
| |
|
| | ## Chat Template |
| |
|
| | Uses Mistral v7 (Non-Tekken) format. See the original model card for usage details. |
| |
|
| | ## Credits |
| |
|
| | - Original Model: [TheDrummer](https://huggingface.co/TheDrummer) |
| | - Quantization: TheHouseOfTheDude |
| | - Quantization Framework: NVIDIA ModelOpt |
| |
|
