amd
/

DeepSeek-R1-MXFP4-Preview

8-bit precision

Model card Files Files and versions

linzhao-amd commited on Aug 4

Commit

5751a6a

·

verified ·

1 Parent(s): 2c85981

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -126,7 +126,7 @@ lm_eval --model local-completions \
 ```
 lm_eval \
     --model vllm \
-    --model_args pretrained=amd/DeepSeek-R1-MXFP4,dtype=auto,add_bos_token=True,tensor_parallel_size=$tp_size,gpu_memory_utilization=0.8,max_model_len=38768, \
     --tasks gsm8k \
     --num_fewshot 8 \
     --batch_size auto \

 ```
 lm_eval \
     --model vllm \
+    --model_args pretrained=amd/DeepSeek-R1-MXFP4,dtype=auto,add_bos_token=True,tensor_parallel_size=8,gpu_memory_utilization=0.8,max_model_len=38768, \
     --tasks gsm8k \
     --num_fewshot 8 \
     --batch_size auto \