Update README.md

README.md

@@ -1,7 +1,7 @@
 ---
-datasets: open-r1/numina-ALL-V4-verify-s1k
+datasets: open-r1/openr1-220k-math
 library_name: transformers
-model_name: R2-Q7B-GR1-ALL-s1k-5e-5-weight-decay-1e-4
+model_name: OpenR1-Qwen-7B
 tags:
 - generated_from_trainer
 - trl
@@ -9,50 +9,41 @@ tags:
 licence: license
 ---
 
-# Model Card for R2-Q7B-GR1-ALL-s1k-5e-5-weight-decay-1e-4
+# OpenR1-Qwen-7B
 
-This model is a fine-tuned version of [None](https://huggingface.co/None) on the [open-r1/numina-ALL-V4-verify-s1k](https://huggingface.co/datasets/open-r1/numina-ALL-V4-verify-s1k) dataset.
-It has been trained using [TRL](https://github.com/huggingface/trl).
+This is a finetune of [Qwen2.5-Math-Instruct](https://huggingface.co/Qwen/Qwen2.5-Math-7B-Instruct) on [OpenR1-220k-Math](https://huggingface.co/datasets/open-r1/openr1-220k-math) (`default` split).
 
 ## Quick start
 
 ```python
-from transformers import pipeline
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="HuggingFaceH4/R2-Q7B-GR1-ALL-s1k-5e-5-weight-decay-1e-4", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
-```
-
-## Training procedure
-
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/eliebak/huggingface/runs/e4oglhrp) 
+model_name = "open-r1/OpenR1-Qwen-7B"
+device = "cuda" 
 
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
 
-This model was trained with SFT.
+prompt = "Find the value of $x$ that satisfies the equation $4x+5 = 6x+7$."
 
-### Framework versions
-
-- TRL: 0.15.0.dev0
-- Transformers: 4.49.0.dev0
-- Pytorch: 2.5.1+cu121
-- Datasets: 3.2.0
-- Tokenizers: 0.21.0
+messages = [
+    {"role": "system", "content": "Please reason step by step, and put your final answer within \\boxed{}."},
+    {"role": "user", "content": prompt}
+]
+```
 
-## Citations
+## Training 
 
+We train the model on the `default` split of [OpenR1-220k-Math](https://huggingface.co/datasets/open-r1/openr1-220k-math) for 3 epochs. We use learning rate of 5e-5 and extend the context length from 4k to 32k, by increasing RoPE frequency to 300k. The training follows a linear learning rate schedule with a 10% warmup phase. The table below compares the performance of OpenR1-Qwen-7B to DeepSeek-Distill-Qwen-7B and OpenThinker-7B using [lighteval](https://github.com/huggingface/open-r1/tree/main?tab=readme-ov-file#evaluating-models).
 
+You can find the training and evaluation code at: https://github.com/huggingface/open-r1/
 
-Cite TRL as:
-    
-```bibtex
-@misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
-}
-```
+| Model | MATH-500 | AIME24 |
+| --- | --- | --- |
+| DeepSeek-Distill-Qwen-7B | 91.8 | 53.3 |
+| OpenR1-Qwen-7B | 91.0 | 53.3 |
+| OpenThinker-7B | 85.4 | 30.0 |