Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-660/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-720/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-780/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-900/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-960/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-840/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-120/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-120/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-120/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-120/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-120/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-120/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

checkpoint-120/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 17500,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-120/trainer_state.json

@@ -0,0 +1,993 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.980269989615784,
+  "eval_steps": 500,
+  "global_step": 120,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.016614745586708203,
+      "grad_norm": 0.050998032093048096,
+      "learning_rate": 4.999991432639962e-05,
+      "loss": 0.5487,
+      "num_input_tokens_seen": 70408,
+      "step": 1
+    },
+    {
+      "epoch": 0.033229491173416406,
+      "grad_norm": 0.049370743334293365,
+      "learning_rate": 4.999965730618567e-05,
+      "loss": 0.4981,
+      "num_input_tokens_seen": 139640,
+      "step": 2
+    },
+    {
+      "epoch": 0.04984423676012461,
+      "grad_norm": 0.05077400803565979,
+      "learning_rate": 4.9999228941119745e-05,
+      "loss": 0.5505,
+      "num_input_tokens_seen": 223656,
+      "step": 3
+    },
+    {
+      "epoch": 0.06645898234683281,
+      "grad_norm": 0.04397282376885414,
+      "learning_rate": 4.999862923413781e-05,
+      "loss": 0.504,
+      "num_input_tokens_seen": 300688,
+      "step": 4
+    },
+    {
+      "epoch": 0.08307372793354102,
+      "grad_norm": 0.05225864797830582,
+      "learning_rate": 4.999785818935018e-05,
+      "loss": 0.4925,
+      "num_input_tokens_seen": 366368,
+      "step": 5
+    },
+    {
+      "epoch": 0.09968847352024922,
+      "grad_norm": 0.049482282251119614,
+      "learning_rate": 4.999691581204152e-05,
+      "loss": 0.4771,
+      "num_input_tokens_seen": 445808,
+      "step": 6
+    },
+    {
+      "epoch": 0.11630321910695743,
+      "grad_norm": 0.05594080314040184,
+      "learning_rate": 4.9995802108670775e-05,
+      "loss": 0.4986,
+      "num_input_tokens_seen": 522800,
+      "step": 7
+    },
+    {
+      "epoch": 0.13291796469366562,
+      "grad_norm": 0.051852282136678696,
+      "learning_rate": 4.999451708687114e-05,
+      "loss": 0.5171,
+      "num_input_tokens_seen": 599608,
+      "step": 8
+    },
+    {
+      "epoch": 0.14953271028037382,
+      "grad_norm": 0.045517683029174805,
+      "learning_rate": 4.9993060755450015e-05,
+      "loss": 0.5669,
+      "num_input_tokens_seen": 681424,
+      "step": 9
+    },
+    {
+      "epoch": 0.16614745586708204,
+      "grad_norm": 0.044325754046440125,
+      "learning_rate": 4.999143312438893e-05,
+      "loss": 0.4218,
+      "num_input_tokens_seen": 756744,
+      "step": 10
+    },
+    {
+      "epoch": 0.18276220145379024,
+      "grad_norm": 0.04328459873795509,
+      "learning_rate": 4.998963420484349e-05,
+      "loss": 0.434,
+      "num_input_tokens_seen": 842576,
+      "step": 11
+    },
+    {
+      "epoch": 0.19937694704049844,
+      "grad_norm": 0.04725787043571472,
+      "learning_rate": 4.998766400914329e-05,
+      "loss": 0.4287,
+      "num_input_tokens_seen": 917232,
+      "step": 12
+    },
+    {
+      "epoch": 0.21599169262720663,
+      "grad_norm": 0.03806879371404648,
+      "learning_rate": 4.9985522550791825e-05,
+      "loss": 0.3454,
+      "num_input_tokens_seen": 1006800,
+      "step": 13
+    },
+    {
+      "epoch": 0.23260643821391486,
+      "grad_norm": 0.05201176926493645,
+      "learning_rate": 4.998320984446641e-05,
+      "loss": 0.436,
+      "num_input_tokens_seen": 1085824,
+      "step": 14
+    },
+    {
+      "epoch": 0.24922118380062305,
+      "grad_norm": 0.047955628484487534,
+      "learning_rate": 4.9980725906018074e-05,
+      "loss": 0.4625,
+      "num_input_tokens_seen": 1164160,
+      "step": 15
+    },
+    {
+      "epoch": 0.26583592938733125,
+      "grad_norm": 0.05529098957777023,
+      "learning_rate": 4.997807075247146e-05,
+      "loss": 0.5035,
+      "num_input_tokens_seen": 1242264,
+      "step": 16
+    },
+    {
+      "epoch": 0.2824506749740395,
+      "grad_norm": 0.04751162976026535,
+      "learning_rate": 4.997524440202469e-05,
+      "loss": 0.4354,
+      "num_input_tokens_seen": 1325904,
+      "step": 17
+    },
+    {
+      "epoch": 0.29906542056074764,
+      "grad_norm": 0.06726882606744766,
+      "learning_rate": 4.9972246874049254e-05,
+      "loss": 0.5439,
+      "num_input_tokens_seen": 1385632,
+      "step": 18
+    },
+    {
+      "epoch": 0.31568016614745587,
+      "grad_norm": 0.05245920270681381,
+      "learning_rate": 4.996907818908987e-05,
+      "loss": 0.3727,
+      "num_input_tokens_seen": 1470632,
+      "step": 19
+    },
+    {
+      "epoch": 0.3322949117341641,
+      "grad_norm": 0.05745376646518707,
+      "learning_rate": 4.996573836886435e-05,
+      "loss": 0.4894,
+      "num_input_tokens_seen": 1547536,
+      "step": 20
+    },
+    {
+      "epoch": 0.34890965732087226,
+      "grad_norm": 0.056607529520988464,
+      "learning_rate": 4.9962227436263453e-05,
+      "loss": 0.3846,
+      "num_input_tokens_seen": 1615528,
+      "step": 21
+    },
+    {
+      "epoch": 0.3655244029075805,
+      "grad_norm": 0.06150667741894722,
+      "learning_rate": 4.995854541535071e-05,
+      "loss": 0.4362,
+      "num_input_tokens_seen": 1694352,
+      "step": 22
+    },
+    {
+      "epoch": 0.3821391484942887,
+      "grad_norm": 0.056484442204236984,
+      "learning_rate": 4.9954692331362294e-05,
+      "loss": 0.4438,
+      "num_input_tokens_seen": 1753776,
+      "step": 23
+    },
+    {
+      "epoch": 0.3987538940809969,
+      "grad_norm": 0.0704159140586853,
+      "learning_rate": 4.995066821070679e-05,
+      "loss": 0.4496,
+      "num_input_tokens_seen": 1809048,
+      "step": 24
+    },
+    {
+      "epoch": 0.4153686396677051,
+      "grad_norm": 0.06202029809355736,
+      "learning_rate": 4.994647308096509e-05,
+      "loss": 0.5096,
+      "num_input_tokens_seen": 1884264,
+      "step": 25
+    },
+    {
+      "epoch": 0.43198338525441327,
+      "grad_norm": 0.04237145930528641,
+      "learning_rate": 4.994210697089014e-05,
+      "loss": 0.3722,
+      "num_input_tokens_seen": 1981704,
+      "step": 26
+    },
+    {
+      "epoch": 0.4485981308411215,
+      "grad_norm": 0.06920398026704788,
+      "learning_rate": 4.9937569910406756e-05,
+      "loss": 0.4103,
+      "num_input_tokens_seen": 2044144,
+      "step": 27
+    },
+    {
+      "epoch": 0.4652128764278297,
+      "grad_norm": 0.062432270497083664,
+      "learning_rate": 4.9932861930611454e-05,
+      "loss": 0.357,
+      "num_input_tokens_seen": 2107584,
+      "step": 28
+    },
+    {
+      "epoch": 0.4818276220145379,
+      "grad_norm": 0.06791180372238159,
+      "learning_rate": 4.9927983063772196e-05,
+      "loss": 0.3889,
+      "num_input_tokens_seen": 2169248,
+      "step": 29
+    },
+    {
+      "epoch": 0.4984423676012461,
+      "grad_norm": 0.07219590991735458,
+      "learning_rate": 4.99229333433282e-05,
+      "loss": 0.3543,
+      "num_input_tokens_seen": 2230344,
+      "step": 30
+    },
+    {
+      "epoch": 0.5150571131879543,
+      "grad_norm": 0.0647474005818367,
+      "learning_rate": 4.9917712803889674e-05,
+      "loss": 0.3453,
+      "num_input_tokens_seen": 2302368,
+      "step": 31
+    },
+    {
+      "epoch": 0.5316718587746625,
+      "grad_norm": 0.07434642314910889,
+      "learning_rate": 4.991232148123761e-05,
+      "loss": 0.435,
+      "num_input_tokens_seen": 2369984,
+      "step": 32
+    },
+    {
+      "epoch": 0.5482866043613707,
+      "grad_norm": 0.05302443355321884,
+      "learning_rate": 4.990675941232353e-05,
+      "loss": 0.3981,
+      "num_input_tokens_seen": 2453032,
+      "step": 33
+    },
+    {
+      "epoch": 0.564901349948079,
+      "grad_norm": 0.053745292127132416,
+      "learning_rate": 4.990102663526924e-05,
+      "loss": 0.3755,
+      "num_input_tokens_seen": 2527464,
+      "step": 34
+    },
+    {
+      "epoch": 0.5815160955347871,
+      "grad_norm": 0.06717613339424133,
+      "learning_rate": 4.989512318936655e-05,
+      "loss": 0.3699,
+      "num_input_tokens_seen": 2597032,
+      "step": 35
+    },
+    {
+      "epoch": 0.5981308411214953,
+      "grad_norm": 0.071847103536129,
+      "learning_rate": 4.9889049115077005e-05,
+      "loss": 0.3705,
+      "num_input_tokens_seen": 2671704,
+      "step": 36
+    },
+    {
+      "epoch": 0.6147455867082036,
+      "grad_norm": 0.0460306741297245,
+      "learning_rate": 4.988280445403164e-05,
+      "loss": 0.3797,
+      "num_input_tokens_seen": 2767640,
+      "step": 37
+    },
+    {
+      "epoch": 0.6313603322949117,
+      "grad_norm": 0.053273387253284454,
+      "learning_rate": 4.987638924903067e-05,
+      "loss": 0.3799,
+      "num_input_tokens_seen": 2843720,
+      "step": 38
+    },
+    {
+      "epoch": 0.6479750778816199,
+      "grad_norm": 0.05600422993302345,
+      "learning_rate": 4.9869803544043166e-05,
+      "loss": 0.2866,
+      "num_input_tokens_seen": 2921472,
+      "step": 39
+    },
+    {
+      "epoch": 0.6645898234683282,
+      "grad_norm": 0.06414052098989487,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.4115,
+      "num_input_tokens_seen": 2998400,
+      "step": 40
+    },
+    {
+      "epoch": 0.6812045690550363,
+      "grad_norm": 0.09214208275079727,
+      "learning_rate": 4.985612081582764e-05,
+      "loss": 0.3804,
+      "num_input_tokens_seen": 3059648,
+      "step": 41
+    },
+    {
+      "epoch": 0.6978193146417445,
+      "grad_norm": 0.0555964931845665,
+      "learning_rate": 4.98490238863795e-05,
+      "loss": 0.3121,
+      "num_input_tokens_seen": 3140184,
+      "step": 42
+    },
+    {
+      "epoch": 0.7144340602284528,
+      "grad_norm": 0.06256969273090363,
+      "learning_rate": 4.984175664450397e-05,
+      "loss": 0.3271,
+      "num_input_tokens_seen": 3207184,
+      "step": 43
+    },
+    {
+      "epoch": 0.731048805815161,
+      "grad_norm": 0.0543232187628746,
+      "learning_rate": 4.983431914000991e-05,
+      "loss": 0.364,
+      "num_input_tokens_seen": 3292344,
+      "step": 44
+    },
+    {
+      "epoch": 0.7476635514018691,
+      "grad_norm": 0.06077824532985687,
+      "learning_rate": 4.982671142387316e-05,
+      "loss": 0.3894,
+      "num_input_tokens_seen": 3365384,
+      "step": 45
+    },
+    {
+      "epoch": 0.7642782969885774,
+      "grad_norm": 0.06091070920228958,
+      "learning_rate": 4.981893354823614e-05,
+      "loss": 0.3354,
+      "num_input_tokens_seen": 3440720,
+      "step": 46
+    },
+    {
+      "epoch": 0.7808930425752856,
+      "grad_norm": 0.054153311997652054,
+      "learning_rate": 4.9810985566407544e-05,
+      "loss": 0.3058,
+      "num_input_tokens_seen": 3533576,
+      "step": 47
+    },
+    {
+      "epoch": 0.7975077881619937,
+      "grad_norm": 0.06662417948246002,
+      "learning_rate": 4.980286753286195e-05,
+      "loss": 0.4658,
+      "num_input_tokens_seen": 3599744,
+      "step": 48
+    },
+    {
+      "epoch": 0.814122533748702,
+      "grad_norm": 0.05790851265192032,
+      "learning_rate": 4.979457950323945e-05,
+      "loss": 0.3647,
+      "num_input_tokens_seen": 3689520,
+      "step": 49
+    },
+    {
+      "epoch": 0.8307372793354102,
+      "grad_norm": 0.10742159187793732,
+      "learning_rate": 4.9786121534345265e-05,
+      "loss": 0.343,
+      "num_input_tokens_seen": 3751808,
+      "step": 50
+    },
+    {
+      "epoch": 0.8473520249221184,
+      "grad_norm": 0.05565556138753891,
+      "learning_rate": 4.9777493684149375e-05,
+      "loss": 0.3317,
+      "num_input_tokens_seen": 3839096,
+      "step": 51
+    },
+    {
+      "epoch": 0.8639667705088265,
+      "grad_norm": 0.05752381682395935,
+      "learning_rate": 4.976869601178609e-05,
+      "loss": 0.38,
+      "num_input_tokens_seen": 3919824,
+      "step": 52
+    },
+    {
+      "epoch": 0.8805815160955348,
+      "grad_norm": 0.06406434625387192,
+      "learning_rate": 4.975972857755369e-05,
+      "loss": 0.2676,
+      "num_input_tokens_seen": 3989312,
+      "step": 53
+    },
+    {
+      "epoch": 0.897196261682243,
+      "grad_norm": 0.0653691440820694,
+      "learning_rate": 4.975059144291394e-05,
+      "loss": 0.3516,
+      "num_input_tokens_seen": 4060528,
+      "step": 54
+    },
+    {
+      "epoch": 0.9138110072689511,
+      "grad_norm": 0.06272953748703003,
+      "learning_rate": 4.974128467049176e-05,
+      "loss": 0.3004,
+      "num_input_tokens_seen": 4129368,
+      "step": 55
+    },
+    {
+      "epoch": 0.9304257528556594,
+      "grad_norm": 0.08054930716753006,
+      "learning_rate": 4.9731808324074717e-05,
+      "loss": 0.3009,
+      "num_input_tokens_seen": 4175208,
+      "step": 56
+    },
+    {
+      "epoch": 0.9470404984423676,
+      "grad_norm": 0.07523038238286972,
+      "learning_rate": 4.972216246861262e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 4218096,
+      "step": 57
+    },
+    {
+      "epoch": 0.9636552440290758,
+      "grad_norm": 0.07347433269023895,
+      "learning_rate": 4.971234717021709e-05,
+      "loss": 0.3321,
+      "num_input_tokens_seen": 4275968,
+      "step": 58
+    },
+    {
+      "epoch": 0.980269989615784,
+      "grad_norm": 0.05830248445272446,
+      "learning_rate": 4.9702362496161085e-05,
+      "loss": 0.2881,
+      "num_input_tokens_seen": 4346616,
+      "step": 59
+    },
+    {
+      "epoch": 0.9968847352024922,
+      "grad_norm": 0.061629410833120346,
+      "learning_rate": 4.9692208514878444e-05,
+      "loss": 0.2993,
+      "num_input_tokens_seen": 4425064,
+      "step": 60
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.13380740582942963,
+      "learning_rate": 4.968188529596342e-05,
+      "loss": 0.2511,
+      "num_input_tokens_seen": 4435328,
+      "step": 61
+    },
+    {
+      "epoch": 1.0166147455867083,
+      "grad_norm": 0.0726238414645195,
+      "learning_rate": 4.9671392910170185e-05,
+      "loss": 0.3127,
+      "num_input_tokens_seen": 4500104,
+      "step": 62
+    },
+    {
+      "epoch": 1.0332294911734163,
+      "grad_norm": 0.05980083718895912,
+      "learning_rate": 4.966073142941239e-05,
+      "loss": 0.3601,
+      "num_input_tokens_seen": 4581976,
+      "step": 63
+    },
+    {
+      "epoch": 1.0498442367601246,
+      "grad_norm": 0.06445376574993134,
+      "learning_rate": 4.964990092676263e-05,
+      "loss": 0.3049,
+      "num_input_tokens_seen": 4652160,
+      "step": 64
+    },
+    {
+      "epoch": 1.066458982346833,
+      "grad_norm": 0.07824505120515823,
+      "learning_rate": 4.9638901476451946e-05,
+      "loss": 0.3099,
+      "num_input_tokens_seen": 4709368,
+      "step": 65
+    },
+    {
+      "epoch": 1.083073727933541,
+      "grad_norm": 0.058268457651138306,
+      "learning_rate": 4.962773315386935e-05,
+      "loss": 0.3273,
+      "num_input_tokens_seen": 4798256,
+      "step": 66
+    },
+    {
+      "epoch": 1.0996884735202492,
+      "grad_norm": 0.07069691270589828,
+      "learning_rate": 4.961639603556127e-05,
+      "loss": 0.282,
+      "num_input_tokens_seen": 4859200,
+      "step": 67
+    },
+    {
+      "epoch": 1.1163032191069575,
+      "grad_norm": 0.0775996670126915,
+      "learning_rate": 4.960489019923105e-05,
+      "loss": 0.3642,
+      "num_input_tokens_seen": 4925992,
+      "step": 68
+    },
+    {
+      "epoch": 1.1329179646936656,
+      "grad_norm": 0.07044171541929245,
+      "learning_rate": 4.9593215723738404e-05,
+      "loss": 0.2896,
+      "num_input_tokens_seen": 4998808,
+      "step": 69
+    },
+    {
+      "epoch": 1.1495327102803738,
+      "grad_norm": 0.05971802771091461,
+      "learning_rate": 4.958137268909887e-05,
+      "loss": 0.2578,
+      "num_input_tokens_seen": 5089672,
+      "step": 70
+    },
+    {
+      "epoch": 1.1661474558670821,
+      "grad_norm": 0.07145556062459946,
+      "learning_rate": 4.9569361176483286e-05,
+      "loss": 0.3243,
+      "num_input_tokens_seen": 5166744,
+      "step": 71
+    },
+    {
+      "epoch": 1.1827622014537902,
+      "grad_norm": 0.07455787807703018,
+      "learning_rate": 4.9557181268217227e-05,
+      "loss": 0.3949,
+      "num_input_tokens_seen": 5228264,
+      "step": 72
+    },
+    {
+      "epoch": 1.1993769470404985,
+      "grad_norm": 0.055582575500011444,
+      "learning_rate": 4.9544833047780394e-05,
+      "loss": 0.2877,
+      "num_input_tokens_seen": 5338224,
+      "step": 73
+    },
+    {
+      "epoch": 1.2159916926272065,
+      "grad_norm": 0.07675391435623169,
+      "learning_rate": 4.9532316599806124e-05,
+      "loss": 0.3152,
+      "num_input_tokens_seen": 5399848,
+      "step": 74
+    },
+    {
+      "epoch": 1.2326064382139148,
+      "grad_norm": 0.08048644661903381,
+      "learning_rate": 4.951963201008076e-05,
+      "loss": 0.2976,
+      "num_input_tokens_seen": 5468624,
+      "step": 75
+    },
+    {
+      "epoch": 1.249221183800623,
+      "grad_norm": 0.07579060643911362,
+      "learning_rate": 4.9506779365543046e-05,
+      "loss": 0.2982,
+      "num_input_tokens_seen": 5536776,
+      "step": 76
+    },
+    {
+      "epoch": 1.2658359293873311,
+      "grad_norm": 0.07828006893396378,
+      "learning_rate": 4.949375875428357e-05,
+      "loss": 0.3272,
+      "num_input_tokens_seen": 5609296,
+      "step": 77
+    },
+    {
+      "epoch": 1.2824506749740394,
+      "grad_norm": 0.08079098165035248,
+      "learning_rate": 4.9480570265544144e-05,
+      "loss": 0.2768,
+      "num_input_tokens_seen": 5663824,
+      "step": 78
+    },
+    {
+      "epoch": 1.2990654205607477,
+      "grad_norm": 0.07579358667135239,
+      "learning_rate": 4.94672139897172e-05,
+      "loss": 0.318,
+      "num_input_tokens_seen": 5742032,
+      "step": 79
+    },
+    {
+      "epoch": 1.3156801661474558,
+      "grad_norm": 0.07588379085063934,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.3007,
+      "num_input_tokens_seen": 5816864,
+      "step": 80
+    },
+    {
+      "epoch": 1.332294911734164,
+      "grad_norm": 0.08709035068750381,
+      "learning_rate": 4.943999844411977e-05,
+      "loss": 0.2797,
+      "num_input_tokens_seen": 5881624,
+      "step": 81
+    },
+    {
+      "epoch": 1.3489096573208723,
+      "grad_norm": 0.05975884944200516,
+      "learning_rate": 4.94261393608816e-05,
+      "loss": 0.2591,
+      "num_input_tokens_seen": 5970272,
+      "step": 82
+    },
+    {
+      "epoch": 1.3655244029075804,
+      "grad_norm": 0.07372818142175674,
+      "learning_rate": 4.941211286361922e-05,
+      "loss": 0.2687,
+      "num_input_tokens_seen": 6058752,
+      "step": 83
+    },
+    {
+      "epoch": 1.3821391484942886,
+      "grad_norm": 0.09071576595306396,
+      "learning_rate": 4.939791904846869e-05,
+      "loss": 0.2979,
+      "num_input_tokens_seen": 6120064,
+      "step": 84
+    },
+    {
+      "epoch": 1.398753894080997,
+      "grad_norm": 0.0849960595369339,
+      "learning_rate": 4.938355801271282e-05,
+      "loss": 0.2927,
+      "num_input_tokens_seen": 6182072,
+      "step": 85
+    },
+    {
+      "epoch": 1.415368639667705,
+      "grad_norm": 0.08258760720491409,
+      "learning_rate": 4.936902985478055e-05,
+      "loss": 0.295,
+      "num_input_tokens_seen": 6269680,
+      "step": 86
+    },
+    {
+      "epoch": 1.4319833852544133,
+      "grad_norm": 0.0851503536105156,
+      "learning_rate": 4.935433467424624e-05,
+      "loss": 0.2925,
+      "num_input_tokens_seen": 6347424,
+      "step": 87
+    },
+    {
+      "epoch": 1.4485981308411215,
+      "grad_norm": 0.08852345496416092,
+      "learning_rate": 4.933947257182901e-05,
+      "loss": 0.3153,
+      "num_input_tokens_seen": 6412584,
+      "step": 88
+    },
+    {
+      "epoch": 1.4652128764278296,
+      "grad_norm": 0.08184897154569626,
+      "learning_rate": 4.932444364939205e-05,
+      "loss": 0.292,
+      "num_input_tokens_seen": 6482728,
+      "step": 89
+    },
+    {
+      "epoch": 1.4818276220145379,
+      "grad_norm": 0.08270515501499176,
+      "learning_rate": 4.9309248009941914e-05,
+      "loss": 0.3472,
+      "num_input_tokens_seen": 6562104,
+      "step": 90
+    },
+    {
+      "epoch": 1.4984423676012462,
+      "grad_norm": 0.07407747954130173,
+      "learning_rate": 4.929388575762782e-05,
+      "loss": 0.2995,
+      "num_input_tokens_seen": 6656552,
+      "step": 91
+    },
+    {
+      "epoch": 1.5150571131879542,
+      "grad_norm": 0.08710360527038574,
+      "learning_rate": 4.9278356997740904e-05,
+      "loss": 0.2549,
+      "num_input_tokens_seen": 6714184,
+      "step": 92
+    },
+    {
+      "epoch": 1.5316718587746625,
+      "grad_norm": 0.0773790255188942,
+      "learning_rate": 4.9262661836713564e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 6793552,
+      "step": 93
+    },
+    {
+      "epoch": 1.5482866043613708,
+      "grad_norm": 0.1002134457230568,
+      "learning_rate": 4.924680038211867e-05,
+      "loss": 0.2876,
+      "num_input_tokens_seen": 6865256,
+      "step": 94
+    },
+    {
+      "epoch": 1.5649013499480788,
+      "grad_norm": 0.09670394659042358,
+      "learning_rate": 4.9230772742668866e-05,
+      "loss": 0.2846,
+      "num_input_tokens_seen": 6931152,
+      "step": 95
+    },
+    {
+      "epoch": 1.5815160955347871,
+      "grad_norm": 0.08910100907087326,
+      "learning_rate": 4.9214579028215776e-05,
+      "loss": 0.2944,
+      "num_input_tokens_seen": 6998408,
+      "step": 96
+    },
+    {
+      "epoch": 1.5981308411214954,
+      "grad_norm": 0.09202459454536438,
+      "learning_rate": 4.919821934974933e-05,
+      "loss": 0.251,
+      "num_input_tokens_seen": 7053008,
+      "step": 97
+    },
+    {
+      "epoch": 1.6147455867082035,
+      "grad_norm": 0.10218881815671921,
+      "learning_rate": 4.918169381939692e-05,
+      "loss": 0.2851,
+      "num_input_tokens_seen": 7106440,
+      "step": 98
+    },
+    {
+      "epoch": 1.6313603322949117,
+      "grad_norm": 0.09290914982557297,
+      "learning_rate": 4.916500255042268e-05,
+      "loss": 0.2959,
+      "num_input_tokens_seen": 7167032,
+      "step": 99
+    },
+    {
+      "epoch": 1.64797507788162,
+      "grad_norm": 0.07791033387184143,
+      "learning_rate": 4.914814565722671e-05,
+      "loss": 0.2481,
+      "num_input_tokens_seen": 7245720,
+      "step": 100
+    },
+    {
+      "epoch": 1.664589823468328,
+      "grad_norm": 0.08885534107685089,
+      "learning_rate": 4.913112325534426e-05,
+      "loss": 0.3168,
+      "num_input_tokens_seen": 7326320,
+      "step": 101
+    },
+    {
+      "epoch": 1.6812045690550363,
+      "grad_norm": 0.08569750934839249,
+      "learning_rate": 4.9113935461444955e-05,
+      "loss": 0.2805,
+      "num_input_tokens_seen": 7442232,
+      "step": 102
+    },
+    {
+      "epoch": 1.6978193146417446,
+      "grad_norm": 0.1112508773803711,
+      "learning_rate": 4.9096582393332025e-05,
+      "loss": 0.2675,
+      "num_input_tokens_seen": 7502496,
+      "step": 103
+    },
+    {
+      "epoch": 1.7144340602284527,
+      "grad_norm": 0.09654372185468674,
+      "learning_rate": 4.907906416994146e-05,
+      "loss": 0.3038,
+      "num_input_tokens_seen": 7566496,
+      "step": 104
+    },
+    {
+      "epoch": 1.731048805815161,
+      "grad_norm": 0.10022995620965958,
+      "learning_rate": 4.906138091134118e-05,
+      "loss": 0.3639,
+      "num_input_tokens_seen": 7629056,
+      "step": 105
+    },
+    {
+      "epoch": 1.7476635514018692,
+      "grad_norm": 0.08336564153432846,
+      "learning_rate": 4.9043532738730284e-05,
+      "loss": 0.2944,
+      "num_input_tokens_seen": 7706096,
+      "step": 106
+    },
+    {
+      "epoch": 1.7642782969885773,
+      "grad_norm": 0.08539658784866333,
+      "learning_rate": 4.9025519774438136e-05,
+      "loss": 0.2392,
+      "num_input_tokens_seen": 7780072,
+      "step": 107
+    },
+    {
+      "epoch": 1.7808930425752856,
+      "grad_norm": 0.09139693528413773,
+      "learning_rate": 4.900734214192358e-05,
+      "loss": 0.2685,
+      "num_input_tokens_seen": 7857712,
+      "step": 108
+    },
+    {
+      "epoch": 1.7975077881619939,
+      "grad_norm": 0.1043916717171669,
+      "learning_rate": 4.898899996577407e-05,
+      "loss": 0.2513,
+      "num_input_tokens_seen": 7916832,
+      "step": 109
+    },
+    {
+      "epoch": 1.814122533748702,
+      "grad_norm": 0.09203662723302841,
+      "learning_rate": 4.8970493371704826e-05,
+      "loss": 0.2974,
+      "num_input_tokens_seen": 7993056,
+      "step": 110
+    },
+    {
+      "epoch": 1.8307372793354102,
+      "grad_norm": 0.09319474548101425,
+      "learning_rate": 4.8951822486557986e-05,
+      "loss": 0.3096,
+      "num_input_tokens_seen": 8090056,
+      "step": 111
+    },
+    {
+      "epoch": 1.8473520249221185,
+      "grad_norm": 0.10193445533514023,
+      "learning_rate": 4.893298743830168e-05,
+      "loss": 0.2633,
+      "num_input_tokens_seen": 8164808,
+      "step": 112
+    },
+    {
+      "epoch": 1.8639667705088265,
+      "grad_norm": 0.11407948285341263,
+      "learning_rate": 4.891398835602925e-05,
+      "loss": 0.2584,
+      "num_input_tokens_seen": 8223568,
+      "step": 113
+    },
+    {
+      "epoch": 1.8805815160955348,
+      "grad_norm": 0.11977085471153259,
+      "learning_rate": 4.8894825369958255e-05,
+      "loss": 0.2619,
+      "num_input_tokens_seen": 8276160,
+      "step": 114
+    },
+    {
+      "epoch": 1.897196261682243,
+      "grad_norm": 0.10925433784723282,
+      "learning_rate": 4.8875498611429674e-05,
+      "loss": 0.2762,
+      "num_input_tokens_seen": 8354904,
+      "step": 115
+    },
+    {
+      "epoch": 1.9138110072689511,
+      "grad_norm": 0.09673939645290375,
+      "learning_rate": 4.8856008212906925e-05,
+      "loss": 0.3152,
+      "num_input_tokens_seen": 8442584,
+      "step": 116
+    },
+    {
+      "epoch": 1.9304257528556594,
+      "grad_norm": 0.10827789455652237,
+      "learning_rate": 4.8836354307975026e-05,
+      "loss": 0.2759,
+      "num_input_tokens_seen": 8506688,
+      "step": 117
+    },
+    {
+      "epoch": 1.9470404984423677,
+      "grad_norm": 0.08390220254659653,
+      "learning_rate": 4.881653703133966e-05,
+      "loss": 0.2192,
+      "num_input_tokens_seen": 8610712,
+      "step": 118
+    },
+    {
+      "epoch": 1.9636552440290758,
+      "grad_norm": 0.09252211451530457,
+      "learning_rate": 4.87965565188262e-05,
+      "loss": 0.2618,
+      "num_input_tokens_seen": 8692624,
+      "step": 119
+    },
+    {
+      "epoch": 1.980269989615784,
+      "grad_norm": 0.1107102632522583,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.2666,
+      "num_input_tokens_seen": 8772208,
+      "step": 120
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 1200,
+  "num_input_tokens_seen": 8772208,
+  "num_train_epochs": 20,
+  "save_steps": 60,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.400381371991982e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-180/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-180/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-180/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-180/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-180/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 17500,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-240/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-240/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-240/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-240/trainer_state.json

@@ -0,0 +1,1953 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.9470404984423677,
+  "eval_steps": 500,
+  "global_step": 240,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.016614745586708203,
+      "grad_norm": 0.050998032093048096,
+      "learning_rate": 4.999991432639962e-05,
+      "loss": 0.5487,
+      "num_input_tokens_seen": 70408,
+      "step": 1
+    },
+    {
+      "epoch": 0.033229491173416406,
+      "grad_norm": 0.049370743334293365,
+      "learning_rate": 4.999965730618567e-05,
+      "loss": 0.4981,
+      "num_input_tokens_seen": 139640,
+      "step": 2
+    },
+    {
+      "epoch": 0.04984423676012461,
+      "grad_norm": 0.05077400803565979,
+      "learning_rate": 4.9999228941119745e-05,
+      "loss": 0.5505,
+      "num_input_tokens_seen": 223656,
+      "step": 3
+    },
+    {
+      "epoch": 0.06645898234683281,
+      "grad_norm": 0.04397282376885414,
+      "learning_rate": 4.999862923413781e-05,
+      "loss": 0.504,
+      "num_input_tokens_seen": 300688,
+      "step": 4
+    },
+    {
+      "epoch": 0.08307372793354102,
+      "grad_norm": 0.05225864797830582,
+      "learning_rate": 4.999785818935018e-05,
+      "loss": 0.4925,
+      "num_input_tokens_seen": 366368,
+      "step": 5
+    },
+    {
+      "epoch": 0.09968847352024922,
+      "grad_norm": 0.049482282251119614,
+      "learning_rate": 4.999691581204152e-05,
+      "loss": 0.4771,
+      "num_input_tokens_seen": 445808,
+      "step": 6
+    },
+    {
+      "epoch": 0.11630321910695743,
+      "grad_norm": 0.05594080314040184,
+      "learning_rate": 4.9995802108670775e-05,
+      "loss": 0.4986,
+      "num_input_tokens_seen": 522800,
+      "step": 7
+    },
+    {
+      "epoch": 0.13291796469366562,
+      "grad_norm": 0.051852282136678696,
+      "learning_rate": 4.999451708687114e-05,
+      "loss": 0.5171,
+      "num_input_tokens_seen": 599608,
+      "step": 8
+    },
+    {
+      "epoch": 0.14953271028037382,
+      "grad_norm": 0.045517683029174805,
+      "learning_rate": 4.9993060755450015e-05,
+      "loss": 0.5669,
+      "num_input_tokens_seen": 681424,
+      "step": 9
+    },
+    {
+      "epoch": 0.16614745586708204,
+      "grad_norm": 0.044325754046440125,
+      "learning_rate": 4.999143312438893e-05,
+      "loss": 0.4218,
+      "num_input_tokens_seen": 756744,
+      "step": 10
+    },
+    {
+      "epoch": 0.18276220145379024,
+      "grad_norm": 0.04328459873795509,
+      "learning_rate": 4.998963420484349e-05,
+      "loss": 0.434,
+      "num_input_tokens_seen": 842576,
+      "step": 11
+    },
+    {
+      "epoch": 0.19937694704049844,
+      "grad_norm": 0.04725787043571472,
+      "learning_rate": 4.998766400914329e-05,
+      "loss": 0.4287,
+      "num_input_tokens_seen": 917232,
+      "step": 12
+    },
+    {
+      "epoch": 0.21599169262720663,
+      "grad_norm": 0.03806879371404648,
+      "learning_rate": 4.9985522550791825e-05,
+      "loss": 0.3454,
+      "num_input_tokens_seen": 1006800,
+      "step": 13
+    },
+    {
+      "epoch": 0.23260643821391486,
+      "grad_norm": 0.05201176926493645,
+      "learning_rate": 4.998320984446641e-05,
+      "loss": 0.436,
+      "num_input_tokens_seen": 1085824,
+      "step": 14
+    },
+    {
+      "epoch": 0.24922118380062305,
+      "grad_norm": 0.047955628484487534,
+      "learning_rate": 4.9980725906018074e-05,
+      "loss": 0.4625,
+      "num_input_tokens_seen": 1164160,
+      "step": 15
+    },
+    {
+      "epoch": 0.26583592938733125,
+      "grad_norm": 0.05529098957777023,
+      "learning_rate": 4.997807075247146e-05,
+      "loss": 0.5035,
+      "num_input_tokens_seen": 1242264,
+      "step": 16
+    },
+    {
+      "epoch": 0.2824506749740395,
+      "grad_norm": 0.04751162976026535,
+      "learning_rate": 4.997524440202469e-05,
+      "loss": 0.4354,
+      "num_input_tokens_seen": 1325904,
+      "step": 17
+    },
+    {
+      "epoch": 0.29906542056074764,
+      "grad_norm": 0.06726882606744766,
+      "learning_rate": 4.9972246874049254e-05,
+      "loss": 0.5439,
+      "num_input_tokens_seen": 1385632,
+      "step": 18
+    },
+    {
+      "epoch": 0.31568016614745587,
+      "grad_norm": 0.05245920270681381,
+      "learning_rate": 4.996907818908987e-05,
+      "loss": 0.3727,
+      "num_input_tokens_seen": 1470632,
+      "step": 19
+    },
+    {
+      "epoch": 0.3322949117341641,
+      "grad_norm": 0.05745376646518707,
+      "learning_rate": 4.996573836886435e-05,
+      "loss": 0.4894,
+      "num_input_tokens_seen": 1547536,
+      "step": 20
+    },
+    {
+      "epoch": 0.34890965732087226,
+      "grad_norm": 0.056607529520988464,
+      "learning_rate": 4.9962227436263453e-05,
+      "loss": 0.3846,
+      "num_input_tokens_seen": 1615528,
+      "step": 21
+    },
+    {
+      "epoch": 0.3655244029075805,
+      "grad_norm": 0.06150667741894722,
+      "learning_rate": 4.995854541535071e-05,
+      "loss": 0.4362,
+      "num_input_tokens_seen": 1694352,
+      "step": 22
+    },
+    {
+      "epoch": 0.3821391484942887,
+      "grad_norm": 0.056484442204236984,
+      "learning_rate": 4.9954692331362294e-05,
+      "loss": 0.4438,
+      "num_input_tokens_seen": 1753776,
+      "step": 23
+    },
+    {
+      "epoch": 0.3987538940809969,
+      "grad_norm": 0.0704159140586853,
+      "learning_rate": 4.995066821070679e-05,
+      "loss": 0.4496,
+      "num_input_tokens_seen": 1809048,
+      "step": 24
+    },
+    {
+      "epoch": 0.4153686396677051,
+      "grad_norm": 0.06202029809355736,
+      "learning_rate": 4.994647308096509e-05,
+      "loss": 0.5096,
+      "num_input_tokens_seen": 1884264,
+      "step": 25
+    },
+    {
+      "epoch": 0.43198338525441327,
+      "grad_norm": 0.04237145930528641,
+      "learning_rate": 4.994210697089014e-05,
+      "loss": 0.3722,
+      "num_input_tokens_seen": 1981704,
+      "step": 26
+    },
+    {
+      "epoch": 0.4485981308411215,
+      "grad_norm": 0.06920398026704788,
+      "learning_rate": 4.9937569910406756e-05,
+      "loss": 0.4103,
+      "num_input_tokens_seen": 2044144,
+      "step": 27
+    },
+    {
+      "epoch": 0.4652128764278297,
+      "grad_norm": 0.062432270497083664,
+      "learning_rate": 4.9932861930611454e-05,
+      "loss": 0.357,
+      "num_input_tokens_seen": 2107584,
+      "step": 28
+    },
+    {
+      "epoch": 0.4818276220145379,
+      "grad_norm": 0.06791180372238159,
+      "learning_rate": 4.9927983063772196e-05,
+      "loss": 0.3889,
+      "num_input_tokens_seen": 2169248,
+      "step": 29
+    },
+    {
+      "epoch": 0.4984423676012461,
+      "grad_norm": 0.07219590991735458,
+      "learning_rate": 4.99229333433282e-05,
+      "loss": 0.3543,
+      "num_input_tokens_seen": 2230344,
+      "step": 30
+    },
+    {
+      "epoch": 0.5150571131879543,
+      "grad_norm": 0.0647474005818367,
+      "learning_rate": 4.9917712803889674e-05,
+      "loss": 0.3453,
+      "num_input_tokens_seen": 2302368,
+      "step": 31
+    },
+    {
+      "epoch": 0.5316718587746625,
+      "grad_norm": 0.07434642314910889,
+      "learning_rate": 4.991232148123761e-05,
+      "loss": 0.435,
+      "num_input_tokens_seen": 2369984,
+      "step": 32
+    },
+    {
+      "epoch": 0.5482866043613707,
+      "grad_norm": 0.05302443355321884,
+      "learning_rate": 4.990675941232353e-05,
+      "loss": 0.3981,
+      "num_input_tokens_seen": 2453032,
+      "step": 33
+    },
+    {
+      "epoch": 0.564901349948079,
+      "grad_norm": 0.053745292127132416,
+      "learning_rate": 4.990102663526924e-05,
+      "loss": 0.3755,
+      "num_input_tokens_seen": 2527464,
+      "step": 34
+    },
+    {
+      "epoch": 0.5815160955347871,
+      "grad_norm": 0.06717613339424133,
+      "learning_rate": 4.989512318936655e-05,
+      "loss": 0.3699,
+      "num_input_tokens_seen": 2597032,
+      "step": 35
+    },
+    {
+      "epoch": 0.5981308411214953,
+      "grad_norm": 0.071847103536129,
+      "learning_rate": 4.9889049115077005e-05,
+      "loss": 0.3705,
+      "num_input_tokens_seen": 2671704,
+      "step": 36
+    },
+    {
+      "epoch": 0.6147455867082036,
+      "grad_norm": 0.0460306741297245,
+      "learning_rate": 4.988280445403164e-05,
+      "loss": 0.3797,
+      "num_input_tokens_seen": 2767640,
+      "step": 37
+    },
+    {
+      "epoch": 0.6313603322949117,
+      "grad_norm": 0.053273387253284454,
+      "learning_rate": 4.987638924903067e-05,
+      "loss": 0.3799,
+      "num_input_tokens_seen": 2843720,
+      "step": 38
+    },
+    {
+      "epoch": 0.6479750778816199,
+      "grad_norm": 0.05600422993302345,
+      "learning_rate": 4.9869803544043166e-05,
+      "loss": 0.2866,
+      "num_input_tokens_seen": 2921472,
+      "step": 39
+    },
+    {
+      "epoch": 0.6645898234683282,
+      "grad_norm": 0.06414052098989487,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.4115,
+      "num_input_tokens_seen": 2998400,
+      "step": 40
+    },
+    {
+      "epoch": 0.6812045690550363,
+      "grad_norm": 0.09214208275079727,
+      "learning_rate": 4.985612081582764e-05,
+      "loss": 0.3804,
+      "num_input_tokens_seen": 3059648,
+      "step": 41
+    },
+    {
+      "epoch": 0.6978193146417445,
+      "grad_norm": 0.0555964931845665,
+      "learning_rate": 4.98490238863795e-05,
+      "loss": 0.3121,
+      "num_input_tokens_seen": 3140184,
+      "step": 42
+    },
+    {
+      "epoch": 0.7144340602284528,
+      "grad_norm": 0.06256969273090363,
+      "learning_rate": 4.984175664450397e-05,
+      "loss": 0.3271,
+      "num_input_tokens_seen": 3207184,
+      "step": 43
+    },
+    {
+      "epoch": 0.731048805815161,
+      "grad_norm": 0.0543232187628746,
+      "learning_rate": 4.983431914000991e-05,
+      "loss": 0.364,
+      "num_input_tokens_seen": 3292344,
+      "step": 44
+    },
+    {
+      "epoch": 0.7476635514018691,
+      "grad_norm": 0.06077824532985687,
+      "learning_rate": 4.982671142387316e-05,
+      "loss": 0.3894,
+      "num_input_tokens_seen": 3365384,
+      "step": 45
+    },
+    {
+      "epoch": 0.7642782969885774,
+      "grad_norm": 0.06091070920228958,
+      "learning_rate": 4.981893354823614e-05,
+      "loss": 0.3354,
+      "num_input_tokens_seen": 3440720,
+      "step": 46
+    },
+    {
+      "epoch": 0.7808930425752856,
+      "grad_norm": 0.054153311997652054,
+      "learning_rate": 4.9810985566407544e-05,
+      "loss": 0.3058,
+      "num_input_tokens_seen": 3533576,
+      "step": 47
+    },
+    {
+      "epoch": 0.7975077881619937,
+      "grad_norm": 0.06662417948246002,
+      "learning_rate": 4.980286753286195e-05,
+      "loss": 0.4658,
+      "num_input_tokens_seen": 3599744,
+      "step": 48
+    },
+    {
+      "epoch": 0.814122533748702,
+      "grad_norm": 0.05790851265192032,
+      "learning_rate": 4.979457950323945e-05,
+      "loss": 0.3647,
+      "num_input_tokens_seen": 3689520,
+      "step": 49
+    },
+    {
+      "epoch": 0.8307372793354102,
+      "grad_norm": 0.10742159187793732,
+      "learning_rate": 4.9786121534345265e-05,
+      "loss": 0.343,
+      "num_input_tokens_seen": 3751808,
+      "step": 50
+    },
+    {
+      "epoch": 0.8473520249221184,
+      "grad_norm": 0.05565556138753891,
+      "learning_rate": 4.9777493684149375e-05,
+      "loss": 0.3317,
+      "num_input_tokens_seen": 3839096,
+      "step": 51
+    },
+    {
+      "epoch": 0.8639667705088265,
+      "grad_norm": 0.05752381682395935,
+      "learning_rate": 4.976869601178609e-05,
+      "loss": 0.38,
+      "num_input_tokens_seen": 3919824,
+      "step": 52
+    },
+    {
+      "epoch": 0.8805815160955348,
+      "grad_norm": 0.06406434625387192,
+      "learning_rate": 4.975972857755369e-05,
+      "loss": 0.2676,
+      "num_input_tokens_seen": 3989312,
+      "step": 53
+    },
+    {
+      "epoch": 0.897196261682243,
+      "grad_norm": 0.0653691440820694,
+      "learning_rate": 4.975059144291394e-05,
+      "loss": 0.3516,
+      "num_input_tokens_seen": 4060528,
+      "step": 54
+    },
+    {
+      "epoch": 0.9138110072689511,
+      "grad_norm": 0.06272953748703003,
+      "learning_rate": 4.974128467049176e-05,
+      "loss": 0.3004,
+      "num_input_tokens_seen": 4129368,
+      "step": 55
+    },
+    {
+      "epoch": 0.9304257528556594,
+      "grad_norm": 0.08054930716753006,
+      "learning_rate": 4.9731808324074717e-05,
+      "loss": 0.3009,
+      "num_input_tokens_seen": 4175208,
+      "step": 56
+    },
+    {
+      "epoch": 0.9470404984423676,
+      "grad_norm": 0.07523038238286972,
+      "learning_rate": 4.972216246861262e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 4218096,
+      "step": 57
+    },
+    {
+      "epoch": 0.9636552440290758,
+      "grad_norm": 0.07347433269023895,
+      "learning_rate": 4.971234717021709e-05,
+      "loss": 0.3321,
+      "num_input_tokens_seen": 4275968,
+      "step": 58
+    },
+    {
+      "epoch": 0.980269989615784,
+      "grad_norm": 0.05830248445272446,
+      "learning_rate": 4.9702362496161085e-05,
+      "loss": 0.2881,
+      "num_input_tokens_seen": 4346616,
+      "step": 59
+    },
+    {
+      "epoch": 0.9968847352024922,
+      "grad_norm": 0.061629410833120346,
+      "learning_rate": 4.9692208514878444e-05,
+      "loss": 0.2993,
+      "num_input_tokens_seen": 4425064,
+      "step": 60
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.13380740582942963,
+      "learning_rate": 4.968188529596342e-05,
+      "loss": 0.2511,
+      "num_input_tokens_seen": 4435328,
+      "step": 61
+    },
+    {
+      "epoch": 1.0166147455867083,
+      "grad_norm": 0.0726238414645195,
+      "learning_rate": 4.9671392910170185e-05,
+      "loss": 0.3127,
+      "num_input_tokens_seen": 4500104,
+      "step": 62
+    },
+    {
+      "epoch": 1.0332294911734163,
+      "grad_norm": 0.05980083718895912,
+      "learning_rate": 4.966073142941239e-05,
+      "loss": 0.3601,
+      "num_input_tokens_seen": 4581976,
+      "step": 63
+    },
+    {
+      "epoch": 1.0498442367601246,
+      "grad_norm": 0.06445376574993134,
+      "learning_rate": 4.964990092676263e-05,
+      "loss": 0.3049,
+      "num_input_tokens_seen": 4652160,
+      "step": 64
+    },
+    {
+      "epoch": 1.066458982346833,
+      "grad_norm": 0.07824505120515823,
+      "learning_rate": 4.9638901476451946e-05,
+      "loss": 0.3099,
+      "num_input_tokens_seen": 4709368,
+      "step": 65
+    },
+    {
+      "epoch": 1.083073727933541,
+      "grad_norm": 0.058268457651138306,
+      "learning_rate": 4.962773315386935e-05,
+      "loss": 0.3273,
+      "num_input_tokens_seen": 4798256,
+      "step": 66
+    },
+    {
+      "epoch": 1.0996884735202492,
+      "grad_norm": 0.07069691270589828,
+      "learning_rate": 4.961639603556127e-05,
+      "loss": 0.282,
+      "num_input_tokens_seen": 4859200,
+      "step": 67
+    },
+    {
+      "epoch": 1.1163032191069575,
+      "grad_norm": 0.0775996670126915,
+      "learning_rate": 4.960489019923105e-05,
+      "loss": 0.3642,
+      "num_input_tokens_seen": 4925992,
+      "step": 68
+    },
+    {
+      "epoch": 1.1329179646936656,
+      "grad_norm": 0.07044171541929245,
+      "learning_rate": 4.9593215723738404e-05,
+      "loss": 0.2896,
+      "num_input_tokens_seen": 4998808,
+      "step": 69
+    },
+    {
+      "epoch": 1.1495327102803738,
+      "grad_norm": 0.05971802771091461,
+      "learning_rate": 4.958137268909887e-05,
+      "loss": 0.2578,
+      "num_input_tokens_seen": 5089672,
+      "step": 70
+    },
+    {
+      "epoch": 1.1661474558670821,
+      "grad_norm": 0.07145556062459946,
+      "learning_rate": 4.9569361176483286e-05,
+      "loss": 0.3243,
+      "num_input_tokens_seen": 5166744,
+      "step": 71
+    },
+    {
+      "epoch": 1.1827622014537902,
+      "grad_norm": 0.07455787807703018,
+      "learning_rate": 4.9557181268217227e-05,
+      "loss": 0.3949,
+      "num_input_tokens_seen": 5228264,
+      "step": 72
+    },
+    {
+      "epoch": 1.1993769470404985,
+      "grad_norm": 0.055582575500011444,
+      "learning_rate": 4.9544833047780394e-05,
+      "loss": 0.2877,
+      "num_input_tokens_seen": 5338224,
+      "step": 73
+    },
+    {
+      "epoch": 1.2159916926272065,
+      "grad_norm": 0.07675391435623169,
+      "learning_rate": 4.9532316599806124e-05,
+      "loss": 0.3152,
+      "num_input_tokens_seen": 5399848,
+      "step": 74
+    },
+    {
+      "epoch": 1.2326064382139148,
+      "grad_norm": 0.08048644661903381,
+      "learning_rate": 4.951963201008076e-05,
+      "loss": 0.2976,
+      "num_input_tokens_seen": 5468624,
+      "step": 75
+    },
+    {
+      "epoch": 1.249221183800623,
+      "grad_norm": 0.07579060643911362,
+      "learning_rate": 4.9506779365543046e-05,
+      "loss": 0.2982,
+      "num_input_tokens_seen": 5536776,
+      "step": 76
+    },
+    {
+      "epoch": 1.2658359293873311,
+      "grad_norm": 0.07828006893396378,
+      "learning_rate": 4.949375875428357e-05,
+      "loss": 0.3272,
+      "num_input_tokens_seen": 5609296,
+      "step": 77
+    },
+    {
+      "epoch": 1.2824506749740394,
+      "grad_norm": 0.08079098165035248,
+      "learning_rate": 4.9480570265544144e-05,
+      "loss": 0.2768,
+      "num_input_tokens_seen": 5663824,
+      "step": 78
+    },
+    {
+      "epoch": 1.2990654205607477,
+      "grad_norm": 0.07579358667135239,
+      "learning_rate": 4.94672139897172e-05,
+      "loss": 0.318,
+      "num_input_tokens_seen": 5742032,
+      "step": 79
+    },
+    {
+      "epoch": 1.3156801661474558,
+      "grad_norm": 0.07588379085063934,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.3007,
+      "num_input_tokens_seen": 5816864,
+      "step": 80
+    },
+    {
+      "epoch": 1.332294911734164,
+      "grad_norm": 0.08709035068750381,
+      "learning_rate": 4.943999844411977e-05,
+      "loss": 0.2797,
+      "num_input_tokens_seen": 5881624,
+      "step": 81
+    },
+    {
+      "epoch": 1.3489096573208723,
+      "grad_norm": 0.05975884944200516,
+      "learning_rate": 4.94261393608816e-05,
+      "loss": 0.2591,
+      "num_input_tokens_seen": 5970272,
+      "step": 82
+    },
+    {
+      "epoch": 1.3655244029075804,
+      "grad_norm": 0.07372818142175674,
+      "learning_rate": 4.941211286361922e-05,
+      "loss": 0.2687,
+      "num_input_tokens_seen": 6058752,
+      "step": 83
+    },
+    {
+      "epoch": 1.3821391484942886,
+      "grad_norm": 0.09071576595306396,
+      "learning_rate": 4.939791904846869e-05,
+      "loss": 0.2979,
+      "num_input_tokens_seen": 6120064,
+      "step": 84
+    },
+    {
+      "epoch": 1.398753894080997,
+      "grad_norm": 0.0849960595369339,
+      "learning_rate": 4.938355801271282e-05,
+      "loss": 0.2927,
+      "num_input_tokens_seen": 6182072,
+      "step": 85
+    },
+    {
+      "epoch": 1.415368639667705,
+      "grad_norm": 0.08258760720491409,
+      "learning_rate": 4.936902985478055e-05,
+      "loss": 0.295,
+      "num_input_tokens_seen": 6269680,
+      "step": 86
+    },
+    {
+      "epoch": 1.4319833852544133,
+      "grad_norm": 0.0851503536105156,
+      "learning_rate": 4.935433467424624e-05,
+      "loss": 0.2925,
+      "num_input_tokens_seen": 6347424,
+      "step": 87
+    },
+    {
+      "epoch": 1.4485981308411215,
+      "grad_norm": 0.08852345496416092,
+      "learning_rate": 4.933947257182901e-05,
+      "loss": 0.3153,
+      "num_input_tokens_seen": 6412584,
+      "step": 88
+    },
+    {
+      "epoch": 1.4652128764278296,
+      "grad_norm": 0.08184897154569626,
+      "learning_rate": 4.932444364939205e-05,
+      "loss": 0.292,
+      "num_input_tokens_seen": 6482728,
+      "step": 89
+    },
+    {
+      "epoch": 1.4818276220145379,
+      "grad_norm": 0.08270515501499176,
+      "learning_rate": 4.9309248009941914e-05,
+      "loss": 0.3472,
+      "num_input_tokens_seen": 6562104,
+      "step": 90
+    },
+    {
+      "epoch": 1.4984423676012462,
+      "grad_norm": 0.07407747954130173,
+      "learning_rate": 4.929388575762782e-05,
+      "loss": 0.2995,
+      "num_input_tokens_seen": 6656552,
+      "step": 91
+    },
+    {
+      "epoch": 1.5150571131879542,
+      "grad_norm": 0.08710360527038574,
+      "learning_rate": 4.9278356997740904e-05,
+      "loss": 0.2549,
+      "num_input_tokens_seen": 6714184,
+      "step": 92
+    },
+    {
+      "epoch": 1.5316718587746625,
+      "grad_norm": 0.0773790255188942,
+      "learning_rate": 4.9262661836713564e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 6793552,
+      "step": 93
+    },
+    {
+      "epoch": 1.5482866043613708,
+      "grad_norm": 0.1002134457230568,
+      "learning_rate": 4.924680038211867e-05,
+      "loss": 0.2876,
+      "num_input_tokens_seen": 6865256,
+      "step": 94
+    },
+    {
+      "epoch": 1.5649013499480788,
+      "grad_norm": 0.09670394659042358,
+      "learning_rate": 4.9230772742668866e-05,
+      "loss": 0.2846,
+      "num_input_tokens_seen": 6931152,
+      "step": 95
+    },
+    {
+      "epoch": 1.5815160955347871,
+      "grad_norm": 0.08910100907087326,
+      "learning_rate": 4.9214579028215776e-05,
+      "loss": 0.2944,
+      "num_input_tokens_seen": 6998408,
+      "step": 96
+    },
+    {
+      "epoch": 1.5981308411214954,
+      "grad_norm": 0.09202459454536438,
+      "learning_rate": 4.919821934974933e-05,
+      "loss": 0.251,
+      "num_input_tokens_seen": 7053008,
+      "step": 97
+    },
+    {
+      "epoch": 1.6147455867082035,
+      "grad_norm": 0.10218881815671921,
+      "learning_rate": 4.918169381939692e-05,
+      "loss": 0.2851,
+      "num_input_tokens_seen": 7106440,
+      "step": 98
+    },
+    {
+      "epoch": 1.6313603322949117,
+      "grad_norm": 0.09290914982557297,
+      "learning_rate": 4.916500255042268e-05,
+      "loss": 0.2959,
+      "num_input_tokens_seen": 7167032,
+      "step": 99
+    },
+    {
+      "epoch": 1.64797507788162,
+      "grad_norm": 0.07791033387184143,
+      "learning_rate": 4.914814565722671e-05,
+      "loss": 0.2481,
+      "num_input_tokens_seen": 7245720,
+      "step": 100
+    },
+    {
+      "epoch": 1.664589823468328,
+      "grad_norm": 0.08885534107685089,
+      "learning_rate": 4.913112325534426e-05,
+      "loss": 0.3168,
+      "num_input_tokens_seen": 7326320,
+      "step": 101
+    },
+    {
+      "epoch": 1.6812045690550363,
+      "grad_norm": 0.08569750934839249,
+      "learning_rate": 4.9113935461444955e-05,
+      "loss": 0.2805,
+      "num_input_tokens_seen": 7442232,
+      "step": 102
+    },
+    {
+      "epoch": 1.6978193146417446,
+      "grad_norm": 0.1112508773803711,
+      "learning_rate": 4.9096582393332025e-05,
+      "loss": 0.2675,
+      "num_input_tokens_seen": 7502496,
+      "step": 103
+    },
+    {
+      "epoch": 1.7144340602284527,
+      "grad_norm": 0.09654372185468674,
+      "learning_rate": 4.907906416994146e-05,
+      "loss": 0.3038,
+      "num_input_tokens_seen": 7566496,
+      "step": 104
+    },
+    {
+      "epoch": 1.731048805815161,
+      "grad_norm": 0.10022995620965958,
+      "learning_rate": 4.906138091134118e-05,
+      "loss": 0.3639,
+      "num_input_tokens_seen": 7629056,
+      "step": 105
+    },
+    {
+      "epoch": 1.7476635514018692,
+      "grad_norm": 0.08336564153432846,
+      "learning_rate": 4.9043532738730284e-05,
+      "loss": 0.2944,
+      "num_input_tokens_seen": 7706096,
+      "step": 106
+    },
+    {
+      "epoch": 1.7642782969885773,
+      "grad_norm": 0.08539658784866333,
+      "learning_rate": 4.9025519774438136e-05,
+      "loss": 0.2392,
+      "num_input_tokens_seen": 7780072,
+      "step": 107
+    },
+    {
+      "epoch": 1.7808930425752856,
+      "grad_norm": 0.09139693528413773,
+      "learning_rate": 4.900734214192358e-05,
+      "loss": 0.2685,
+      "num_input_tokens_seen": 7857712,
+      "step": 108
+    },
+    {
+      "epoch": 1.7975077881619939,
+      "grad_norm": 0.1043916717171669,
+      "learning_rate": 4.898899996577407e-05,
+      "loss": 0.2513,
+      "num_input_tokens_seen": 7916832,
+      "step": 109
+    },
+    {
+      "epoch": 1.814122533748702,
+      "grad_norm": 0.09203662723302841,
+      "learning_rate": 4.8970493371704826e-05,
+      "loss": 0.2974,
+      "num_input_tokens_seen": 7993056,
+      "step": 110
+    },
+    {
+      "epoch": 1.8307372793354102,
+      "grad_norm": 0.09319474548101425,
+      "learning_rate": 4.8951822486557986e-05,
+      "loss": 0.3096,
+      "num_input_tokens_seen": 8090056,
+      "step": 111
+    },
+    {
+      "epoch": 1.8473520249221185,
+      "grad_norm": 0.10193445533514023,
+      "learning_rate": 4.893298743830168e-05,
+      "loss": 0.2633,
+      "num_input_tokens_seen": 8164808,
+      "step": 112
+    },
+    {
+      "epoch": 1.8639667705088265,
+      "grad_norm": 0.11407948285341263,
+      "learning_rate": 4.891398835602925e-05,
+      "loss": 0.2584,
+      "num_input_tokens_seen": 8223568,
+      "step": 113
+    },
+    {
+      "epoch": 1.8805815160955348,
+      "grad_norm": 0.11977085471153259,
+      "learning_rate": 4.8894825369958255e-05,
+      "loss": 0.2619,
+      "num_input_tokens_seen": 8276160,
+      "step": 114
+    },
+    {
+      "epoch": 1.897196261682243,
+      "grad_norm": 0.10925433784723282,
+      "learning_rate": 4.8875498611429674e-05,
+      "loss": 0.2762,
+      "num_input_tokens_seen": 8354904,
+      "step": 115
+    },
+    {
+      "epoch": 1.9138110072689511,
+      "grad_norm": 0.09673939645290375,
+      "learning_rate": 4.8856008212906925e-05,
+      "loss": 0.3152,
+      "num_input_tokens_seen": 8442584,
+      "step": 116
+    },
+    {
+      "epoch": 1.9304257528556594,
+      "grad_norm": 0.10827789455652237,
+      "learning_rate": 4.8836354307975026e-05,
+      "loss": 0.2759,
+      "num_input_tokens_seen": 8506688,
+      "step": 117
+    },
+    {
+      "epoch": 1.9470404984423677,
+      "grad_norm": 0.08390220254659653,
+      "learning_rate": 4.881653703133966e-05,
+      "loss": 0.2192,
+      "num_input_tokens_seen": 8610712,
+      "step": 118
+    },
+    {
+      "epoch": 1.9636552440290758,
+      "grad_norm": 0.09252211451530457,
+      "learning_rate": 4.87965565188262e-05,
+      "loss": 0.2618,
+      "num_input_tokens_seen": 8692624,
+      "step": 119
+    },
+    {
+      "epoch": 1.980269989615784,
+      "grad_norm": 0.1107102632522583,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.2666,
+      "num_input_tokens_seen": 8772208,
+      "step": 120
+    },
+    {
+      "epoch": 1.9968847352024923,
+      "grad_norm": 0.0917077362537384,
+      "learning_rate": 4.8756106335059646e-05,
+      "loss": 0.253,
+      "num_input_tokens_seen": 8854904,
+      "step": 121
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.2606711685657501,
+      "learning_rate": 4.87356369410476e-05,
+      "loss": 0.235,
+      "num_input_tokens_seen": 8872656,
+      "step": 122
+    },
+    {
+      "epoch": 2.016614745586708,
+      "grad_norm": 0.10363993793725967,
+      "learning_rate": 4.8715004865637614e-05,
+      "loss": 0.266,
+      "num_input_tokens_seen": 8946480,
+      "step": 123
+    },
+    {
+      "epoch": 2.0332294911734166,
+      "grad_norm": 0.09997844696044922,
+      "learning_rate": 4.869421025023965e-05,
+      "loss": 0.2696,
+      "num_input_tokens_seen": 9023328,
+      "step": 124
+    },
+    {
+      "epoch": 2.0498442367601246,
+      "grad_norm": 0.13349319994449615,
+      "learning_rate": 4.867325323737765e-05,
+      "loss": 0.2552,
+      "num_input_tokens_seen": 9074320,
+      "step": 125
+    },
+    {
+      "epoch": 2.0664589823468327,
+      "grad_norm": 0.11201464384794235,
+      "learning_rate": 4.8652133970688636e-05,
+      "loss": 0.2486,
+      "num_input_tokens_seen": 9148784,
+      "step": 126
+    },
+    {
+      "epoch": 2.083073727933541,
+      "grad_norm": 0.10193142294883728,
+      "learning_rate": 4.8630852594921706e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 9246624,
+      "step": 127
+    },
+    {
+      "epoch": 2.0996884735202492,
+      "grad_norm": 0.1305130422115326,
+      "learning_rate": 4.860940925593703e-05,
+      "loss": 0.304,
+      "num_input_tokens_seen": 9328176,
+      "step": 128
+    },
+    {
+      "epoch": 2.1163032191069573,
+      "grad_norm": 0.1137692779302597,
+      "learning_rate": 4.8587804100704845e-05,
+      "loss": 0.2427,
+      "num_input_tokens_seen": 9388936,
+      "step": 129
+    },
+    {
+      "epoch": 2.132917964693666,
+      "grad_norm": 0.12126237154006958,
+      "learning_rate": 4.856603727730447e-05,
+      "loss": 0.2485,
+      "num_input_tokens_seen": 9461664,
+      "step": 130
+    },
+    {
+      "epoch": 2.149532710280374,
+      "grad_norm": 0.11567176878452301,
+      "learning_rate": 4.854410893492326e-05,
+      "loss": 0.2628,
+      "num_input_tokens_seen": 9535000,
+      "step": 131
+    },
+    {
+      "epoch": 2.166147455867082,
+      "grad_norm": 0.1399552971124649,
+      "learning_rate": 4.852201922385564e-05,
+      "loss": 0.2518,
+      "num_input_tokens_seen": 9600296,
+      "step": 132
+    },
+    {
+      "epoch": 2.1827622014537904,
+      "grad_norm": 0.13912151753902435,
+      "learning_rate": 4.8499768295502004e-05,
+      "loss": 0.2429,
+      "num_input_tokens_seen": 9686784,
+      "step": 133
+    },
+    {
+      "epoch": 2.1993769470404985,
+      "grad_norm": 0.11130474507808685,
+      "learning_rate": 4.847735630236773e-05,
+      "loss": 0.2775,
+      "num_input_tokens_seen": 9781112,
+      "step": 134
+    },
+    {
+      "epoch": 2.2159916926272065,
+      "grad_norm": 0.12169156968593597,
+      "learning_rate": 4.8454783398062106e-05,
+      "loss": 0.2439,
+      "num_input_tokens_seen": 9849528,
+      "step": 135
+    },
+    {
+      "epoch": 2.232606438213915,
+      "grad_norm": 0.11766713112592697,
+      "learning_rate": 4.843204973729729e-05,
+      "loss": 0.2538,
+      "num_input_tokens_seen": 9931080,
+      "step": 136
+    },
+    {
+      "epoch": 2.249221183800623,
+      "grad_norm": 0.11854218691587448,
+      "learning_rate": 4.840915547588725e-05,
+      "loss": 0.2782,
+      "num_input_tokens_seen": 10011176,
+      "step": 137
+    },
+    {
+      "epoch": 2.265835929387331,
+      "grad_norm": 0.1340581178665161,
+      "learning_rate": 4.838610077074669e-05,
+      "loss": 0.248,
+      "num_input_tokens_seen": 10084128,
+      "step": 138
+    },
+    {
+      "epoch": 2.2824506749740396,
+      "grad_norm": 0.12075436115264893,
+      "learning_rate": 4.836288577988996e-05,
+      "loss": 0.2582,
+      "num_input_tokens_seen": 10155536,
+      "step": 139
+    },
+    {
+      "epoch": 2.2990654205607477,
+      "grad_norm": 0.10599923878908157,
+      "learning_rate": 4.8339510662430046e-05,
+      "loss": 0.2199,
+      "num_input_tokens_seen": 10251160,
+      "step": 140
+    },
+    {
+      "epoch": 2.3156801661474558,
+      "grad_norm": 0.1117846742272377,
+      "learning_rate": 4.8315975578577355e-05,
+      "loss": 0.2324,
+      "num_input_tokens_seen": 10345864,
+      "step": 141
+    },
+    {
+      "epoch": 2.3322949117341643,
+      "grad_norm": 0.13972057402133942,
+      "learning_rate": 4.8292280689638725e-05,
+      "loss": 0.4072,
+      "num_input_tokens_seen": 10417616,
+      "step": 142
+    },
+    {
+      "epoch": 2.3489096573208723,
+      "grad_norm": 0.13837860524654388,
+      "learning_rate": 4.826842615801628e-05,
+      "loss": 0.2607,
+      "num_input_tokens_seen": 10481816,
+      "step": 143
+    },
+    {
+      "epoch": 2.3655244029075804,
+      "grad_norm": 0.14040137827396393,
+      "learning_rate": 4.8244412147206284e-05,
+      "loss": 0.3094,
+      "num_input_tokens_seen": 10562056,
+      "step": 144
+    },
+    {
+      "epoch": 2.382139148494289,
+      "grad_norm": 0.1393299251794815,
+      "learning_rate": 4.822023882179811e-05,
+      "loss": 0.2407,
+      "num_input_tokens_seen": 10612808,
+      "step": 145
+    },
+    {
+      "epoch": 2.398753894080997,
+      "grad_norm": 0.13878698647022247,
+      "learning_rate": 4.8195906347473e-05,
+      "loss": 0.2481,
+      "num_input_tokens_seen": 10682328,
+      "step": 146
+    },
+    {
+      "epoch": 2.415368639667705,
+      "grad_norm": 0.10430227965116501,
+      "learning_rate": 4.817141489100302e-05,
+      "loss": 0.2528,
+      "num_input_tokens_seen": 10771912,
+      "step": 147
+    },
+    {
+      "epoch": 2.431983385254413,
+      "grad_norm": 0.12963703274726868,
+      "learning_rate": 4.814676462024988e-05,
+      "loss": 0.2739,
+      "num_input_tokens_seen": 10842232,
+      "step": 148
+    },
+    {
+      "epoch": 2.4485981308411215,
+      "grad_norm": 0.13274963200092316,
+      "learning_rate": 4.8121955704163745e-05,
+      "loss": 0.2407,
+      "num_input_tokens_seen": 10902264,
+      "step": 149
+    },
+    {
+      "epoch": 2.4652128764278296,
+      "grad_norm": 0.11079717427492142,
+      "learning_rate": 4.8096988312782174e-05,
+      "loss": 0.2142,
+      "num_input_tokens_seen": 10992744,
+      "step": 150
+    },
+    {
+      "epoch": 2.4818276220145377,
+      "grad_norm": 0.08429212868213654,
+      "learning_rate": 4.8071862617228855e-05,
+      "loss": 0.1428,
+      "num_input_tokens_seen": 11090064,
+      "step": 151
+    },
+    {
+      "epoch": 2.498442367601246,
+      "grad_norm": 0.12903761863708496,
+      "learning_rate": 4.8046578789712515e-05,
+      "loss": 0.2268,
+      "num_input_tokens_seen": 11162864,
+      "step": 152
+    },
+    {
+      "epoch": 2.515057113187954,
+      "grad_norm": 0.14638672769069672,
+      "learning_rate": 4.8021137003525664e-05,
+      "loss": 0.2388,
+      "num_input_tokens_seen": 11224368,
+      "step": 153
+    },
+    {
+      "epoch": 2.5316718587746623,
+      "grad_norm": 0.1372838169336319,
+      "learning_rate": 4.7995537433043446e-05,
+      "loss": 0.2588,
+      "num_input_tokens_seen": 11291056,
+      "step": 154
+    },
+    {
+      "epoch": 2.5482866043613708,
+      "grad_norm": 0.15665481984615326,
+      "learning_rate": 4.796978025372246e-05,
+      "loss": 0.2225,
+      "num_input_tokens_seen": 11345464,
+      "step": 155
+    },
+    {
+      "epoch": 2.564901349948079,
+      "grad_norm": 0.13234855234622955,
+      "learning_rate": 4.794386564209953e-05,
+      "loss": 0.275,
+      "num_input_tokens_seen": 11418912,
+      "step": 156
+    },
+    {
+      "epoch": 2.581516095534787,
+      "grad_norm": 0.13585953414440155,
+      "learning_rate": 4.79177937757905e-05,
+      "loss": 0.2407,
+      "num_input_tokens_seen": 11491216,
+      "step": 157
+    },
+    {
+      "epoch": 2.5981308411214954,
+      "grad_norm": 0.1423913538455963,
+      "learning_rate": 4.7891564833489035e-05,
+      "loss": 0.1971,
+      "num_input_tokens_seen": 11558016,
+      "step": 158
+    },
+    {
+      "epoch": 2.6147455867082035,
+      "grad_norm": 0.13013511896133423,
+      "learning_rate": 4.7865178994965344e-05,
+      "loss": 0.2362,
+      "num_input_tokens_seen": 11630432,
+      "step": 159
+    },
+    {
+      "epoch": 2.6313603322949115,
+      "grad_norm": 0.1587141752243042,
+      "learning_rate": 4.783863644106502e-05,
+      "loss": 0.2252,
+      "num_input_tokens_seen": 11684624,
+      "step": 160
+    },
+    {
+      "epoch": 2.64797507788162,
+      "grad_norm": 0.12592960894107819,
+      "learning_rate": 4.781193735370777e-05,
+      "loss": 0.2506,
+      "num_input_tokens_seen": 11770232,
+      "step": 161
+    },
+    {
+      "epoch": 2.664589823468328,
+      "grad_norm": 0.1583249419927597,
+      "learning_rate": 4.7785081915886134e-05,
+      "loss": 0.2352,
+      "num_input_tokens_seen": 11828360,
+      "step": 162
+    },
+    {
+      "epoch": 2.681204569055036,
+      "grad_norm": 0.14881783723831177,
+      "learning_rate": 4.775807031166428e-05,
+      "loss": 0.2308,
+      "num_input_tokens_seen": 11915944,
+      "step": 163
+    },
+    {
+      "epoch": 2.6978193146417446,
+      "grad_norm": 0.1607823222875595,
+      "learning_rate": 4.773090272617672e-05,
+      "loss": 0.2238,
+      "num_input_tokens_seen": 11981792,
+      "step": 164
+    },
+    {
+      "epoch": 2.7144340602284527,
+      "grad_norm": 0.13583113253116608,
+      "learning_rate": 4.7703579345627035e-05,
+      "loss": 0.3196,
+      "num_input_tokens_seen": 12044024,
+      "step": 165
+    },
+    {
+      "epoch": 2.7310488058151607,
+      "grad_norm": 0.19167298078536987,
+      "learning_rate": 4.7676100357286624e-05,
+      "loss": 0.2745,
+      "num_input_tokens_seen": 12093424,
+      "step": 166
+    },
+    {
+      "epoch": 2.7476635514018692,
+      "grad_norm": 0.130703404545784,
+      "learning_rate": 4.76484659494934e-05,
+      "loss": 0.2285,
+      "num_input_tokens_seen": 12167792,
+      "step": 167
+    },
+    {
+      "epoch": 2.7642782969885773,
+      "grad_norm": 0.14331185817718506,
+      "learning_rate": 4.762067631165049e-05,
+      "loss": 0.2506,
+      "num_input_tokens_seen": 12233712,
+      "step": 168
+    },
+    {
+      "epoch": 2.7808930425752854,
+      "grad_norm": 0.12700341641902924,
+      "learning_rate": 4.7592731634224966e-05,
+      "loss": 0.2052,
+      "num_input_tokens_seen": 12310544,
+      "step": 169
+    },
+    {
+      "epoch": 2.797507788161994,
+      "grad_norm": 0.15118420124053955,
+      "learning_rate": 4.756463210874652e-05,
+      "loss": 0.2309,
+      "num_input_tokens_seen": 12400160,
+      "step": 170
+    },
+    {
+      "epoch": 2.814122533748702,
+      "grad_norm": 0.14001020789146423,
+      "learning_rate": 4.753637792780614e-05,
+      "loss": 0.2544,
+      "num_input_tokens_seen": 12480432,
+      "step": 171
+    },
+    {
+      "epoch": 2.83073727933541,
+      "grad_norm": 0.12076311558485031,
+      "learning_rate": 4.7507969285054845e-05,
+      "loss": 0.2434,
+      "num_input_tokens_seen": 12568064,
+      "step": 172
+    },
+    {
+      "epoch": 2.8473520249221185,
+      "grad_norm": 0.16462342441082,
+      "learning_rate": 4.7479406375202264e-05,
+      "loss": 0.2417,
+      "num_input_tokens_seen": 12647400,
+      "step": 173
+    },
+    {
+      "epoch": 2.8639667705088265,
+      "grad_norm": 0.17294971644878387,
+      "learning_rate": 4.745068939401539e-05,
+      "loss": 0.2121,
+      "num_input_tokens_seen": 12698208,
+      "step": 174
+    },
+    {
+      "epoch": 2.8805815160955346,
+      "grad_norm": 0.16743803024291992,
+      "learning_rate": 4.742181853831721e-05,
+      "loss": 0.2238,
+      "num_input_tokens_seen": 12758528,
+      "step": 175
+    },
+    {
+      "epoch": 2.897196261682243,
+      "grad_norm": 0.14583320915699005,
+      "learning_rate": 4.7392794005985326e-05,
+      "loss": 0.2333,
+      "num_input_tokens_seen": 12837264,
+      "step": 176
+    },
+    {
+      "epoch": 2.913811007268951,
+      "grad_norm": 0.1509270817041397,
+      "learning_rate": 4.7363615995950626e-05,
+      "loss": 0.2179,
+      "num_input_tokens_seen": 12902368,
+      "step": 177
+    },
+    {
+      "epoch": 2.930425752855659,
+      "grad_norm": 0.12910738587379456,
+      "learning_rate": 4.733428470819594e-05,
+      "loss": 0.2144,
+      "num_input_tokens_seen": 12974296,
+      "step": 178
+    },
+    {
+      "epoch": 2.9470404984423677,
+      "grad_norm": 0.142000213265419,
+      "learning_rate": 4.730480034375462e-05,
+      "loss": 0.2413,
+      "num_input_tokens_seen": 13057280,
+      "step": 179
+    },
+    {
+      "epoch": 2.9636552440290758,
+      "grad_norm": 0.131468266248703,
+      "learning_rate": 4.72751631047092e-05,
+      "loss": 0.294,
+      "num_input_tokens_seen": 13158232,
+      "step": 180
+    },
+    {
+      "epoch": 2.980269989615784,
+      "grad_norm": 0.1529342085123062,
+      "learning_rate": 4.7245373194189994e-05,
+      "loss": 0.216,
+      "num_input_tokens_seen": 13229840,
+      "step": 181
+    },
+    {
+      "epoch": 2.9968847352024923,
+      "grad_norm": 0.1573815941810608,
+      "learning_rate": 4.7215430816373726e-05,
+      "loss": 0.2384,
+      "num_input_tokens_seen": 13296520,
+      "step": 182
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.2532118558883667,
+      "learning_rate": 4.718533617648209e-05,
+      "loss": 0.1459,
+      "num_input_tokens_seen": 13309672,
+      "step": 183
+    },
+    {
+      "epoch": 3.016614745586708,
+      "grad_norm": 0.16963432729244232,
+      "learning_rate": 4.715508948078037e-05,
+      "loss": 0.1985,
+      "num_input_tokens_seen": 13371544,
+      "step": 184
+    },
+    {
+      "epoch": 3.0332294911734166,
+      "grad_norm": 0.18877384066581726,
+      "learning_rate": 4.712469093657605e-05,
+      "loss": 0.1856,
+      "num_input_tokens_seen": 13432984,
+      "step": 185
+    },
+    {
+      "epoch": 3.0498442367601246,
+      "grad_norm": 0.14922884106636047,
+      "learning_rate": 4.709414075221734e-05,
+      "loss": 0.2385,
+      "num_input_tokens_seen": 13500016,
+      "step": 186
+    },
+    {
+      "epoch": 3.0664589823468327,
+      "grad_norm": 0.2028326541185379,
+      "learning_rate": 4.706343913709178e-05,
+      "loss": 0.2227,
+      "num_input_tokens_seen": 13579672,
+      "step": 187
+    },
+    {
+      "epoch": 3.083073727933541,
+      "grad_norm": 0.19964616000652313,
+      "learning_rate": 4.70325863016248e-05,
+      "loss": 0.2045,
+      "num_input_tokens_seen": 13630704,
+      "step": 188
+    },
+    {
+      "epoch": 3.0996884735202492,
+      "grad_norm": 0.1594657599925995,
+      "learning_rate": 4.7001582457278304e-05,
+      "loss": 0.2648,
+      "num_input_tokens_seen": 13695472,
+      "step": 189
+    },
+    {
+      "epoch": 3.1163032191069573,
+      "grad_norm": 0.16952532529830933,
+      "learning_rate": 4.697042781654913e-05,
+      "loss": 0.22,
+      "num_input_tokens_seen": 13767792,
+      "step": 190
+    },
+    {
+      "epoch": 3.132917964693666,
+      "grad_norm": 0.16775831580162048,
+      "learning_rate": 4.693912259296773e-05,
+      "loss": 0.2667,
+      "num_input_tokens_seen": 13857352,
+      "step": 191
+    },
+    {
+      "epoch": 3.149532710280374,
+      "grad_norm": 0.15529580414295197,
+      "learning_rate": 4.690766700109659e-05,
+      "loss": 0.2154,
+      "num_input_tokens_seen": 13939928,
+      "step": 192
+    },
+    {
+      "epoch": 3.166147455867082,
+      "grad_norm": 0.1619848757982254,
+      "learning_rate": 4.687606125652882e-05,
+      "loss": 0.1963,
+      "num_input_tokens_seen": 14017936,
+      "step": 193
+    },
+    {
+      "epoch": 3.1827622014537904,
+      "grad_norm": 0.18066684901714325,
+      "learning_rate": 4.684430557588664e-05,
+      "loss": 0.1862,
+      "num_input_tokens_seen": 14074176,
+      "step": 194
+    },
+    {
+      "epoch": 3.1993769470404985,
+      "grad_norm": 0.16520777344703674,
+      "learning_rate": 4.681240017681993e-05,
+      "loss": 0.2576,
+      "num_input_tokens_seen": 14167656,
+      "step": 195
+    },
+    {
+      "epoch": 3.2159916926272065,
+      "grad_norm": 0.15385325253009796,
+      "learning_rate": 4.678034527800474e-05,
+      "loss": 0.1813,
+      "num_input_tokens_seen": 14235800,
+      "step": 196
+    },
+    {
+      "epoch": 3.232606438213915,
+      "grad_norm": 0.16897696256637573,
+      "learning_rate": 4.674814109914174e-05,
+      "loss": 0.1741,
+      "num_input_tokens_seen": 14301272,
+      "step": 197
+    },
+    {
+      "epoch": 3.249221183800623,
+      "grad_norm": 0.19556447863578796,
+      "learning_rate": 4.671578786095478e-05,
+      "loss": 0.2186,
+      "num_input_tokens_seen": 14347352,
+      "step": 198
+    },
+    {
+      "epoch": 3.265835929387331,
+      "grad_norm": 0.17333142459392548,
+      "learning_rate": 4.668328578518933e-05,
+      "loss": 0.2892,
+      "num_input_tokens_seen": 14434600,
+      "step": 199
+    },
+    {
+      "epoch": 3.2824506749740396,
+      "grad_norm": 0.20295488834381104,
+      "learning_rate": 4.665063509461097e-05,
+      "loss": 0.2014,
+      "num_input_tokens_seen": 14484104,
+      "step": 200
+    },
+    {
+      "epoch": 3.2990654205607477,
+      "grad_norm": 0.1597638726234436,
+      "learning_rate": 4.661783601300388e-05,
+      "loss": 0.2158,
+      "num_input_tokens_seen": 14567152,
+      "step": 201
+    },
+    {
+      "epoch": 3.3156801661474558,
+      "grad_norm": 0.19849488139152527,
+      "learning_rate": 4.6584888765169296e-05,
+      "loss": 0.2578,
+      "num_input_tokens_seen": 14647040,
+      "step": 202
+    },
+    {
+      "epoch": 3.3322949117341643,
+      "grad_norm": 0.1508200466632843,
+      "learning_rate": 4.6551793576923964e-05,
+      "loss": 0.2213,
+      "num_input_tokens_seen": 14738216,
+      "step": 203
+    },
+    {
+      "epoch": 3.3489096573208723,
+      "grad_norm": 0.1687687337398529,
+      "learning_rate": 4.65185506750986e-05,
+      "loss": 0.1828,
+      "num_input_tokens_seen": 14811216,
+      "step": 204
+    },
+    {
+      "epoch": 3.3655244029075804,
+      "grad_norm": 0.16587376594543457,
+      "learning_rate": 4.648516028753632e-05,
+      "loss": 0.1619,
+      "num_input_tokens_seen": 14885992,
+      "step": 205
+    },
+    {
+      "epoch": 3.382139148494289,
+      "grad_norm": 0.16600169241428375,
+      "learning_rate": 4.645162264309112e-05,
+      "loss": 0.2438,
+      "num_input_tokens_seen": 14961984,
+      "step": 206
+    },
+    {
+      "epoch": 3.398753894080997,
+      "grad_norm": 0.1877703070640564,
+      "learning_rate": 4.6417937971626245e-05,
+      "loss": 0.1771,
+      "num_input_tokens_seen": 15021240,
+      "step": 207
+    },
+    {
+      "epoch": 3.415368639667705,
+      "grad_norm": 0.20105206966400146,
+      "learning_rate": 4.638410650401267e-05,
+      "loss": 0.1742,
+      "num_input_tokens_seen": 15092016,
+      "step": 208
+    },
+    {
+      "epoch": 3.431983385254413,
+      "grad_norm": 0.12934140861034393,
+      "learning_rate": 4.635012847212748e-05,
+      "loss": 0.1725,
+      "num_input_tokens_seen": 15198192,
+      "step": 209
+    },
+    {
+      "epoch": 3.4485981308411215,
+      "grad_norm": 0.18388882279396057,
+      "learning_rate": 4.6316004108852305e-05,
+      "loss": 0.186,
+      "num_input_tokens_seen": 15258432,
+      "step": 210
+    },
+    {
+      "epoch": 3.4652128764278296,
+      "grad_norm": 0.1766858547925949,
+      "learning_rate": 4.628173364807171e-05,
+      "loss": 0.2166,
+      "num_input_tokens_seen": 15329600,
+      "step": 211
+    },
+    {
+      "epoch": 3.4818276220145377,
+      "grad_norm": 0.16214998066425323,
+      "learning_rate": 4.6247317324671605e-05,
+      "loss": 0.2038,
+      "num_input_tokens_seen": 15407920,
+      "step": 212
+    },
+    {
+      "epoch": 3.498442367601246,
+      "grad_norm": 0.16933797299861908,
+      "learning_rate": 4.6212755374537596e-05,
+      "loss": 0.2017,
+      "num_input_tokens_seen": 15479640,
+      "step": 213
+    },
+    {
+      "epoch": 3.515057113187954,
+      "grad_norm": 0.19472749531269073,
+      "learning_rate": 4.617804803455344e-05,
+      "loss": 0.2048,
+      "num_input_tokens_seen": 15561960,
+      "step": 214
+    },
+    {
+      "epoch": 3.5316718587746623,
+      "grad_norm": 0.33335182070732117,
+      "learning_rate": 4.614319554259934e-05,
+      "loss": 0.2358,
+      "num_input_tokens_seen": 15641440,
+      "step": 215
+    },
+    {
+      "epoch": 3.5482866043613708,
+      "grad_norm": 0.19587557017803192,
+      "learning_rate": 4.610819813755038e-05,
+      "loss": 0.2374,
+      "num_input_tokens_seen": 15728872,
+      "step": 216
+    },
+    {
+      "epoch": 3.564901349948079,
+      "grad_norm": 0.19063518941402435,
+      "learning_rate": 4.607305605927487e-05,
+      "loss": 0.1919,
+      "num_input_tokens_seen": 15798112,
+      "step": 217
+    },
+    {
+      "epoch": 3.581516095534787,
+      "grad_norm": 0.19598323106765747,
+      "learning_rate": 4.6037769548632656e-05,
+      "loss": 0.2583,
+      "num_input_tokens_seen": 15865936,
+      "step": 218
+    },
+    {
+      "epoch": 3.5981308411214954,
+      "grad_norm": 0.18066690862178802,
+      "learning_rate": 4.600233884747355e-05,
+      "loss": 0.2337,
+      "num_input_tokens_seen": 15941368,
+      "step": 219
+    },
+    {
+      "epoch": 3.6147455867082035,
+      "grad_norm": 0.16981899738311768,
+      "learning_rate": 4.5966764198635606e-05,
+      "loss": 0.1818,
+      "num_input_tokens_seen": 16028208,
+      "step": 220
+    },
+    {
+      "epoch": 3.6313603322949115,
+      "grad_norm": 0.180410236120224,
+      "learning_rate": 4.5931045845943474e-05,
+      "loss": 0.1646,
+      "num_input_tokens_seen": 16104408,
+      "step": 221
+    },
+    {
+      "epoch": 3.64797507788162,
+      "grad_norm": 0.19180680811405182,
+      "learning_rate": 4.5895184034206765e-05,
+      "loss": 0.3263,
+      "num_input_tokens_seen": 16156800,
+      "step": 222
+    },
+    {
+      "epoch": 3.664589823468328,
+      "grad_norm": 0.16119280457496643,
+      "learning_rate": 4.585917900921829e-05,
+      "loss": 0.2636,
+      "num_input_tokens_seen": 16256712,
+      "step": 223
+    },
+    {
+      "epoch": 3.681204569055036,
+      "grad_norm": 0.18559172749519348,
+      "learning_rate": 4.5823031017752485e-05,
+      "loss": 0.1759,
+      "num_input_tokens_seen": 16330344,
+      "step": 224
+    },
+    {
+      "epoch": 3.6978193146417446,
+      "grad_norm": 0.17767880856990814,
+      "learning_rate": 4.5786740307563636e-05,
+      "loss": 0.196,
+      "num_input_tokens_seen": 16399792,
+      "step": 225
+    },
+    {
+      "epoch": 3.7144340602284527,
+      "grad_norm": 0.17806987464427948,
+      "learning_rate": 4.575030712738419e-05,
+      "loss": 0.186,
+      "num_input_tokens_seen": 16466368,
+      "step": 226
+    },
+    {
+      "epoch": 3.7310488058151607,
+      "grad_norm": 0.1952792853116989,
+      "learning_rate": 4.571373172692309e-05,
+      "loss": 0.1789,
+      "num_input_tokens_seen": 16530976,
+      "step": 227
+    },
+    {
+      "epoch": 3.7476635514018692,
+      "grad_norm": 0.1774374544620514,
+      "learning_rate": 4.567701435686404e-05,
+      "loss": 0.1929,
+      "num_input_tokens_seen": 16600216,
+      "step": 228
+    },
+    {
+      "epoch": 3.7642782969885773,
+      "grad_norm": 0.18798600137233734,
+      "learning_rate": 4.5640155268863796e-05,
+      "loss": 0.2268,
+      "num_input_tokens_seen": 16673192,
+      "step": 229
+    },
+    {
+      "epoch": 3.7808930425752854,
+      "grad_norm": 0.2022520750761032,
+      "learning_rate": 4.5603154715550386e-05,
+      "loss": 0.1716,
+      "num_input_tokens_seen": 16739912,
+      "step": 230
+    },
+    {
+      "epoch": 3.797507788161994,
+      "grad_norm": 0.15170948207378387,
+      "learning_rate": 4.55660129505215e-05,
+      "loss": 0.1844,
+      "num_input_tokens_seen": 16834632,
+      "step": 231
+    },
+    {
+      "epoch": 3.814122533748702,
+      "grad_norm": 0.16655084490776062,
+      "learning_rate": 4.5528730228342605e-05,
+      "loss": 0.1899,
+      "num_input_tokens_seen": 16914728,
+      "step": 232
+    },
+    {
+      "epoch": 3.83073727933541,
+      "grad_norm": 0.19025221467018127,
+      "learning_rate": 4.549130680454532e-05,
+      "loss": 0.2214,
+      "num_input_tokens_seen": 17014304,
+      "step": 233
+    },
+    {
+      "epoch": 3.8473520249221185,
+      "grad_norm": 0.17126557230949402,
+      "learning_rate": 4.545374293562559e-05,
+      "loss": 0.2062,
+      "num_input_tokens_seen": 17106664,
+      "step": 234
+    },
+    {
+      "epoch": 3.8639667705088265,
+      "grad_norm": 0.16162410378456116,
+      "learning_rate": 4.541603887904198e-05,
+      "loss": 0.2016,
+      "num_input_tokens_seen": 17193744,
+      "step": 235
+    },
+    {
+      "epoch": 3.8805815160955346,
+      "grad_norm": 0.2067136913537979,
+      "learning_rate": 4.537819489321386e-05,
+      "loss": 0.1992,
+      "num_input_tokens_seen": 17254656,
+      "step": 236
+    },
+    {
+      "epoch": 3.897196261682243,
+      "grad_norm": 0.200433611869812,
+      "learning_rate": 4.534021123751968e-05,
+      "loss": 0.1961,
+      "num_input_tokens_seen": 17325896,
+      "step": 237
+    },
+    {
+      "epoch": 3.913811007268951,
+      "grad_norm": 0.2062034010887146,
+      "learning_rate": 4.5302088172295156e-05,
+      "loss": 0.2302,
+      "num_input_tokens_seen": 17394424,
+      "step": 238
+    },
+    {
+      "epoch": 3.930425752855659,
+      "grad_norm": 0.1928798407316208,
+      "learning_rate": 4.526382595883152e-05,
+      "loss": 0.1846,
+      "num_input_tokens_seen": 17456352,
+      "step": 239
+    },
+    {
+      "epoch": 3.9470404984423677,
+      "grad_norm": 0.2011859118938446,
+      "learning_rate": 4.522542485937369e-05,
+      "loss": 0.1879,
+      "num_input_tokens_seen": 17519168,
+      "step": 240
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 1200,
+  "num_input_tokens_seen": 17519168,
+  "num_train_epochs": 20,
+  "save_steps": 60,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.477946311264174e+18,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-300/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-300/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-300/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-300/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-300/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 17500,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-360/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-360/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-360/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-360/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 17500,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-360/trainer_state.json

@@ -0,0 +1,2913 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 5.913811007268951,
+  "eval_steps": 500,
+  "global_step": 360,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.016614745586708203,
+      "grad_norm": 0.050998032093048096,
+      "learning_rate": 4.999991432639962e-05,
+      "loss": 0.5487,
+      "num_input_tokens_seen": 70408,
+      "step": 1
+    },
+    {
+      "epoch": 0.033229491173416406,
+      "grad_norm": 0.049370743334293365,
+      "learning_rate": 4.999965730618567e-05,
+      "loss": 0.4981,
+      "num_input_tokens_seen": 139640,
+      "step": 2
+    },
+    {
+      "epoch": 0.04984423676012461,
+      "grad_norm": 0.05077400803565979,
+      "learning_rate": 4.9999228941119745e-05,
+      "loss": 0.5505,
+      "num_input_tokens_seen": 223656,
+      "step": 3
+    },
+    {
+      "epoch": 0.06645898234683281,
+      "grad_norm": 0.04397282376885414,
+      "learning_rate": 4.999862923413781e-05,
+      "loss": 0.504,
+      "num_input_tokens_seen": 300688,
+      "step": 4
+    },
+    {
+      "epoch": 0.08307372793354102,
+      "grad_norm": 0.05225864797830582,
+      "learning_rate": 4.999785818935018e-05,
+      "loss": 0.4925,
+      "num_input_tokens_seen": 366368,
+      "step": 5
+    },
+    {
+      "epoch": 0.09968847352024922,
+      "grad_norm": 0.049482282251119614,
+      "learning_rate": 4.999691581204152e-05,
+      "loss": 0.4771,
+      "num_input_tokens_seen": 445808,
+      "step": 6
+    },
+    {
+      "epoch": 0.11630321910695743,
+      "grad_norm": 0.05594080314040184,
+      "learning_rate": 4.9995802108670775e-05,
+      "loss": 0.4986,
+      "num_input_tokens_seen": 522800,
+      "step": 7
+    },
+    {
+      "epoch": 0.13291796469366562,
+      "grad_norm": 0.051852282136678696,
+      "learning_rate": 4.999451708687114e-05,
+      "loss": 0.5171,
+      "num_input_tokens_seen": 599608,
+      "step": 8
+    },
+    {
+      "epoch": 0.14953271028037382,
+      "grad_norm": 0.045517683029174805,
+      "learning_rate": 4.9993060755450015e-05,
+      "loss": 0.5669,
+      "num_input_tokens_seen": 681424,
+      "step": 9
+    },
+    {
+      "epoch": 0.16614745586708204,
+      "grad_norm": 0.044325754046440125,
+      "learning_rate": 4.999143312438893e-05,
+      "loss": 0.4218,
+      "num_input_tokens_seen": 756744,
+      "step": 10
+    },
+    {
+      "epoch": 0.18276220145379024,
+      "grad_norm": 0.04328459873795509,
+      "learning_rate": 4.998963420484349e-05,
+      "loss": 0.434,
+      "num_input_tokens_seen": 842576,
+      "step": 11
+    },
+    {
+      "epoch": 0.19937694704049844,
+      "grad_norm": 0.04725787043571472,
+      "learning_rate": 4.998766400914329e-05,
+      "loss": 0.4287,
+      "num_input_tokens_seen": 917232,
+      "step": 12
+    },
+    {
+      "epoch": 0.21599169262720663,
+      "grad_norm": 0.03806879371404648,
+      "learning_rate": 4.9985522550791825e-05,
+      "loss": 0.3454,
+      "num_input_tokens_seen": 1006800,
+      "step": 13
+    },
+    {
+      "epoch": 0.23260643821391486,
+      "grad_norm": 0.05201176926493645,
+      "learning_rate": 4.998320984446641e-05,
+      "loss": 0.436,
+      "num_input_tokens_seen": 1085824,
+      "step": 14
+    },
+    {
+      "epoch": 0.24922118380062305,
+      "grad_norm": 0.047955628484487534,
+      "learning_rate": 4.9980725906018074e-05,
+      "loss": 0.4625,
+      "num_input_tokens_seen": 1164160,
+      "step": 15
+    },
+    {
+      "epoch": 0.26583592938733125,
+      "grad_norm": 0.05529098957777023,
+      "learning_rate": 4.997807075247146e-05,
+      "loss": 0.5035,
+      "num_input_tokens_seen": 1242264,
+      "step": 16
+    },
+    {
+      "epoch": 0.2824506749740395,
+      "grad_norm": 0.04751162976026535,
+      "learning_rate": 4.997524440202469e-05,
+      "loss": 0.4354,
+      "num_input_tokens_seen": 1325904,
+      "step": 17
+    },
+    {
+      "epoch": 0.29906542056074764,
+      "grad_norm": 0.06726882606744766,
+      "learning_rate": 4.9972246874049254e-05,
+      "loss": 0.5439,
+      "num_input_tokens_seen": 1385632,
+      "step": 18
+    },
+    {
+      "epoch": 0.31568016614745587,
+      "grad_norm": 0.05245920270681381,
+      "learning_rate": 4.996907818908987e-05,
+      "loss": 0.3727,
+      "num_input_tokens_seen": 1470632,
+      "step": 19
+    },
+    {
+      "epoch": 0.3322949117341641,
+      "grad_norm": 0.05745376646518707,
+      "learning_rate": 4.996573836886435e-05,
+      "loss": 0.4894,
+      "num_input_tokens_seen": 1547536,
+      "step": 20
+    },
+    {
+      "epoch": 0.34890965732087226,
+      "grad_norm": 0.056607529520988464,
+      "learning_rate": 4.9962227436263453e-05,
+      "loss": 0.3846,
+      "num_input_tokens_seen": 1615528,
+      "step": 21
+    },
+    {
+      "epoch": 0.3655244029075805,
+      "grad_norm": 0.06150667741894722,
+      "learning_rate": 4.995854541535071e-05,
+      "loss": 0.4362,
+      "num_input_tokens_seen": 1694352,
+      "step": 22
+    },
+    {
+      "epoch": 0.3821391484942887,
+      "grad_norm": 0.056484442204236984,
+      "learning_rate": 4.9954692331362294e-05,
+      "loss": 0.4438,
+      "num_input_tokens_seen": 1753776,
+      "step": 23
+    },
+    {
+      "epoch": 0.3987538940809969,
+      "grad_norm": 0.0704159140586853,
+      "learning_rate": 4.995066821070679e-05,
+      "loss": 0.4496,
+      "num_input_tokens_seen": 1809048,
+      "step": 24
+    },
+    {
+      "epoch": 0.4153686396677051,
+      "grad_norm": 0.06202029809355736,
+      "learning_rate": 4.994647308096509e-05,
+      "loss": 0.5096,
+      "num_input_tokens_seen": 1884264,
+      "step": 25
+    },
+    {
+      "epoch": 0.43198338525441327,
+      "grad_norm": 0.04237145930528641,
+      "learning_rate": 4.994210697089014e-05,
+      "loss": 0.3722,
+      "num_input_tokens_seen": 1981704,
+      "step": 26
+    },
+    {
+      "epoch": 0.4485981308411215,
+      "grad_norm": 0.06920398026704788,
+      "learning_rate": 4.9937569910406756e-05,
+      "loss": 0.4103,
+      "num_input_tokens_seen": 2044144,
+      "step": 27
+    },
+    {
+      "epoch": 0.4652128764278297,
+      "grad_norm": 0.062432270497083664,
+      "learning_rate": 4.9932861930611454e-05,
+      "loss": 0.357,
+      "num_input_tokens_seen": 2107584,
+      "step": 28
+    },
+    {
+      "epoch": 0.4818276220145379,
+      "grad_norm": 0.06791180372238159,
+      "learning_rate": 4.9927983063772196e-05,
+      "loss": 0.3889,
+      "num_input_tokens_seen": 2169248,
+      "step": 29
+    },
+    {
+      "epoch": 0.4984423676012461,
+      "grad_norm": 0.07219590991735458,
+      "learning_rate": 4.99229333433282e-05,
+      "loss": 0.3543,
+      "num_input_tokens_seen": 2230344,
+      "step": 30
+    },
+    {
+      "epoch": 0.5150571131879543,
+      "grad_norm": 0.0647474005818367,
+      "learning_rate": 4.9917712803889674e-05,
+      "loss": 0.3453,
+      "num_input_tokens_seen": 2302368,
+      "step": 31
+    },
+    {
+      "epoch": 0.5316718587746625,
+      "grad_norm": 0.07434642314910889,
+      "learning_rate": 4.991232148123761e-05,
+      "loss": 0.435,
+      "num_input_tokens_seen": 2369984,
+      "step": 32
+    },
+    {
+      "epoch": 0.5482866043613707,
+      "grad_norm": 0.05302443355321884,
+      "learning_rate": 4.990675941232353e-05,
+      "loss": 0.3981,
+      "num_input_tokens_seen": 2453032,
+      "step": 33
+    },
+    {
+      "epoch": 0.564901349948079,
+      "grad_norm": 0.053745292127132416,
+      "learning_rate": 4.990102663526924e-05,
+      "loss": 0.3755,
+      "num_input_tokens_seen": 2527464,
+      "step": 34
+    },
+    {
+      "epoch": 0.5815160955347871,
+      "grad_norm": 0.06717613339424133,
+      "learning_rate": 4.989512318936655e-05,
+      "loss": 0.3699,
+      "num_input_tokens_seen": 2597032,
+      "step": 35
+    },
+    {
+      "epoch": 0.5981308411214953,
+      "grad_norm": 0.071847103536129,
+      "learning_rate": 4.9889049115077005e-05,
+      "loss": 0.3705,
+      "num_input_tokens_seen": 2671704,
+      "step": 36
+    },
+    {
+      "epoch": 0.6147455867082036,
+      "grad_norm": 0.0460306741297245,
+      "learning_rate": 4.988280445403164e-05,
+      "loss": 0.3797,
+      "num_input_tokens_seen": 2767640,
+      "step": 37
+    },
+    {
+      "epoch": 0.6313603322949117,
+      "grad_norm": 0.053273387253284454,
+      "learning_rate": 4.987638924903067e-05,
+      "loss": 0.3799,
+      "num_input_tokens_seen": 2843720,
+      "step": 38
+    },
+    {
+      "epoch": 0.6479750778816199,
+      "grad_norm": 0.05600422993302345,
+      "learning_rate": 4.9869803544043166e-05,
+      "loss": 0.2866,
+      "num_input_tokens_seen": 2921472,
+      "step": 39
+    },
+    {
+      "epoch": 0.6645898234683282,
+      "grad_norm": 0.06414052098989487,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.4115,
+      "num_input_tokens_seen": 2998400,
+      "step": 40
+    },
+    {
+      "epoch": 0.6812045690550363,
+      "grad_norm": 0.09214208275079727,
+      "learning_rate": 4.985612081582764e-05,
+      "loss": 0.3804,
+      "num_input_tokens_seen": 3059648,
+      "step": 41
+    },
+    {
+      "epoch": 0.6978193146417445,
+      "grad_norm": 0.0555964931845665,
+      "learning_rate": 4.98490238863795e-05,
+      "loss": 0.3121,
+      "num_input_tokens_seen": 3140184,
+      "step": 42
+    },
+    {
+      "epoch": 0.7144340602284528,
+      "grad_norm": 0.06256969273090363,
+      "learning_rate": 4.984175664450397e-05,
+      "loss": 0.3271,
+      "num_input_tokens_seen": 3207184,
+      "step": 43
+    },
+    {
+      "epoch": 0.731048805815161,
+      "grad_norm": 0.0543232187628746,
+      "learning_rate": 4.983431914000991e-05,
+      "loss": 0.364,
+      "num_input_tokens_seen": 3292344,
+      "step": 44
+    },
+    {
+      "epoch": 0.7476635514018691,
+      "grad_norm": 0.06077824532985687,
+      "learning_rate": 4.982671142387316e-05,
+      "loss": 0.3894,
+      "num_input_tokens_seen": 3365384,
+      "step": 45
+    },
+    {
+      "epoch": 0.7642782969885774,
+      "grad_norm": 0.06091070920228958,
+      "learning_rate": 4.981893354823614e-05,
+      "loss": 0.3354,
+      "num_input_tokens_seen": 3440720,
+      "step": 46
+    },
+    {
+      "epoch": 0.7808930425752856,
+      "grad_norm": 0.054153311997652054,
+      "learning_rate": 4.9810985566407544e-05,
+      "loss": 0.3058,
+      "num_input_tokens_seen": 3533576,
+      "step": 47
+    },
+    {
+      "epoch": 0.7975077881619937,
+      "grad_norm": 0.06662417948246002,
+      "learning_rate": 4.980286753286195e-05,
+      "loss": 0.4658,
+      "num_input_tokens_seen": 3599744,
+      "step": 48
+    },
+    {
+      "epoch": 0.814122533748702,
+      "grad_norm": 0.05790851265192032,
+      "learning_rate": 4.979457950323945e-05,
+      "loss": 0.3647,
+      "num_input_tokens_seen": 3689520,
+      "step": 49
+    },
+    {
+      "epoch": 0.8307372793354102,
+      "grad_norm": 0.10742159187793732,
+      "learning_rate": 4.9786121534345265e-05,
+      "loss": 0.343,
+      "num_input_tokens_seen": 3751808,
+      "step": 50
+    },
+    {
+      "epoch": 0.8473520249221184,
+      "grad_norm": 0.05565556138753891,
+      "learning_rate": 4.9777493684149375e-05,
+      "loss": 0.3317,
+      "num_input_tokens_seen": 3839096,
+      "step": 51
+    },
+    {
+      "epoch": 0.8639667705088265,
+      "grad_norm": 0.05752381682395935,
+      "learning_rate": 4.976869601178609e-05,
+      "loss": 0.38,
+      "num_input_tokens_seen": 3919824,
+      "step": 52
+    },
+    {
+      "epoch": 0.8805815160955348,
+      "grad_norm": 0.06406434625387192,
+      "learning_rate": 4.975972857755369e-05,
+      "loss": 0.2676,
+      "num_input_tokens_seen": 3989312,
+      "step": 53
+    },
+    {
+      "epoch": 0.897196261682243,
+      "grad_norm": 0.0653691440820694,
+      "learning_rate": 4.975059144291394e-05,
+      "loss": 0.3516,
+      "num_input_tokens_seen": 4060528,
+      "step": 54
+    },
+    {
+      "epoch": 0.9138110072689511,
+      "grad_norm": 0.06272953748703003,
+      "learning_rate": 4.974128467049176e-05,
+      "loss": 0.3004,
+      "num_input_tokens_seen": 4129368,
+      "step": 55
+    },
+    {
+      "epoch": 0.9304257528556594,
+      "grad_norm": 0.08054930716753006,
+      "learning_rate": 4.9731808324074717e-05,
+      "loss": 0.3009,
+      "num_input_tokens_seen": 4175208,
+      "step": 56
+    },
+    {
+      "epoch": 0.9470404984423676,
+      "grad_norm": 0.07523038238286972,
+      "learning_rate": 4.972216246861262e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 4218096,
+      "step": 57
+    },
+    {
+      "epoch": 0.9636552440290758,
+      "grad_norm": 0.07347433269023895,
+      "learning_rate": 4.971234717021709e-05,
+      "loss": 0.3321,
+      "num_input_tokens_seen": 4275968,
+      "step": 58
+    },
+    {
+      "epoch": 0.980269989615784,
+      "grad_norm": 0.05830248445272446,
+      "learning_rate": 4.9702362496161085e-05,
+      "loss": 0.2881,
+      "num_input_tokens_seen": 4346616,
+      "step": 59
+    },
+    {
+      "epoch": 0.9968847352024922,
+      "grad_norm": 0.061629410833120346,
+      "learning_rate": 4.9692208514878444e-05,
+      "loss": 0.2993,
+      "num_input_tokens_seen": 4425064,
+      "step": 60
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.13380740582942963,
+      "learning_rate": 4.968188529596342e-05,
+      "loss": 0.2511,
+      "num_input_tokens_seen": 4435328,
+      "step": 61
+    },
+    {
+      "epoch": 1.0166147455867083,
+      "grad_norm": 0.0726238414645195,
+      "learning_rate": 4.9671392910170185e-05,
+      "loss": 0.3127,
+      "num_input_tokens_seen": 4500104,
+      "step": 62
+    },
+    {
+      "epoch": 1.0332294911734163,
+      "grad_norm": 0.05980083718895912,
+      "learning_rate": 4.966073142941239e-05,
+      "loss": 0.3601,
+      "num_input_tokens_seen": 4581976,
+      "step": 63
+    },
+    {
+      "epoch": 1.0498442367601246,
+      "grad_norm": 0.06445376574993134,
+      "learning_rate": 4.964990092676263e-05,
+      "loss": 0.3049,
+      "num_input_tokens_seen": 4652160,
+      "step": 64
+    },
+    {
+      "epoch": 1.066458982346833,
+      "grad_norm": 0.07824505120515823,
+      "learning_rate": 4.9638901476451946e-05,
+      "loss": 0.3099,
+      "num_input_tokens_seen": 4709368,
+      "step": 65
+    },
+    {
+      "epoch": 1.083073727933541,
+      "grad_norm": 0.058268457651138306,
+      "learning_rate": 4.962773315386935e-05,
+      "loss": 0.3273,
+      "num_input_tokens_seen": 4798256,
+      "step": 66
+    },
+    {
+      "epoch": 1.0996884735202492,
+      "grad_norm": 0.07069691270589828,
+      "learning_rate": 4.961639603556127e-05,
+      "loss": 0.282,
+      "num_input_tokens_seen": 4859200,
+      "step": 67
+    },
+    {
+      "epoch": 1.1163032191069575,
+      "grad_norm": 0.0775996670126915,
+      "learning_rate": 4.960489019923105e-05,
+      "loss": 0.3642,
+      "num_input_tokens_seen": 4925992,
+      "step": 68
+    },
+    {
+      "epoch": 1.1329179646936656,
+      "grad_norm": 0.07044171541929245,
+      "learning_rate": 4.9593215723738404e-05,
+      "loss": 0.2896,
+      "num_input_tokens_seen": 4998808,
+      "step": 69
+    },
+    {
+      "epoch": 1.1495327102803738,
+      "grad_norm": 0.05971802771091461,
+      "learning_rate": 4.958137268909887e-05,
+      "loss": 0.2578,
+      "num_input_tokens_seen": 5089672,
+      "step": 70
+    },
+    {
+      "epoch": 1.1661474558670821,
+      "grad_norm": 0.07145556062459946,
+      "learning_rate": 4.9569361176483286e-05,
+      "loss": 0.3243,
+      "num_input_tokens_seen": 5166744,
+      "step": 71
+    },
+    {
+      "epoch": 1.1827622014537902,
+      "grad_norm": 0.07455787807703018,
+      "learning_rate": 4.9557181268217227e-05,
+      "loss": 0.3949,
+      "num_input_tokens_seen": 5228264,
+      "step": 72
+    },
+    {
+      "epoch": 1.1993769470404985,
+      "grad_norm": 0.055582575500011444,
+      "learning_rate": 4.9544833047780394e-05,
+      "loss": 0.2877,
+      "num_input_tokens_seen": 5338224,
+      "step": 73
+    },
+    {
+      "epoch": 1.2159916926272065,
+      "grad_norm": 0.07675391435623169,
+      "learning_rate": 4.9532316599806124e-05,
+      "loss": 0.3152,
+      "num_input_tokens_seen": 5399848,
+      "step": 74
+    },
+    {
+      "epoch": 1.2326064382139148,
+      "grad_norm": 0.08048644661903381,
+      "learning_rate": 4.951963201008076e-05,
+      "loss": 0.2976,
+      "num_input_tokens_seen": 5468624,
+      "step": 75
+    },
+    {
+      "epoch": 1.249221183800623,
+      "grad_norm": 0.07579060643911362,
+      "learning_rate": 4.9506779365543046e-05,
+      "loss": 0.2982,
+      "num_input_tokens_seen": 5536776,
+      "step": 76
+    },
+    {
+      "epoch": 1.2658359293873311,
+      "grad_norm": 0.07828006893396378,
+      "learning_rate": 4.949375875428357e-05,
+      "loss": 0.3272,
+      "num_input_tokens_seen": 5609296,
+      "step": 77
+    },
+    {
+      "epoch": 1.2824506749740394,
+      "grad_norm": 0.08079098165035248,
+      "learning_rate": 4.9480570265544144e-05,
+      "loss": 0.2768,
+      "num_input_tokens_seen": 5663824,
+      "step": 78
+    },
+    {
+      "epoch": 1.2990654205607477,
+      "grad_norm": 0.07579358667135239,
+      "learning_rate": 4.94672139897172e-05,
+      "loss": 0.318,
+      "num_input_tokens_seen": 5742032,
+      "step": 79
+    },
+    {
+      "epoch": 1.3156801661474558,
+      "grad_norm": 0.07588379085063934,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.3007,
+      "num_input_tokens_seen": 5816864,
+      "step": 80
+    },
+    {
+      "epoch": 1.332294911734164,
+      "grad_norm": 0.08709035068750381,
+      "learning_rate": 4.943999844411977e-05,
+      "loss": 0.2797,
+      "num_input_tokens_seen": 5881624,
+      "step": 81
+    },
+    {
+      "epoch": 1.3489096573208723,
+      "grad_norm": 0.05975884944200516,
+      "learning_rate": 4.94261393608816e-05,
+      "loss": 0.2591,
+      "num_input_tokens_seen": 5970272,
+      "step": 82
+    },
+    {
+      "epoch": 1.3655244029075804,
+      "grad_norm": 0.07372818142175674,
+      "learning_rate": 4.941211286361922e-05,
+      "loss": 0.2687,
+      "num_input_tokens_seen": 6058752,
+      "step": 83
+    },
+    {
+      "epoch": 1.3821391484942886,
+      "grad_norm": 0.09071576595306396,
+      "learning_rate": 4.939791904846869e-05,
+      "loss": 0.2979,
+      "num_input_tokens_seen": 6120064,
+      "step": 84
+    },
+    {
+      "epoch": 1.398753894080997,
+      "grad_norm": 0.0849960595369339,
+      "learning_rate": 4.938355801271282e-05,
+      "loss": 0.2927,
+      "num_input_tokens_seen": 6182072,
+      "step": 85
+    },
+    {
+      "epoch": 1.415368639667705,
+      "grad_norm": 0.08258760720491409,
+      "learning_rate": 4.936902985478055e-05,
+      "loss": 0.295,
+      "num_input_tokens_seen": 6269680,
+      "step": 86
+    },
+    {
+      "epoch": 1.4319833852544133,
+      "grad_norm": 0.0851503536105156,
+      "learning_rate": 4.935433467424624e-05,
+      "loss": 0.2925,
+      "num_input_tokens_seen": 6347424,
+      "step": 87
+    },
+    {
+      "epoch": 1.4485981308411215,
+      "grad_norm": 0.08852345496416092,
+      "learning_rate": 4.933947257182901e-05,
+      "loss": 0.3153,
+      "num_input_tokens_seen": 6412584,
+      "step": 88
+    },
+    {
+      "epoch": 1.4652128764278296,
+      "grad_norm": 0.08184897154569626,
+      "learning_rate": 4.932444364939205e-05,
+      "loss": 0.292,
+      "num_input_tokens_seen": 6482728,
+      "step": 89
+    },
+    {
+      "epoch": 1.4818276220145379,
+      "grad_norm": 0.08270515501499176,
+      "learning_rate": 4.9309248009941914e-05,
+      "loss": 0.3472,
+      "num_input_tokens_seen": 6562104,
+      "step": 90
+    },
+    {
+      "epoch": 1.4984423676012462,
+      "grad_norm": 0.07407747954130173,
+      "learning_rate": 4.929388575762782e-05,
+      "loss": 0.2995,
+      "num_input_tokens_seen": 6656552,
+      "step": 91
+    },
+    {
+      "epoch": 1.5150571131879542,
+      "grad_norm": 0.08710360527038574,
+      "learning_rate": 4.9278356997740904e-05,
+      "loss": 0.2549,
+      "num_input_tokens_seen": 6714184,
+      "step": 92
+    },
+    {
+      "epoch": 1.5316718587746625,
+      "grad_norm": 0.0773790255188942,
+      "learning_rate": 4.9262661836713564e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 6793552,
+      "step": 93
+    },
+    {
+      "epoch": 1.5482866043613708,
+      "grad_norm": 0.1002134457230568,
+      "learning_rate": 4.924680038211867e-05,
+      "loss": 0.2876,
+      "num_input_tokens_seen": 6865256,
+      "step": 94
+    },
+    {
+      "epoch": 1.5649013499480788,
+      "grad_norm": 0.09670394659042358,
+      "learning_rate": 4.9230772742668866e-05,
+      "loss": 0.2846,
+      "num_input_tokens_seen": 6931152,
+      "step": 95
+    },
+    {
+      "epoch": 1.5815160955347871,
+      "grad_norm": 0.08910100907087326,
+      "learning_rate": 4.9214579028215776e-05,
+      "loss": 0.2944,
+      "num_input_tokens_seen": 6998408,
+      "step": 96
+    },
+    {
+      "epoch": 1.5981308411214954,
+      "grad_norm": 0.09202459454536438,
+      "learning_rate": 4.919821934974933e-05,
+      "loss": 0.251,
+      "num_input_tokens_seen": 7053008,
+      "step": 97
+    },
+    {
+      "epoch": 1.6147455867082035,
+      "grad_norm": 0.10218881815671921,
+      "learning_rate": 4.918169381939692e-05,
+      "loss": 0.2851,
+      "num_input_tokens_seen": 7106440,
+      "step": 98
+    },
+    {
+      "epoch": 1.6313603322949117,
+      "grad_norm": 0.09290914982557297,
+      "learning_rate": 4.916500255042268e-05,
+      "loss": 0.2959,
+      "num_input_tokens_seen": 7167032,
+      "step": 99
+    },
+    {
+      "epoch": 1.64797507788162,
+      "grad_norm": 0.07791033387184143,
+      "learning_rate": 4.914814565722671e-05,
+      "loss": 0.2481,
+      "num_input_tokens_seen": 7245720,
+      "step": 100
+    },
+    {
+      "epoch": 1.664589823468328,
+      "grad_norm": 0.08885534107685089,
+      "learning_rate": 4.913112325534426e-05,
+      "loss": 0.3168,
+      "num_input_tokens_seen": 7326320,
+      "step": 101
+    },
+    {
+      "epoch": 1.6812045690550363,
+      "grad_norm": 0.08569750934839249,
+      "learning_rate": 4.9113935461444955e-05,
+      "loss": 0.2805,
+      "num_input_tokens_seen": 7442232,
+      "step": 102
+    },
+    {
+      "epoch": 1.6978193146417446,
+      "grad_norm": 0.1112508773803711,
+      "learning_rate": 4.9096582393332025e-05,
+      "loss": 0.2675,
+      "num_input_tokens_seen": 7502496,
+      "step": 103
+    },
+    {
+      "epoch": 1.7144340602284527,
+      "grad_norm": 0.09654372185468674,
+      "learning_rate": 4.907906416994146e-05,
+      "loss": 0.3038,
+      "num_input_tokens_seen": 7566496,
+      "step": 104
+    },
+    {
+      "epoch": 1.731048805815161,
+      "grad_norm": 0.10022995620965958,
+      "learning_rate": 4.906138091134118e-05,
+      "loss": 0.3639,
+      "num_input_tokens_seen": 7629056,
+      "step": 105
+    },
+    {
+      "epoch": 1.7476635514018692,
+      "grad_norm": 0.08336564153432846,
+      "learning_rate": 4.9043532738730284e-05,
+      "loss": 0.2944,
+      "num_input_tokens_seen": 7706096,
+      "step": 106
+    },
+    {
+      "epoch": 1.7642782969885773,
+      "grad_norm": 0.08539658784866333,
+      "learning_rate": 4.9025519774438136e-05,
+      "loss": 0.2392,
+      "num_input_tokens_seen": 7780072,
+      "step": 107
+    },
+    {
+      "epoch": 1.7808930425752856,
+      "grad_norm": 0.09139693528413773,
+      "learning_rate": 4.900734214192358e-05,
+      "loss": 0.2685,
+      "num_input_tokens_seen": 7857712,
+      "step": 108
+    },
+    {
+      "epoch": 1.7975077881619939,
+      "grad_norm": 0.1043916717171669,
+      "learning_rate": 4.898899996577407e-05,
+      "loss": 0.2513,
+      "num_input_tokens_seen": 7916832,
+      "step": 109
+    },
+    {
+      "epoch": 1.814122533748702,
+      "grad_norm": 0.09203662723302841,
+      "learning_rate": 4.8970493371704826e-05,
+      "loss": 0.2974,
+      "num_input_tokens_seen": 7993056,
+      "step": 110
+    },
+    {
+      "epoch": 1.8307372793354102,
+      "grad_norm": 0.09319474548101425,
+      "learning_rate": 4.8951822486557986e-05,
+      "loss": 0.3096,
+      "num_input_tokens_seen": 8090056,
+      "step": 111
+    },
+    {
+      "epoch": 1.8473520249221185,
+      "grad_norm": 0.10193445533514023,
+      "learning_rate": 4.893298743830168e-05,
+      "loss": 0.2633,
+      "num_input_tokens_seen": 8164808,
+      "step": 112
+    },
+    {
+      "epoch": 1.8639667705088265,
+      "grad_norm": 0.11407948285341263,
+      "learning_rate": 4.891398835602925e-05,
+      "loss": 0.2584,
+      "num_input_tokens_seen": 8223568,
+      "step": 113
+    },
+    {
+      "epoch": 1.8805815160955348,
+      "grad_norm": 0.11977085471153259,
+      "learning_rate": 4.8894825369958255e-05,
+      "loss": 0.2619,
+      "num_input_tokens_seen": 8276160,
+      "step": 114
+    },
+    {
+      "epoch": 1.897196261682243,
+      "grad_norm": 0.10925433784723282,
+      "learning_rate": 4.8875498611429674e-05,
+      "loss": 0.2762,
+      "num_input_tokens_seen": 8354904,
+      "step": 115
+    },
+    {
+      "epoch": 1.9138110072689511,
+      "grad_norm": 0.09673939645290375,
+      "learning_rate": 4.8856008212906925e-05,
+      "loss": 0.3152,
+      "num_input_tokens_seen": 8442584,
+      "step": 116
+    },
+    {
+      "epoch": 1.9304257528556594,
+      "grad_norm": 0.10827789455652237,
+      "learning_rate": 4.8836354307975026e-05,
+      "loss": 0.2759,
+      "num_input_tokens_seen": 8506688,
+      "step": 117
+    },
+    {
+      "epoch": 1.9470404984423677,
+      "grad_norm": 0.08390220254659653,
+      "learning_rate": 4.881653703133966e-05,
+      "loss": 0.2192,
+      "num_input_tokens_seen": 8610712,
+      "step": 118
+    },
+    {
+      "epoch": 1.9636552440290758,
+      "grad_norm": 0.09252211451530457,
+      "learning_rate": 4.87965565188262e-05,
+      "loss": 0.2618,
+      "num_input_tokens_seen": 8692624,
+      "step": 119
+    },
+    {
+      "epoch": 1.980269989615784,
+      "grad_norm": 0.1107102632522583,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.2666,
+      "num_input_tokens_seen": 8772208,
+      "step": 120
+    },
+    {
+      "epoch": 1.9968847352024923,
+      "grad_norm": 0.0917077362537384,
+      "learning_rate": 4.8756106335059646e-05,
+      "loss": 0.253,
+      "num_input_tokens_seen": 8854904,
+      "step": 121
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.2606711685657501,
+      "learning_rate": 4.87356369410476e-05,
+      "loss": 0.235,
+      "num_input_tokens_seen": 8872656,
+      "step": 122
+    },
+    {
+      "epoch": 2.016614745586708,
+      "grad_norm": 0.10363993793725967,
+      "learning_rate": 4.8715004865637614e-05,
+      "loss": 0.266,
+      "num_input_tokens_seen": 8946480,
+      "step": 123
+    },
+    {
+      "epoch": 2.0332294911734166,
+      "grad_norm": 0.09997844696044922,
+      "learning_rate": 4.869421025023965e-05,
+      "loss": 0.2696,
+      "num_input_tokens_seen": 9023328,
+      "step": 124
+    },
+    {
+      "epoch": 2.0498442367601246,
+      "grad_norm": 0.13349319994449615,
+      "learning_rate": 4.867325323737765e-05,
+      "loss": 0.2552,
+      "num_input_tokens_seen": 9074320,
+      "step": 125
+    },
+    {
+      "epoch": 2.0664589823468327,
+      "grad_norm": 0.11201464384794235,
+      "learning_rate": 4.8652133970688636e-05,
+      "loss": 0.2486,
+      "num_input_tokens_seen": 9148784,
+      "step": 126
+    },
+    {
+      "epoch": 2.083073727933541,
+      "grad_norm": 0.10193142294883728,
+      "learning_rate": 4.8630852594921706e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 9246624,
+      "step": 127
+    },
+    {
+      "epoch": 2.0996884735202492,
+      "grad_norm": 0.1305130422115326,
+      "learning_rate": 4.860940925593703e-05,
+      "loss": 0.304,
+      "num_input_tokens_seen": 9328176,
+      "step": 128
+    },
+    {
+      "epoch": 2.1163032191069573,
+      "grad_norm": 0.1137692779302597,
+      "learning_rate": 4.8587804100704845e-05,
+      "loss": 0.2427,
+      "num_input_tokens_seen": 9388936,
+      "step": 129
+    },
+    {
+      "epoch": 2.132917964693666,
+      "grad_norm": 0.12126237154006958,
+      "learning_rate": 4.856603727730447e-05,
+      "loss": 0.2485,
+      "num_input_tokens_seen": 9461664,
+      "step": 130
+    },
+    {
+      "epoch": 2.149532710280374,
+      "grad_norm": 0.11567176878452301,
+      "learning_rate": 4.854410893492326e-05,
+      "loss": 0.2628,
+      "num_input_tokens_seen": 9535000,
+      "step": 131
+    },
+    {
+      "epoch": 2.166147455867082,
+      "grad_norm": 0.1399552971124649,
+      "learning_rate": 4.852201922385564e-05,
+      "loss": 0.2518,
+      "num_input_tokens_seen": 9600296,
+      "step": 132
+    },
+    {
+      "epoch": 2.1827622014537904,
+      "grad_norm": 0.13912151753902435,
+      "learning_rate": 4.8499768295502004e-05,
+      "loss": 0.2429,
+      "num_input_tokens_seen": 9686784,
+      "step": 133
+    },
+    {
+      "epoch": 2.1993769470404985,
+      "grad_norm": 0.11130474507808685,
+      "learning_rate": 4.847735630236773e-05,
+      "loss": 0.2775,
+      "num_input_tokens_seen": 9781112,
+      "step": 134
+    },
+    {
+      "epoch": 2.2159916926272065,
+      "grad_norm": 0.12169156968593597,
+      "learning_rate": 4.8454783398062106e-05,
+      "loss": 0.2439,
+      "num_input_tokens_seen": 9849528,
+      "step": 135
+    },
+    {
+      "epoch": 2.232606438213915,
+      "grad_norm": 0.11766713112592697,
+      "learning_rate": 4.843204973729729e-05,
+      "loss": 0.2538,
+      "num_input_tokens_seen": 9931080,
+      "step": 136
+    },
+    {
+      "epoch": 2.249221183800623,
+      "grad_norm": 0.11854218691587448,
+      "learning_rate": 4.840915547588725e-05,
+      "loss": 0.2782,
+      "num_input_tokens_seen": 10011176,
+      "step": 137
+    },
+    {
+      "epoch": 2.265835929387331,
+      "grad_norm": 0.1340581178665161,
+      "learning_rate": 4.838610077074669e-05,
+      "loss": 0.248,
+      "num_input_tokens_seen": 10084128,
+      "step": 138
+    },
+    {
+      "epoch": 2.2824506749740396,
+      "grad_norm": 0.12075436115264893,
+      "learning_rate": 4.836288577988996e-05,
+      "loss": 0.2582,
+      "num_input_tokens_seen": 10155536,
+      "step": 139
+    },
+    {
+      "epoch": 2.2990654205607477,
+      "grad_norm": 0.10599923878908157,
+      "learning_rate": 4.8339510662430046e-05,
+      "loss": 0.2199,
+      "num_input_tokens_seen": 10251160,
+      "step": 140
+    },
+    {
+      "epoch": 2.3156801661474558,
+      "grad_norm": 0.1117846742272377,
+      "learning_rate": 4.8315975578577355e-05,
+      "loss": 0.2324,
+      "num_input_tokens_seen": 10345864,
+      "step": 141
+    },
+    {
+      "epoch": 2.3322949117341643,
+      "grad_norm": 0.13972057402133942,
+      "learning_rate": 4.8292280689638725e-05,
+      "loss": 0.4072,
+      "num_input_tokens_seen": 10417616,
+      "step": 142
+    },
+    {
+      "epoch": 2.3489096573208723,
+      "grad_norm": 0.13837860524654388,
+      "learning_rate": 4.826842615801628e-05,
+      "loss": 0.2607,
+      "num_input_tokens_seen": 10481816,
+      "step": 143
+    },
+    {
+      "epoch": 2.3655244029075804,
+      "grad_norm": 0.14040137827396393,
+      "learning_rate": 4.8244412147206284e-05,
+      "loss": 0.3094,
+      "num_input_tokens_seen": 10562056,
+      "step": 144
+    },
+    {
+      "epoch": 2.382139148494289,
+      "grad_norm": 0.1393299251794815,
+      "learning_rate": 4.822023882179811e-05,
+      "loss": 0.2407,
+      "num_input_tokens_seen": 10612808,
+      "step": 145
+    },
+    {
+      "epoch": 2.398753894080997,
+      "grad_norm": 0.13878698647022247,
+      "learning_rate": 4.8195906347473e-05,
+      "loss": 0.2481,
+      "num_input_tokens_seen": 10682328,
+      "step": 146
+    },
+    {
+      "epoch": 2.415368639667705,
+      "grad_norm": 0.10430227965116501,
+      "learning_rate": 4.817141489100302e-05,
+      "loss": 0.2528,
+      "num_input_tokens_seen": 10771912,
+      "step": 147
+    },
+    {
+      "epoch": 2.431983385254413,
+      "grad_norm": 0.12963703274726868,
+      "learning_rate": 4.814676462024988e-05,
+      "loss": 0.2739,
+      "num_input_tokens_seen": 10842232,
+      "step": 148
+    },
+    {
+      "epoch": 2.4485981308411215,
+      "grad_norm": 0.13274963200092316,
+      "learning_rate": 4.8121955704163745e-05,
+      "loss": 0.2407,
+      "num_input_tokens_seen": 10902264,
+      "step": 149
+    },
+    {
+      "epoch": 2.4652128764278296,
+      "grad_norm": 0.11079717427492142,
+      "learning_rate": 4.8096988312782174e-05,
+      "loss": 0.2142,
+      "num_input_tokens_seen": 10992744,
+      "step": 150
+    },
+    {
+      "epoch": 2.4818276220145377,
+      "grad_norm": 0.08429212868213654,
+      "learning_rate": 4.8071862617228855e-05,
+      "loss": 0.1428,
+      "num_input_tokens_seen": 11090064,
+      "step": 151
+    },
+    {
+      "epoch": 2.498442367601246,
+      "grad_norm": 0.12903761863708496,
+      "learning_rate": 4.8046578789712515e-05,
+      "loss": 0.2268,
+      "num_input_tokens_seen": 11162864,
+      "step": 152
+    },
+    {
+      "epoch": 2.515057113187954,
+      "grad_norm": 0.14638672769069672,
+      "learning_rate": 4.8021137003525664e-05,
+      "loss": 0.2388,
+      "num_input_tokens_seen": 11224368,
+      "step": 153
+    },
+    {
+      "epoch": 2.5316718587746623,
+      "grad_norm": 0.1372838169336319,
+      "learning_rate": 4.7995537433043446e-05,
+      "loss": 0.2588,
+      "num_input_tokens_seen": 11291056,
+      "step": 154
+    },
+    {
+      "epoch": 2.5482866043613708,
+      "grad_norm": 0.15665481984615326,
+      "learning_rate": 4.796978025372246e-05,
+      "loss": 0.2225,
+      "num_input_tokens_seen": 11345464,
+      "step": 155
+    },
+    {
+      "epoch": 2.564901349948079,
+      "grad_norm": 0.13234855234622955,
+      "learning_rate": 4.794386564209953e-05,
+      "loss": 0.275,
+      "num_input_tokens_seen": 11418912,
+      "step": 156
+    },
+    {
+      "epoch": 2.581516095534787,
+      "grad_norm": 0.13585953414440155,
+      "learning_rate": 4.79177937757905e-05,
+      "loss": 0.2407,
+      "num_input_tokens_seen": 11491216,
+      "step": 157
+    },
+    {
+      "epoch": 2.5981308411214954,
+      "grad_norm": 0.1423913538455963,
+      "learning_rate": 4.7891564833489035e-05,
+      "loss": 0.1971,
+      "num_input_tokens_seen": 11558016,
+      "step": 158
+    },
+    {
+      "epoch": 2.6147455867082035,
+      "grad_norm": 0.13013511896133423,
+      "learning_rate": 4.7865178994965344e-05,
+      "loss": 0.2362,
+      "num_input_tokens_seen": 11630432,
+      "step": 159
+    },
+    {
+      "epoch": 2.6313603322949115,
+      "grad_norm": 0.1587141752243042,
+      "learning_rate": 4.783863644106502e-05,
+      "loss": 0.2252,
+      "num_input_tokens_seen": 11684624,
+      "step": 160
+    },
+    {
+      "epoch": 2.64797507788162,
+      "grad_norm": 0.12592960894107819,
+      "learning_rate": 4.781193735370777e-05,
+      "loss": 0.2506,
+      "num_input_tokens_seen": 11770232,
+      "step": 161
+    },
+    {
+      "epoch": 2.664589823468328,
+      "grad_norm": 0.1583249419927597,
+      "learning_rate": 4.7785081915886134e-05,
+      "loss": 0.2352,
+      "num_input_tokens_seen": 11828360,
+      "step": 162
+    },
+    {
+      "epoch": 2.681204569055036,
+      "grad_norm": 0.14881783723831177,
+      "learning_rate": 4.775807031166428e-05,
+      "loss": 0.2308,
+      "num_input_tokens_seen": 11915944,
+      "step": 163
+    },
+    {
+      "epoch": 2.6978193146417446,
+      "grad_norm": 0.1607823222875595,
+      "learning_rate": 4.773090272617672e-05,
+      "loss": 0.2238,
+      "num_input_tokens_seen": 11981792,
+      "step": 164
+    },
+    {
+      "epoch": 2.7144340602284527,
+      "grad_norm": 0.13583113253116608,
+      "learning_rate": 4.7703579345627035e-05,
+      "loss": 0.3196,
+      "num_input_tokens_seen": 12044024,
+      "step": 165
+    },
+    {
+      "epoch": 2.7310488058151607,
+      "grad_norm": 0.19167298078536987,
+      "learning_rate": 4.7676100357286624e-05,
+      "loss": 0.2745,
+      "num_input_tokens_seen": 12093424,
+      "step": 166
+    },
+    {
+      "epoch": 2.7476635514018692,
+      "grad_norm": 0.130703404545784,
+      "learning_rate": 4.76484659494934e-05,
+      "loss": 0.2285,
+      "num_input_tokens_seen": 12167792,
+      "step": 167
+    },
+    {
+      "epoch": 2.7642782969885773,
+      "grad_norm": 0.14331185817718506,
+      "learning_rate": 4.762067631165049e-05,
+      "loss": 0.2506,
+      "num_input_tokens_seen": 12233712,
+      "step": 168
+    },
+    {
+      "epoch": 2.7808930425752854,
+      "grad_norm": 0.12700341641902924,
+      "learning_rate": 4.7592731634224966e-05,
+      "loss": 0.2052,
+      "num_input_tokens_seen": 12310544,
+      "step": 169
+    },
+    {
+      "epoch": 2.797507788161994,
+      "grad_norm": 0.15118420124053955,
+      "learning_rate": 4.756463210874652e-05,
+      "loss": 0.2309,
+      "num_input_tokens_seen": 12400160,
+      "step": 170
+    },
+    {
+      "epoch": 2.814122533748702,
+      "grad_norm": 0.14001020789146423,
+      "learning_rate": 4.753637792780614e-05,
+      "loss": 0.2544,
+      "num_input_tokens_seen": 12480432,
+      "step": 171
+    },
+    {
+      "epoch": 2.83073727933541,
+      "grad_norm": 0.12076311558485031,
+      "learning_rate": 4.7507969285054845e-05,
+      "loss": 0.2434,
+      "num_input_tokens_seen": 12568064,
+      "step": 172
+    },
+    {
+      "epoch": 2.8473520249221185,
+      "grad_norm": 0.16462342441082,
+      "learning_rate": 4.7479406375202264e-05,
+      "loss": 0.2417,
+      "num_input_tokens_seen": 12647400,
+      "step": 173
+    },
+    {
+      "epoch": 2.8639667705088265,
+      "grad_norm": 0.17294971644878387,
+      "learning_rate": 4.745068939401539e-05,
+      "loss": 0.2121,
+      "num_input_tokens_seen": 12698208,
+      "step": 174
+    },
+    {
+      "epoch": 2.8805815160955346,
+      "grad_norm": 0.16743803024291992,
+      "learning_rate": 4.742181853831721e-05,
+      "loss": 0.2238,
+      "num_input_tokens_seen": 12758528,
+      "step": 175
+    },
+    {
+      "epoch": 2.897196261682243,
+      "grad_norm": 0.14583320915699005,
+      "learning_rate": 4.7392794005985326e-05,
+      "loss": 0.2333,
+      "num_input_tokens_seen": 12837264,
+      "step": 176
+    },
+    {
+      "epoch": 2.913811007268951,
+      "grad_norm": 0.1509270817041397,
+      "learning_rate": 4.7363615995950626e-05,
+      "loss": 0.2179,
+      "num_input_tokens_seen": 12902368,
+      "step": 177
+    },
+    {
+      "epoch": 2.930425752855659,
+      "grad_norm": 0.12910738587379456,
+      "learning_rate": 4.733428470819594e-05,
+      "loss": 0.2144,
+      "num_input_tokens_seen": 12974296,
+      "step": 178
+    },
+    {
+      "epoch": 2.9470404984423677,
+      "grad_norm": 0.142000213265419,
+      "learning_rate": 4.730480034375462e-05,
+      "loss": 0.2413,
+      "num_input_tokens_seen": 13057280,
+      "step": 179
+    },
+    {
+      "epoch": 2.9636552440290758,
+      "grad_norm": 0.131468266248703,
+      "learning_rate": 4.72751631047092e-05,
+      "loss": 0.294,
+      "num_input_tokens_seen": 13158232,
+      "step": 180
+    },
+    {
+      "epoch": 2.980269989615784,
+      "grad_norm": 0.1529342085123062,
+      "learning_rate": 4.7245373194189994e-05,
+      "loss": 0.216,
+      "num_input_tokens_seen": 13229840,
+      "step": 181
+    },
+    {
+      "epoch": 2.9968847352024923,
+      "grad_norm": 0.1573815941810608,
+      "learning_rate": 4.7215430816373726e-05,
+      "loss": 0.2384,
+      "num_input_tokens_seen": 13296520,
+      "step": 182
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.2532118558883667,
+      "learning_rate": 4.718533617648209e-05,
+      "loss": 0.1459,
+      "num_input_tokens_seen": 13309672,
+      "step": 183
+    },
+    {
+      "epoch": 3.016614745586708,
+      "grad_norm": 0.16963432729244232,
+      "learning_rate": 4.715508948078037e-05,
+      "loss": 0.1985,
+      "num_input_tokens_seen": 13371544,
+      "step": 184
+    },
+    {
+      "epoch": 3.0332294911734166,
+      "grad_norm": 0.18877384066581726,
+      "learning_rate": 4.712469093657605e-05,
+      "loss": 0.1856,
+      "num_input_tokens_seen": 13432984,
+      "step": 185
+    },
+    {
+      "epoch": 3.0498442367601246,
+      "grad_norm": 0.14922884106636047,
+      "learning_rate": 4.709414075221734e-05,
+      "loss": 0.2385,
+      "num_input_tokens_seen": 13500016,
+      "step": 186
+    },
+    {
+      "epoch": 3.0664589823468327,
+      "grad_norm": 0.2028326541185379,
+      "learning_rate": 4.706343913709178e-05,
+      "loss": 0.2227,
+      "num_input_tokens_seen": 13579672,
+      "step": 187
+    },
+    {
+      "epoch": 3.083073727933541,
+      "grad_norm": 0.19964616000652313,
+      "learning_rate": 4.70325863016248e-05,
+      "loss": 0.2045,
+      "num_input_tokens_seen": 13630704,
+      "step": 188
+    },
+    {
+      "epoch": 3.0996884735202492,
+      "grad_norm": 0.1594657599925995,
+      "learning_rate": 4.7001582457278304e-05,
+      "loss": 0.2648,
+      "num_input_tokens_seen": 13695472,
+      "step": 189
+    },
+    {
+      "epoch": 3.1163032191069573,
+      "grad_norm": 0.16952532529830933,
+      "learning_rate": 4.697042781654913e-05,
+      "loss": 0.22,
+      "num_input_tokens_seen": 13767792,
+      "step": 190
+    },
+    {
+      "epoch": 3.132917964693666,
+      "grad_norm": 0.16775831580162048,
+      "learning_rate": 4.693912259296773e-05,
+      "loss": 0.2667,
+      "num_input_tokens_seen": 13857352,
+      "step": 191
+    },
+    {
+      "epoch": 3.149532710280374,
+      "grad_norm": 0.15529580414295197,
+      "learning_rate": 4.690766700109659e-05,
+      "loss": 0.2154,
+      "num_input_tokens_seen": 13939928,
+      "step": 192
+    },
+    {
+      "epoch": 3.166147455867082,
+      "grad_norm": 0.1619848757982254,
+      "learning_rate": 4.687606125652882e-05,
+      "loss": 0.1963,
+      "num_input_tokens_seen": 14017936,
+      "step": 193
+    },
+    {
+      "epoch": 3.1827622014537904,
+      "grad_norm": 0.18066684901714325,
+      "learning_rate": 4.684430557588664e-05,
+      "loss": 0.1862,
+      "num_input_tokens_seen": 14074176,
+      "step": 194
+    },
+    {
+      "epoch": 3.1993769470404985,
+      "grad_norm": 0.16520777344703674,
+      "learning_rate": 4.681240017681993e-05,
+      "loss": 0.2576,
+      "num_input_tokens_seen": 14167656,
+      "step": 195
+    },
+    {
+      "epoch": 3.2159916926272065,
+      "grad_norm": 0.15385325253009796,
+      "learning_rate": 4.678034527800474e-05,
+      "loss": 0.1813,
+      "num_input_tokens_seen": 14235800,
+      "step": 196
+    },
+    {
+      "epoch": 3.232606438213915,
+      "grad_norm": 0.16897696256637573,
+      "learning_rate": 4.674814109914174e-05,
+      "loss": 0.1741,
+      "num_input_tokens_seen": 14301272,
+      "step": 197
+    },
+    {
+      "epoch": 3.249221183800623,
+      "grad_norm": 0.19556447863578796,
+      "learning_rate": 4.671578786095478e-05,
+      "loss": 0.2186,
+      "num_input_tokens_seen": 14347352,
+      "step": 198
+    },
+    {
+      "epoch": 3.265835929387331,
+      "grad_norm": 0.17333142459392548,
+      "learning_rate": 4.668328578518933e-05,
+      "loss": 0.2892,
+      "num_input_tokens_seen": 14434600,
+      "step": 199
+    },
+    {
+      "epoch": 3.2824506749740396,
+      "grad_norm": 0.20295488834381104,
+      "learning_rate": 4.665063509461097e-05,
+      "loss": 0.2014,
+      "num_input_tokens_seen": 14484104,
+      "step": 200
+    },
+    {
+      "epoch": 3.2990654205607477,
+      "grad_norm": 0.1597638726234436,
+      "learning_rate": 4.661783601300388e-05,
+      "loss": 0.2158,
+      "num_input_tokens_seen": 14567152,
+      "step": 201
+    },
+    {
+      "epoch": 3.3156801661474558,
+      "grad_norm": 0.19849488139152527,
+      "learning_rate": 4.6584888765169296e-05,
+      "loss": 0.2578,
+      "num_input_tokens_seen": 14647040,
+      "step": 202
+    },
+    {
+      "epoch": 3.3322949117341643,
+      "grad_norm": 0.1508200466632843,
+      "learning_rate": 4.6551793576923964e-05,
+      "loss": 0.2213,
+      "num_input_tokens_seen": 14738216,
+      "step": 203
+    },
+    {
+      "epoch": 3.3489096573208723,
+      "grad_norm": 0.1687687337398529,
+      "learning_rate": 4.65185506750986e-05,
+      "loss": 0.1828,
+      "num_input_tokens_seen": 14811216,
+      "step": 204
+    },
+    {
+      "epoch": 3.3655244029075804,
+      "grad_norm": 0.16587376594543457,
+      "learning_rate": 4.648516028753632e-05,
+      "loss": 0.1619,
+      "num_input_tokens_seen": 14885992,
+      "step": 205
+    },
+    {
+      "epoch": 3.382139148494289,
+      "grad_norm": 0.16600169241428375,
+      "learning_rate": 4.645162264309112e-05,
+      "loss": 0.2438,
+      "num_input_tokens_seen": 14961984,
+      "step": 206
+    },
+    {
+      "epoch": 3.398753894080997,
+      "grad_norm": 0.1877703070640564,
+      "learning_rate": 4.6417937971626245e-05,
+      "loss": 0.1771,
+      "num_input_tokens_seen": 15021240,
+      "step": 207
+    },
+    {
+      "epoch": 3.415368639667705,
+      "grad_norm": 0.20105206966400146,
+      "learning_rate": 4.638410650401267e-05,
+      "loss": 0.1742,
+      "num_input_tokens_seen": 15092016,
+      "step": 208
+    },
+    {
+      "epoch": 3.431983385254413,
+      "grad_norm": 0.12934140861034393,
+      "learning_rate": 4.635012847212748e-05,
+      "loss": 0.1725,
+      "num_input_tokens_seen": 15198192,
+      "step": 209
+    },
+    {
+      "epoch": 3.4485981308411215,
+      "grad_norm": 0.18388882279396057,
+      "learning_rate": 4.6316004108852305e-05,
+      "loss": 0.186,
+      "num_input_tokens_seen": 15258432,
+      "step": 210
+    },
+    {
+      "epoch": 3.4652128764278296,
+      "grad_norm": 0.1766858547925949,
+      "learning_rate": 4.628173364807171e-05,
+      "loss": 0.2166,
+      "num_input_tokens_seen": 15329600,
+      "step": 211
+    },
+    {
+      "epoch": 3.4818276220145377,
+      "grad_norm": 0.16214998066425323,
+      "learning_rate": 4.6247317324671605e-05,
+      "loss": 0.2038,
+      "num_input_tokens_seen": 15407920,
+      "step": 212
+    },
+    {
+      "epoch": 3.498442367601246,
+      "grad_norm": 0.16933797299861908,
+      "learning_rate": 4.6212755374537596e-05,
+      "loss": 0.2017,
+      "num_input_tokens_seen": 15479640,
+      "step": 213
+    },
+    {
+      "epoch": 3.515057113187954,
+      "grad_norm": 0.19472749531269073,
+      "learning_rate": 4.617804803455344e-05,
+      "loss": 0.2048,
+      "num_input_tokens_seen": 15561960,
+      "step": 214
+    },
+    {
+      "epoch": 3.5316718587746623,
+      "grad_norm": 0.33335182070732117,
+      "learning_rate": 4.614319554259934e-05,
+      "loss": 0.2358,
+      "num_input_tokens_seen": 15641440,
+      "step": 215
+    },
+    {
+      "epoch": 3.5482866043613708,
+      "grad_norm": 0.19587557017803192,
+      "learning_rate": 4.610819813755038e-05,
+      "loss": 0.2374,
+      "num_input_tokens_seen": 15728872,
+      "step": 216
+    },
+    {
+      "epoch": 3.564901349948079,
+      "grad_norm": 0.19063518941402435,
+      "learning_rate": 4.607305605927487e-05,
+      "loss": 0.1919,
+      "num_input_tokens_seen": 15798112,
+      "step": 217
+    },
+    {
+      "epoch": 3.581516095534787,
+      "grad_norm": 0.19598323106765747,
+      "learning_rate": 4.6037769548632656e-05,
+      "loss": 0.2583,
+      "num_input_tokens_seen": 15865936,
+      "step": 218
+    },
+    {
+      "epoch": 3.5981308411214954,
+      "grad_norm": 0.18066690862178802,
+      "learning_rate": 4.600233884747355e-05,
+      "loss": 0.2337,
+      "num_input_tokens_seen": 15941368,
+      "step": 219
+    },
+    {
+      "epoch": 3.6147455867082035,
+      "grad_norm": 0.16981899738311768,
+      "learning_rate": 4.5966764198635606e-05,
+      "loss": 0.1818,
+      "num_input_tokens_seen": 16028208,
+      "step": 220
+    },
+    {
+      "epoch": 3.6313603322949115,
+      "grad_norm": 0.180410236120224,
+      "learning_rate": 4.5931045845943474e-05,
+      "loss": 0.1646,
+      "num_input_tokens_seen": 16104408,
+      "step": 221
+    },
+    {
+      "epoch": 3.64797507788162,
+      "grad_norm": 0.19180680811405182,
+      "learning_rate": 4.5895184034206765e-05,
+      "loss": 0.3263,
+      "num_input_tokens_seen": 16156800,
+      "step": 222
+    },
+    {
+      "epoch": 3.664589823468328,
+      "grad_norm": 0.16119280457496643,
+      "learning_rate": 4.585917900921829e-05,
+      "loss": 0.2636,
+      "num_input_tokens_seen": 16256712,
+      "step": 223
+    },
+    {
+      "epoch": 3.681204569055036,
+      "grad_norm": 0.18559172749519348,
+      "learning_rate": 4.5823031017752485e-05,
+      "loss": 0.1759,
+      "num_input_tokens_seen": 16330344,
+      "step": 224
+    },
+    {
+      "epoch": 3.6978193146417446,
+      "grad_norm": 0.17767880856990814,
+      "learning_rate": 4.5786740307563636e-05,
+      "loss": 0.196,
+      "num_input_tokens_seen": 16399792,
+      "step": 225
+    },
+    {
+      "epoch": 3.7144340602284527,
+      "grad_norm": 0.17806987464427948,
+      "learning_rate": 4.575030712738419e-05,
+      "loss": 0.186,
+      "num_input_tokens_seen": 16466368,
+      "step": 226
+    },
+    {
+      "epoch": 3.7310488058151607,
+      "grad_norm": 0.1952792853116989,
+      "learning_rate": 4.571373172692309e-05,
+      "loss": 0.1789,
+      "num_input_tokens_seen": 16530976,
+      "step": 227
+    },
+    {
+      "epoch": 3.7476635514018692,
+      "grad_norm": 0.1774374544620514,
+      "learning_rate": 4.567701435686404e-05,
+      "loss": 0.1929,
+      "num_input_tokens_seen": 16600216,
+      "step": 228
+    },
+    {
+      "epoch": 3.7642782969885773,
+      "grad_norm": 0.18798600137233734,
+      "learning_rate": 4.5640155268863796e-05,
+      "loss": 0.2268,
+      "num_input_tokens_seen": 16673192,
+      "step": 229
+    },
+    {
+      "epoch": 3.7808930425752854,
+      "grad_norm": 0.2022520750761032,
+      "learning_rate": 4.5603154715550386e-05,
+      "loss": 0.1716,
+      "num_input_tokens_seen": 16739912,
+      "step": 230
+    },
+    {
+      "epoch": 3.797507788161994,
+      "grad_norm": 0.15170948207378387,
+      "learning_rate": 4.55660129505215e-05,
+      "loss": 0.1844,
+      "num_input_tokens_seen": 16834632,
+      "step": 231
+    },
+    {
+      "epoch": 3.814122533748702,
+      "grad_norm": 0.16655084490776062,
+      "learning_rate": 4.5528730228342605e-05,
+      "loss": 0.1899,
+      "num_input_tokens_seen": 16914728,
+      "step": 232
+    },
+    {
+      "epoch": 3.83073727933541,
+      "grad_norm": 0.19025221467018127,
+      "learning_rate": 4.549130680454532e-05,
+      "loss": 0.2214,
+      "num_input_tokens_seen": 17014304,
+      "step": 233
+    },
+    {
+      "epoch": 3.8473520249221185,
+      "grad_norm": 0.17126557230949402,
+      "learning_rate": 4.545374293562559e-05,
+      "loss": 0.2062,
+      "num_input_tokens_seen": 17106664,
+      "step": 234
+    },
+    {
+      "epoch": 3.8639667705088265,
+      "grad_norm": 0.16162410378456116,
+      "learning_rate": 4.541603887904198e-05,
+      "loss": 0.2016,
+      "num_input_tokens_seen": 17193744,
+      "step": 235
+    },
+    {
+      "epoch": 3.8805815160955346,
+      "grad_norm": 0.2067136913537979,
+      "learning_rate": 4.537819489321386e-05,
+      "loss": 0.1992,
+      "num_input_tokens_seen": 17254656,
+      "step": 236
+    },
+    {
+      "epoch": 3.897196261682243,
+      "grad_norm": 0.200433611869812,
+      "learning_rate": 4.534021123751968e-05,
+      "loss": 0.1961,
+      "num_input_tokens_seen": 17325896,
+      "step": 237
+    },
+    {
+      "epoch": 3.913811007268951,
+      "grad_norm": 0.2062034010887146,
+      "learning_rate": 4.5302088172295156e-05,
+      "loss": 0.2302,
+      "num_input_tokens_seen": 17394424,
+      "step": 238
+    },
+    {
+      "epoch": 3.930425752855659,
+      "grad_norm": 0.1928798407316208,
+      "learning_rate": 4.526382595883152e-05,
+      "loss": 0.1846,
+      "num_input_tokens_seen": 17456352,
+      "step": 239
+    },
+    {
+      "epoch": 3.9470404984423677,
+      "grad_norm": 0.2011859118938446,
+      "learning_rate": 4.522542485937369e-05,
+      "loss": 0.1879,
+      "num_input_tokens_seen": 17519168,
+      "step": 240
+    },
+    {
+      "epoch": 3.9636552440290758,
+      "grad_norm": 0.20441657304763794,
+      "learning_rate": 4.51868851371185e-05,
+      "loss": 0.206,
+      "num_input_tokens_seen": 17585144,
+      "step": 241
+    },
+    {
+      "epoch": 3.980269989615784,
+      "grad_norm": 0.18314018845558167,
+      "learning_rate": 4.5148207056212896e-05,
+      "loss": 0.1676,
+      "num_input_tokens_seen": 17662024,
+      "step": 242
+    },
+    {
+      "epoch": 3.9968847352024923,
+      "grad_norm": 0.21530692279338837,
+      "learning_rate": 4.5109390881752114e-05,
+      "loss": 0.1961,
+      "num_input_tokens_seen": 17724360,
+      "step": 243
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.38912200927734375,
+      "learning_rate": 4.5070436879777865e-05,
+      "loss": 0.185,
+      "num_input_tokens_seen": 17746200,
+      "step": 244
+    },
+    {
+      "epoch": 4.0166147455867085,
+      "grad_norm": 0.15166164934635162,
+      "learning_rate": 4.503134531727652e-05,
+      "loss": 0.1674,
+      "num_input_tokens_seen": 17830760,
+      "step": 245
+    },
+    {
+      "epoch": 4.033229491173416,
+      "grad_norm": 0.1999833583831787,
+      "learning_rate": 4.499211646217727e-05,
+      "loss": 0.1739,
+      "num_input_tokens_seen": 17903840,
+      "step": 246
+    },
+    {
+      "epoch": 4.049844236760125,
+      "grad_norm": 0.2024000585079193,
+      "learning_rate": 4.495275058335029e-05,
+      "loss": 0.1753,
+      "num_input_tokens_seen": 17990448,
+      "step": 247
+    },
+    {
+      "epoch": 4.066458982346833,
+      "grad_norm": 0.22637376189231873,
+      "learning_rate": 4.491324795060491e-05,
+      "loss": 0.1896,
+      "num_input_tokens_seen": 18069520,
+      "step": 248
+    },
+    {
+      "epoch": 4.083073727933541,
+      "grad_norm": 0.24361123144626617,
+      "learning_rate": 4.487360883468775e-05,
+      "loss": 0.1688,
+      "num_input_tokens_seen": 18129128,
+      "step": 249
+    },
+    {
+      "epoch": 4.099688473520249,
+      "grad_norm": 0.21949416399002075,
+      "learning_rate": 4.4833833507280884e-05,
+      "loss": 0.1928,
+      "num_input_tokens_seen": 18202472,
+      "step": 250
+    },
+    {
+      "epoch": 4.116303219106958,
+      "grad_norm": 0.22039519250392914,
+      "learning_rate": 4.4793922240999933e-05,
+      "loss": 0.1737,
+      "num_input_tokens_seen": 18267232,
+      "step": 251
+    },
+    {
+      "epoch": 4.132917964693665,
+      "grad_norm": 0.23173294961452484,
+      "learning_rate": 4.4753875309392266e-05,
+      "loss": 0.1883,
+      "num_input_tokens_seen": 18325216,
+      "step": 252
+    },
+    {
+      "epoch": 4.149532710280374,
+      "grad_norm": 0.24100351333618164,
+      "learning_rate": 4.471369298693505e-05,
+      "loss": 0.2042,
+      "num_input_tokens_seen": 18406184,
+      "step": 253
+    },
+    {
+      "epoch": 4.166147455867082,
+      "grad_norm": 0.1888919323682785,
+      "learning_rate": 4.467337554903344e-05,
+      "loss": 0.1656,
+      "num_input_tokens_seen": 18481056,
+      "step": 254
+    },
+    {
+      "epoch": 4.18276220145379,
+      "grad_norm": 0.17849119007587433,
+      "learning_rate": 4.463292327201862e-05,
+      "loss": 0.1454,
+      "num_input_tokens_seen": 18554864,
+      "step": 255
+    },
+    {
+      "epoch": 4.1993769470404985,
+      "grad_norm": 0.24600732326507568,
+      "learning_rate": 4.4592336433146e-05,
+      "loss": 0.2039,
+      "num_input_tokens_seen": 18612120,
+      "step": 256
+    },
+    {
+      "epoch": 4.215991692627207,
+      "grad_norm": 0.23695628345012665,
+      "learning_rate": 4.4551615310593195e-05,
+      "loss": 0.2112,
+      "num_input_tokens_seen": 18710408,
+      "step": 257
+    },
+    {
+      "epoch": 4.232606438213915,
+      "grad_norm": 0.2511826753616333,
+      "learning_rate": 4.451076018345825e-05,
+      "loss": 0.1831,
+      "num_input_tokens_seen": 18769400,
+      "step": 258
+    },
+    {
+      "epoch": 4.249221183800623,
+      "grad_norm": 0.1971820890903473,
+      "learning_rate": 4.4469771331757604e-05,
+      "loss": 0.1722,
+      "num_input_tokens_seen": 18849704,
+      "step": 259
+    },
+    {
+      "epoch": 4.265835929387332,
+      "grad_norm": 0.23203876614570618,
+      "learning_rate": 4.442864903642428e-05,
+      "loss": 0.1981,
+      "num_input_tokens_seen": 18943328,
+      "step": 260
+    },
+    {
+      "epoch": 4.282450674974039,
+      "grad_norm": 0.23434185981750488,
+      "learning_rate": 4.4387393579305865e-05,
+      "loss": 0.2014,
+      "num_input_tokens_seen": 19022536,
+      "step": 261
+    },
+    {
+      "epoch": 4.299065420560748,
+      "grad_norm": 0.2373885214328766,
+      "learning_rate": 4.434600524316266e-05,
+      "loss": 0.167,
+      "num_input_tokens_seen": 19089200,
+      "step": 262
+    },
+    {
+      "epoch": 4.315680166147456,
+      "grad_norm": 0.20130722224712372,
+      "learning_rate": 4.430448431166567e-05,
+      "loss": 0.2747,
+      "num_input_tokens_seen": 19171216,
+      "step": 263
+    },
+    {
+      "epoch": 4.332294911734164,
+      "grad_norm": 0.17947593331336975,
+      "learning_rate": 4.426283106939474e-05,
+      "loss": 0.1508,
+      "num_input_tokens_seen": 19271872,
+      "step": 264
+    },
+    {
+      "epoch": 4.348909657320872,
+      "grad_norm": 0.23321041464805603,
+      "learning_rate": 4.4221045801836494e-05,
+      "loss": 0.2585,
+      "num_input_tokens_seen": 19342984,
+      "step": 265
+    },
+    {
+      "epoch": 4.365524402907581,
+      "grad_norm": 0.26581740379333496,
+      "learning_rate": 4.41791287953825e-05,
+      "loss": 0.1789,
+      "num_input_tokens_seen": 19391640,
+      "step": 266
+    },
+    {
+      "epoch": 4.382139148494288,
+      "grad_norm": 0.20715415477752686,
+      "learning_rate": 4.4137080337327205e-05,
+      "loss": 0.1953,
+      "num_input_tokens_seen": 19463232,
+      "step": 267
+    },
+    {
+      "epoch": 4.398753894080997,
+      "grad_norm": 0.20641866326332092,
+      "learning_rate": 4.4094900715866064e-05,
+      "loss": 0.1752,
+      "num_input_tokens_seen": 19523728,
+      "step": 268
+    },
+    {
+      "epoch": 4.415368639667705,
+      "grad_norm": 0.23343385756015778,
+      "learning_rate": 4.4052590220093446e-05,
+      "loss": 0.1904,
+      "num_input_tokens_seen": 19598960,
+      "step": 269
+    },
+    {
+      "epoch": 4.431983385254413,
+      "grad_norm": 0.20117436349391937,
+      "learning_rate": 4.401014914000078e-05,
+      "loss": 0.1801,
+      "num_input_tokens_seen": 19666136,
+      "step": 270
+    },
+    {
+      "epoch": 4.4485981308411215,
+      "grad_norm": 0.24009813368320465,
+      "learning_rate": 4.3967577766474455e-05,
+      "loss": 0.1798,
+      "num_input_tokens_seen": 19728600,
+      "step": 271
+    },
+    {
+      "epoch": 4.46521287642783,
+      "grad_norm": 0.2242031991481781,
+      "learning_rate": 4.3924876391293915e-05,
+      "loss": 0.2221,
+      "num_input_tokens_seen": 19801032,
+      "step": 272
+    },
+    {
+      "epoch": 4.481827622014538,
+      "grad_norm": 0.22890391945838928,
+      "learning_rate": 4.3882045307129594e-05,
+      "loss": 0.1906,
+      "num_input_tokens_seen": 19885496,
+      "step": 273
+    },
+    {
+      "epoch": 4.498442367601246,
+      "grad_norm": 0.21996937692165375,
+      "learning_rate": 4.383908480754095e-05,
+      "loss": 0.1775,
+      "num_input_tokens_seen": 19952072,
+      "step": 274
+    },
+    {
+      "epoch": 4.515057113187955,
+      "grad_norm": 0.1860388070344925,
+      "learning_rate": 4.379599518697444e-05,
+      "loss": 0.1593,
+      "num_input_tokens_seen": 20026536,
+      "step": 275
+    },
+    {
+      "epoch": 4.531671858774662,
+      "grad_norm": 0.20987707376480103,
+      "learning_rate": 4.375277674076149e-05,
+      "loss": 0.1409,
+      "num_input_tokens_seen": 20079112,
+      "step": 276
+    },
+    {
+      "epoch": 4.548286604361371,
+      "grad_norm": 0.21347324550151825,
+      "learning_rate": 4.3709429765116504e-05,
+      "loss": 0.2701,
+      "num_input_tokens_seen": 20144264,
+      "step": 277
+    },
+    {
+      "epoch": 4.564901349948079,
+      "grad_norm": 0.27563896775245667,
+      "learning_rate": 4.366595455713479e-05,
+      "loss": 0.1856,
+      "num_input_tokens_seen": 20207568,
+      "step": 278
+    },
+    {
+      "epoch": 4.581516095534787,
+      "grad_norm": 0.21850791573524475,
+      "learning_rate": 4.3622351414790554e-05,
+      "loss": 0.2204,
+      "num_input_tokens_seen": 20292376,
+      "step": 279
+    },
+    {
+      "epoch": 4.598130841121495,
+      "grad_norm": 0.19672711193561554,
+      "learning_rate": 4.357862063693486e-05,
+      "loss": 0.1397,
+      "num_input_tokens_seen": 20383048,
+      "step": 280
+    },
+    {
+      "epoch": 4.614745586708204,
+      "grad_norm": 0.23507343232631683,
+      "learning_rate": 4.353476252329356e-05,
+      "loss": 0.1655,
+      "num_input_tokens_seen": 20463376,
+      "step": 281
+    },
+    {
+      "epoch": 4.6313603322949115,
+      "grad_norm": 0.23900464177131653,
+      "learning_rate": 4.349077737446525e-05,
+      "loss": 0.1511,
+      "num_input_tokens_seen": 20537808,
+      "step": 282
+    },
+    {
+      "epoch": 4.64797507788162,
+      "grad_norm": 0.2231033891439438,
+      "learning_rate": 4.344666549191921e-05,
+      "loss": 0.1783,
+      "num_input_tokens_seen": 20605496,
+      "step": 283
+    },
+    {
+      "epoch": 4.6645898234683285,
+      "grad_norm": 0.20772908627986908,
+      "learning_rate": 4.3402427177993366e-05,
+      "loss": 0.2135,
+      "num_input_tokens_seen": 20692096,
+      "step": 284
+    },
+    {
+      "epoch": 4.681204569055036,
+      "grad_norm": 0.24927115440368652,
+      "learning_rate": 4.335806273589214e-05,
+      "loss": 0.1919,
+      "num_input_tokens_seen": 20762800,
+      "step": 285
+    },
+    {
+      "epoch": 4.697819314641745,
+      "grad_norm": 0.2112305462360382,
+      "learning_rate": 4.3313572469684474e-05,
+      "loss": 0.1546,
+      "num_input_tokens_seen": 20831584,
+      "step": 286
+    },
+    {
+      "epoch": 4.714434060228453,
+      "grad_norm": 0.22639551758766174,
+      "learning_rate": 4.326895668430166e-05,
+      "loss": 0.124,
+      "num_input_tokens_seen": 20897320,
+      "step": 287
+    },
+    {
+      "epoch": 4.731048805815161,
+      "grad_norm": 0.2295934110879898,
+      "learning_rate": 4.3224215685535294e-05,
+      "loss": 0.1639,
+      "num_input_tokens_seen": 20966136,
+      "step": 288
+    },
+    {
+      "epoch": 4.747663551401869,
+      "grad_norm": 0.2341577112674713,
+      "learning_rate": 4.317934978003517e-05,
+      "loss": 0.1584,
+      "num_input_tokens_seen": 21034800,
+      "step": 289
+    },
+    {
+      "epoch": 4.764278296988578,
+      "grad_norm": 0.2542404234409332,
+      "learning_rate": 4.313435927530719e-05,
+      "loss": 0.1918,
+      "num_input_tokens_seen": 21098672,
+      "step": 290
+    },
+    {
+      "epoch": 4.780893042575285,
+      "grad_norm": 0.23311223089694977,
+      "learning_rate": 4.3089244479711236e-05,
+      "loss": 0.1597,
+      "num_input_tokens_seen": 21177632,
+      "step": 291
+    },
+    {
+      "epoch": 4.797507788161994,
+      "grad_norm": 0.2642923593521118,
+      "learning_rate": 4.304400570245906e-05,
+      "loss": 0.1847,
+      "num_input_tokens_seen": 21240896,
+      "step": 292
+    },
+    {
+      "epoch": 4.814122533748702,
+      "grad_norm": 0.18841278553009033,
+      "learning_rate": 4.299864325361217e-05,
+      "loss": 0.1472,
+      "num_input_tokens_seen": 21322984,
+      "step": 293
+    },
+    {
+      "epoch": 4.83073727933541,
+      "grad_norm": 0.22440434992313385,
+      "learning_rate": 4.295315744407972e-05,
+      "loss": 0.1607,
+      "num_input_tokens_seen": 21389128,
+      "step": 294
+    },
+    {
+      "epoch": 4.8473520249221185,
+      "grad_norm": 0.22145289182662964,
+      "learning_rate": 4.290754858561637e-05,
+      "loss": 0.1851,
+      "num_input_tokens_seen": 21469912,
+      "step": 295
+    },
+    {
+      "epoch": 4.863966770508826,
+      "grad_norm": 0.22817087173461914,
+      "learning_rate": 4.2861816990820084e-05,
+      "loss": 0.1531,
+      "num_input_tokens_seen": 21540320,
+      "step": 296
+    },
+    {
+      "epoch": 4.880581516095535,
+      "grad_norm": 0.22014038264751434,
+      "learning_rate": 4.281596297313013e-05,
+      "loss": 0.1815,
+      "num_input_tokens_seen": 21626312,
+      "step": 297
+    },
+    {
+      "epoch": 4.897196261682243,
+      "grad_norm": 0.2234148383140564,
+      "learning_rate": 4.2769986846824815e-05,
+      "loss": 0.1667,
+      "num_input_tokens_seen": 21702792,
+      "step": 298
+    },
+    {
+      "epoch": 4.913811007268951,
+      "grad_norm": 0.2851375341415405,
+      "learning_rate": 4.272388892701934e-05,
+      "loss": 0.1805,
+      "num_input_tokens_seen": 21771880,
+      "step": 299
+    },
+    {
+      "epoch": 4.930425752855659,
+      "grad_norm": 0.2221265286207199,
+      "learning_rate": 4.267766952966369e-05,
+      "loss": 0.1653,
+      "num_input_tokens_seen": 21844024,
+      "step": 300
+    },
+    {
+      "epoch": 4.947040498442368,
+      "grad_norm": 0.20688939094543457,
+      "learning_rate": 4.2631328971540444e-05,
+      "loss": 0.1654,
+      "num_input_tokens_seen": 21925632,
+      "step": 301
+    },
+    {
+      "epoch": 4.963655244029075,
+      "grad_norm": 0.2270977795124054,
+      "learning_rate": 4.2584867570262597e-05,
+      "loss": 0.1774,
+      "num_input_tokens_seen": 21981952,
+      "step": 302
+    },
+    {
+      "epoch": 4.980269989615784,
+      "grad_norm": 0.18652501702308655,
+      "learning_rate": 4.25382856442714e-05,
+      "loss": 0.1452,
+      "num_input_tokens_seen": 22070440,
+      "step": 303
+    },
+    {
+      "epoch": 4.996884735202492,
+      "grad_norm": 0.19792407751083374,
+      "learning_rate": 4.249158351283414e-05,
+      "loss": 0.1806,
+      "num_input_tokens_seen": 22170184,
+      "step": 304
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.5933757424354553,
+      "learning_rate": 4.244476149604201e-05,
+      "loss": 0.2115,
+      "num_input_tokens_seen": 22181856,
+      "step": 305
+    },
+    {
+      "epoch": 5.0166147455867085,
+      "grad_norm": 0.22399979829788208,
+      "learning_rate": 4.2397819914807856e-05,
+      "loss": 0.1614,
+      "num_input_tokens_seen": 22256808,
+      "step": 306
+    },
+    {
+      "epoch": 5.033229491173416,
+      "grad_norm": 0.2595834732055664,
+      "learning_rate": 4.2350759090864046e-05,
+      "loss": 0.1838,
+      "num_input_tokens_seen": 22325224,
+      "step": 307
+    },
+    {
+      "epoch": 5.049844236760125,
+      "grad_norm": 0.188430517911911,
+      "learning_rate": 4.230357934676017e-05,
+      "loss": 0.1421,
+      "num_input_tokens_seen": 22389624,
+      "step": 308
+    },
+    {
+      "epoch": 5.066458982346833,
+      "grad_norm": 0.325431764125824,
+      "learning_rate": 4.225628100586093e-05,
+      "loss": 0.1852,
+      "num_input_tokens_seen": 22463872,
+      "step": 309
+    },
+    {
+      "epoch": 5.083073727933541,
+      "grad_norm": 0.27097174525260925,
+      "learning_rate": 4.220886439234385e-05,
+      "loss": 0.1526,
+      "num_input_tokens_seen": 22515824,
+      "step": 310
+    },
+    {
+      "epoch": 5.099688473520249,
+      "grad_norm": 0.21379193663597107,
+      "learning_rate": 4.2161329831197095e-05,
+      "loss": 0.151,
+      "num_input_tokens_seen": 22602336,
+      "step": 311
+    },
+    {
+      "epoch": 5.116303219106958,
+      "grad_norm": 0.24827998876571655,
+      "learning_rate": 4.211367764821722e-05,
+      "loss": 0.142,
+      "num_input_tokens_seen": 22655176,
+      "step": 312
+    },
+    {
+      "epoch": 5.132917964693665,
+      "grad_norm": 0.26246964931488037,
+      "learning_rate": 4.2065908170006955e-05,
+      "loss": 0.1589,
+      "num_input_tokens_seen": 22728680,
+      "step": 313
+    },
+    {
+      "epoch": 5.149532710280374,
+      "grad_norm": 0.24459198117256165,
+      "learning_rate": 4.201802172397295e-05,
+      "loss": 0.1435,
+      "num_input_tokens_seen": 22806784,
+      "step": 314
+    },
+    {
+      "epoch": 5.166147455867082,
+      "grad_norm": 0.26540517807006836,
+      "learning_rate": 4.197001863832355e-05,
+      "loss": 0.1447,
+      "num_input_tokens_seen": 22880648,
+      "step": 315
+    },
+    {
+      "epoch": 5.18276220145379,
+      "grad_norm": 0.25646644830703735,
+      "learning_rate": 4.192189924206652e-05,
+      "loss": 0.1418,
+      "num_input_tokens_seen": 22953184,
+      "step": 316
+    },
+    {
+      "epoch": 5.1993769470404985,
+      "grad_norm": 0.2358384132385254,
+      "learning_rate": 4.187366386500683e-05,
+      "loss": 0.1845,
+      "num_input_tokens_seen": 23037392,
+      "step": 317
+    },
+    {
+      "epoch": 5.215991692627207,
+      "grad_norm": 0.2270258218050003,
+      "learning_rate": 4.182531283774434e-05,
+      "loss": 0.2668,
+      "num_input_tokens_seen": 23086552,
+      "step": 318
+    },
+    {
+      "epoch": 5.232606438213915,
+      "grad_norm": 0.24396558105945587,
+      "learning_rate": 4.177684649167158e-05,
+      "loss": 0.1567,
+      "num_input_tokens_seen": 23153152,
+      "step": 319
+    },
+    {
+      "epoch": 5.249221183800623,
+      "grad_norm": 0.2542375922203064,
+      "learning_rate": 4.172826515897146e-05,
+      "loss": 0.1617,
+      "num_input_tokens_seen": 23240928,
+      "step": 320
+    },
+    {
+      "epoch": 5.265835929387332,
+      "grad_norm": 0.2268146276473999,
+      "learning_rate": 4.1679569172614996e-05,
+      "loss": 0.1573,
+      "num_input_tokens_seen": 23325912,
+      "step": 321
+    },
+    {
+      "epoch": 5.282450674974039,
+      "grad_norm": 0.26405712962150574,
+      "learning_rate": 4.163075886635902e-05,
+      "loss": 0.1738,
+      "num_input_tokens_seen": 23401952,
+      "step": 322
+    },
+    {
+      "epoch": 5.299065420560748,
+      "grad_norm": 0.2599943280220032,
+      "learning_rate": 4.1581834574743915e-05,
+      "loss": 0.1433,
+      "num_input_tokens_seen": 23463760,
+      "step": 323
+    },
+    {
+      "epoch": 5.315680166147456,
+      "grad_norm": 0.24650180339813232,
+      "learning_rate": 4.1532796633091296e-05,
+      "loss": 0.1511,
+      "num_input_tokens_seen": 23535272,
+      "step": 324
+    },
+    {
+      "epoch": 5.332294911734164,
+      "grad_norm": 0.22445832192897797,
+      "learning_rate": 4.148364537750172e-05,
+      "loss": 0.1296,
+      "num_input_tokens_seen": 23607752,
+      "step": 325
+    },
+    {
+      "epoch": 5.348909657320872,
+      "grad_norm": 0.20715995132923126,
+      "learning_rate": 4.14343811448524e-05,
+      "loss": 0.142,
+      "num_input_tokens_seen": 23674872,
+      "step": 326
+    },
+    {
+      "epoch": 5.365524402907581,
+      "grad_norm": 0.2707969546318054,
+      "learning_rate": 4.138500427279485e-05,
+      "loss": 0.1679,
+      "num_input_tokens_seen": 23736384,
+      "step": 327
+    },
+    {
+      "epoch": 5.382139148494288,
+      "grad_norm": 0.2017841637134552,
+      "learning_rate": 4.133551509975264e-05,
+      "loss": 0.1346,
+      "num_input_tokens_seen": 23835000,
+      "step": 328
+    },
+    {
+      "epoch": 5.398753894080997,
+      "grad_norm": 0.21116195619106293,
+      "learning_rate": 4.128591396491901e-05,
+      "loss": 0.1364,
+      "num_input_tokens_seen": 23912552,
+      "step": 329
+    },
+    {
+      "epoch": 5.415368639667705,
+      "grad_norm": 0.2331131547689438,
+      "learning_rate": 4.123620120825459e-05,
+      "loss": 0.1719,
+      "num_input_tokens_seen": 23987368,
+      "step": 330
+    },
+    {
+      "epoch": 5.431983385254413,
+      "grad_norm": 0.27115845680236816,
+      "learning_rate": 4.118637717048506e-05,
+      "loss": 0.1468,
+      "num_input_tokens_seen": 24050848,
+      "step": 331
+    },
+    {
+      "epoch": 5.4485981308411215,
+      "grad_norm": 0.21654783189296722,
+      "learning_rate": 4.113644219309877e-05,
+      "loss": 0.1418,
+      "num_input_tokens_seen": 24146104,
+      "step": 332
+    },
+    {
+      "epoch": 5.46521287642783,
+      "grad_norm": 0.2783348560333252,
+      "learning_rate": 4.1086396618344476e-05,
+      "loss": 0.1502,
+      "num_input_tokens_seen": 24194184,
+      "step": 333
+    },
+    {
+      "epoch": 5.481827622014538,
+      "grad_norm": 0.23255467414855957,
+      "learning_rate": 4.1036240789228954e-05,
+      "loss": 0.1571,
+      "num_input_tokens_seen": 24275368,
+      "step": 334
+    },
+    {
+      "epoch": 5.498442367601246,
+      "grad_norm": 0.2655453681945801,
+      "learning_rate": 4.098597504951462e-05,
+      "loss": 0.1607,
+      "num_input_tokens_seen": 24329192,
+      "step": 335
+    },
+    {
+      "epoch": 5.515057113187955,
+      "grad_norm": 0.23245719075202942,
+      "learning_rate": 4.093559974371725e-05,
+      "loss": 0.1453,
+      "num_input_tokens_seen": 24426696,
+      "step": 336
+    },
+    {
+      "epoch": 5.531671858774662,
+      "grad_norm": 0.20036327838897705,
+      "learning_rate": 4.088511521710352e-05,
+      "loss": 0.1411,
+      "num_input_tokens_seen": 24514344,
+      "step": 337
+    },
+    {
+      "epoch": 5.548286604361371,
+      "grad_norm": 0.32294195890426636,
+      "learning_rate": 4.083452181568875e-05,
+      "loss": 0.1467,
+      "num_input_tokens_seen": 24584464,
+      "step": 338
+    },
+    {
+      "epoch": 5.564901349948079,
+      "grad_norm": 0.23969624936580658,
+      "learning_rate": 4.0783819886234445e-05,
+      "loss": 0.1304,
+      "num_input_tokens_seen": 24660600,
+      "step": 339
+    },
+    {
+      "epoch": 5.581516095534787,
+      "grad_norm": 0.2539989948272705,
+      "learning_rate": 4.073300977624594e-05,
+      "loss": 0.1374,
+      "num_input_tokens_seen": 24717088,
+      "step": 340
+    },
+    {
+      "epoch": 5.598130841121495,
+      "grad_norm": 0.26608580350875854,
+      "learning_rate": 4.068209183397004e-05,
+      "loss": 0.1519,
+      "num_input_tokens_seen": 24775352,
+      "step": 341
+    },
+    {
+      "epoch": 5.614745586708204,
+      "grad_norm": 0.2161550372838974,
+      "learning_rate": 4.063106640839264e-05,
+      "loss": 0.1409,
+      "num_input_tokens_seen": 24860072,
+      "step": 342
+    },
+    {
+      "epoch": 5.6313603322949115,
+      "grad_norm": 0.22482600808143616,
+      "learning_rate": 4.057993384923626e-05,
+      "loss": 0.1393,
+      "num_input_tokens_seen": 24947856,
+      "step": 343
+    },
+    {
+      "epoch": 5.64797507788162,
+      "grad_norm": 0.2367829829454422,
+      "learning_rate": 4.052869450695776e-05,
+      "loss": 0.1539,
+      "num_input_tokens_seen": 25024992,
+      "step": 344
+    },
+    {
+      "epoch": 5.6645898234683285,
+      "grad_norm": 0.29229775071144104,
+      "learning_rate": 4.047734873274586e-05,
+      "loss": 0.1522,
+      "num_input_tokens_seen": 25092248,
+      "step": 345
+    },
+    {
+      "epoch": 5.681204569055036,
+      "grad_norm": 0.2589828670024872,
+      "learning_rate": 4.042589687851872e-05,
+      "loss": 0.1493,
+      "num_input_tokens_seen": 25170496,
+      "step": 346
+    },
+    {
+      "epoch": 5.697819314641745,
+      "grad_norm": 0.23003339767456055,
+      "learning_rate": 4.037433929692161e-05,
+      "loss": 0.1529,
+      "num_input_tokens_seen": 25268720,
+      "step": 347
+    },
+    {
+      "epoch": 5.714434060228453,
+      "grad_norm": 0.26932114362716675,
+      "learning_rate": 4.0322676341324415e-05,
+      "loss": 0.1499,
+      "num_input_tokens_seen": 25332688,
+      "step": 348
+    },
+    {
+      "epoch": 5.731048805815161,
+      "grad_norm": 0.27059391140937805,
+      "learning_rate": 4.027090836581925e-05,
+      "loss": 0.173,
+      "num_input_tokens_seen": 25413904,
+      "step": 349
+    },
+    {
+      "epoch": 5.747663551401869,
+      "grad_norm": 0.24265804886817932,
+      "learning_rate": 4.021903572521802e-05,
+      "loss": 0.1531,
+      "num_input_tokens_seen": 25503720,
+      "step": 350
+    },
+    {
+      "epoch": 5.764278296988578,
+      "grad_norm": 0.28688696026802063,
+      "learning_rate": 4.0167058775049996e-05,
+      "loss": 0.1615,
+      "num_input_tokens_seen": 25568560,
+      "step": 351
+    },
+    {
+      "epoch": 5.780893042575285,
+      "grad_norm": 0.26201075315475464,
+      "learning_rate": 4.011497787155938e-05,
+      "loss": 0.1452,
+      "num_input_tokens_seen": 25635184,
+      "step": 352
+    },
+    {
+      "epoch": 5.797507788161994,
+      "grad_norm": 0.22841767966747284,
+      "learning_rate": 4.006279337170283e-05,
+      "loss": 0.148,
+      "num_input_tokens_seen": 25719768,
+      "step": 353
+    },
+    {
+      "epoch": 5.814122533748702,
+      "grad_norm": 0.28246188163757324,
+      "learning_rate": 4.0010505633147106e-05,
+      "loss": 0.1446,
+      "num_input_tokens_seen": 25795016,
+      "step": 354
+    },
+    {
+      "epoch": 5.83073727933541,
+      "grad_norm": 0.2533949315547943,
+      "learning_rate": 3.995811501426648e-05,
+      "loss": 0.1297,
+      "num_input_tokens_seen": 25863184,
+      "step": 355
+    },
+    {
+      "epoch": 5.8473520249221185,
+      "grad_norm": 0.25117793679237366,
+      "learning_rate": 3.99056218741404e-05,
+      "loss": 0.1479,
+      "num_input_tokens_seen": 25935752,
+      "step": 356
+    },
+    {
+      "epoch": 5.863966770508826,
+      "grad_norm": 0.2759612202644348,
+      "learning_rate": 3.985302657255097e-05,
+      "loss": 0.1466,
+      "num_input_tokens_seen": 25995760,
+      "step": 357
+    },
+    {
+      "epoch": 5.880581516095535,
+      "grad_norm": 0.24218028783798218,
+      "learning_rate": 3.980032946998049e-05,
+      "loss": 0.1297,
+      "num_input_tokens_seen": 26061240,
+      "step": 358
+    },
+    {
+      "epoch": 5.897196261682243,
+      "grad_norm": 0.2628185749053955,
+      "learning_rate": 3.974753092760901e-05,
+      "loss": 0.1421,
+      "num_input_tokens_seen": 26131024,
+      "step": 359
+    },
+    {
+      "epoch": 5.913811007268951,
+      "grad_norm": 0.18742328882217407,
+      "learning_rate": 3.969463130731183e-05,
+      "loss": 0.1737,
+      "num_input_tokens_seen": 26233672,
+      "step": 360
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 1200,
+  "num_input_tokens_seen": 26233672,
+  "num_train_epochs": 20,
+  "save_steps": 60,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.2131164458254336e+18,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-420/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-420/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-420/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 17500,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-420/trainer_state.json

@@ -0,0 +1,3393 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 6.897196261682243,
+  "eval_steps": 500,
+  "global_step": 420,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.016614745586708203,
+      "grad_norm": 0.050998032093048096,
+      "learning_rate": 4.999991432639962e-05,
+      "loss": 0.5487,
+      "num_input_tokens_seen": 70408,
+      "step": 1
+    },
+    {
+      "epoch": 0.033229491173416406,
+      "grad_norm": 0.049370743334293365,
+      "learning_rate": 4.999965730618567e-05,
+      "loss": 0.4981,
+      "num_input_tokens_seen": 139640,
+      "step": 2
+    },
+    {
+      "epoch": 0.04984423676012461,
+      "grad_norm": 0.05077400803565979,
+      "learning_rate": 4.9999228941119745e-05,
+      "loss": 0.5505,
+      "num_input_tokens_seen": 223656,
+      "step": 3
+    },
+    {
+      "epoch": 0.06645898234683281,
+      "grad_norm": 0.04397282376885414,
+      "learning_rate": 4.999862923413781e-05,
+      "loss": 0.504,
+      "num_input_tokens_seen": 300688,
+      "step": 4
+    },
+    {
+      "epoch": 0.08307372793354102,
+      "grad_norm": 0.05225864797830582,
+      "learning_rate": 4.999785818935018e-05,
+      "loss": 0.4925,
+      "num_input_tokens_seen": 366368,
+      "step": 5
+    },
+    {
+      "epoch": 0.09968847352024922,
+      "grad_norm": 0.049482282251119614,
+      "learning_rate": 4.999691581204152e-05,
+      "loss": 0.4771,
+      "num_input_tokens_seen": 445808,
+      "step": 6
+    },
+    {
+      "epoch": 0.11630321910695743,
+      "grad_norm": 0.05594080314040184,
+      "learning_rate": 4.9995802108670775e-05,
+      "loss": 0.4986,
+      "num_input_tokens_seen": 522800,
+      "step": 7
+    },
+    {
+      "epoch": 0.13291796469366562,
+      "grad_norm": 0.051852282136678696,
+      "learning_rate": 4.999451708687114e-05,
+      "loss": 0.5171,
+      "num_input_tokens_seen": 599608,
+      "step": 8
+    },
+    {
+      "epoch": 0.14953271028037382,
+      "grad_norm": 0.045517683029174805,
+      "learning_rate": 4.9993060755450015e-05,
+      "loss": 0.5669,
+      "num_input_tokens_seen": 681424,
+      "step": 9
+    },
+    {
+      "epoch": 0.16614745586708204,
+      "grad_norm": 0.044325754046440125,
+      "learning_rate": 4.999143312438893e-05,
+      "loss": 0.4218,
+      "num_input_tokens_seen": 756744,
+      "step": 10
+    },
+    {
+      "epoch": 0.18276220145379024,
+      "grad_norm": 0.04328459873795509,
+      "learning_rate": 4.998963420484349e-05,
+      "loss": 0.434,
+      "num_input_tokens_seen": 842576,
+      "step": 11
+    },
+    {
+      "epoch": 0.19937694704049844,
+      "grad_norm": 0.04725787043571472,
+      "learning_rate": 4.998766400914329e-05,
+      "loss": 0.4287,
+      "num_input_tokens_seen": 917232,
+      "step": 12
+    },
+    {
+      "epoch": 0.21599169262720663,
+      "grad_norm": 0.03806879371404648,
+      "learning_rate": 4.9985522550791825e-05,
+      "loss": 0.3454,
+      "num_input_tokens_seen": 1006800,
+      "step": 13
+    },
+    {
+      "epoch": 0.23260643821391486,
+      "grad_norm": 0.05201176926493645,
+      "learning_rate": 4.998320984446641e-05,
+      "loss": 0.436,
+      "num_input_tokens_seen": 1085824,
+      "step": 14
+    },
+    {
+      "epoch": 0.24922118380062305,
+      "grad_norm": 0.047955628484487534,
+      "learning_rate": 4.9980725906018074e-05,
+      "loss": 0.4625,
+      "num_input_tokens_seen": 1164160,
+      "step": 15
+    },
+    {
+      "epoch": 0.26583592938733125,
+      "grad_norm": 0.05529098957777023,
+      "learning_rate": 4.997807075247146e-05,
+      "loss": 0.5035,
+      "num_input_tokens_seen": 1242264,
+      "step": 16
+    },
+    {
+      "epoch": 0.2824506749740395,
+      "grad_norm": 0.04751162976026535,
+      "learning_rate": 4.997524440202469e-05,
+      "loss": 0.4354,
+      "num_input_tokens_seen": 1325904,
+      "step": 17
+    },
+    {
+      "epoch": 0.29906542056074764,
+      "grad_norm": 0.06726882606744766,
+      "learning_rate": 4.9972246874049254e-05,
+      "loss": 0.5439,
+      "num_input_tokens_seen": 1385632,
+      "step": 18
+    },
+    {
+      "epoch": 0.31568016614745587,
+      "grad_norm": 0.05245920270681381,
+      "learning_rate": 4.996907818908987e-05,
+      "loss": 0.3727,
+      "num_input_tokens_seen": 1470632,
+      "step": 19
+    },
+    {
+      "epoch": 0.3322949117341641,
+      "grad_norm": 0.05745376646518707,
+      "learning_rate": 4.996573836886435e-05,
+      "loss": 0.4894,
+      "num_input_tokens_seen": 1547536,
+      "step": 20
+    },
+    {
+      "epoch": 0.34890965732087226,
+      "grad_norm": 0.056607529520988464,
+      "learning_rate": 4.9962227436263453e-05,
+      "loss": 0.3846,
+      "num_input_tokens_seen": 1615528,
+      "step": 21
+    },
+    {
+      "epoch": 0.3655244029075805,
+      "grad_norm": 0.06150667741894722,
+      "learning_rate": 4.995854541535071e-05,
+      "loss": 0.4362,
+      "num_input_tokens_seen": 1694352,
+      "step": 22
+    },
+    {
+      "epoch": 0.3821391484942887,
+      "grad_norm": 0.056484442204236984,
+      "learning_rate": 4.9954692331362294e-05,
+      "loss": 0.4438,
+      "num_input_tokens_seen": 1753776,
+      "step": 23
+    },
+    {
+      "epoch": 0.3987538940809969,
+      "grad_norm": 0.0704159140586853,
+      "learning_rate": 4.995066821070679e-05,
+      "loss": 0.4496,
+      "num_input_tokens_seen": 1809048,
+      "step": 24
+    },
+    {
+      "epoch": 0.4153686396677051,
+      "grad_norm": 0.06202029809355736,
+      "learning_rate": 4.994647308096509e-05,
+      "loss": 0.5096,
+      "num_input_tokens_seen": 1884264,
+      "step": 25
+    },
+    {
+      "epoch": 0.43198338525441327,
+      "grad_norm": 0.04237145930528641,
+      "learning_rate": 4.994210697089014e-05,
+      "loss": 0.3722,
+      "num_input_tokens_seen": 1981704,
+      "step": 26
+    },
+    {
+      "epoch": 0.4485981308411215,
+      "grad_norm": 0.06920398026704788,
+      "learning_rate": 4.9937569910406756e-05,
+      "loss": 0.4103,
+      "num_input_tokens_seen": 2044144,
+      "step": 27
+    },
+    {
+      "epoch": 0.4652128764278297,
+      "grad_norm": 0.062432270497083664,
+      "learning_rate": 4.9932861930611454e-05,
+      "loss": 0.357,
+      "num_input_tokens_seen": 2107584,
+      "step": 28
+    },
+    {
+      "epoch": 0.4818276220145379,
+      "grad_norm": 0.06791180372238159,
+      "learning_rate": 4.9927983063772196e-05,
+      "loss": 0.3889,
+      "num_input_tokens_seen": 2169248,
+      "step": 29
+    },
+    {
+      "epoch": 0.4984423676012461,
+      "grad_norm": 0.07219590991735458,
+      "learning_rate": 4.99229333433282e-05,
+      "loss": 0.3543,
+      "num_input_tokens_seen": 2230344,
+      "step": 30
+    },
+    {
+      "epoch": 0.5150571131879543,
+      "grad_norm": 0.0647474005818367,
+      "learning_rate": 4.9917712803889674e-05,
+      "loss": 0.3453,
+      "num_input_tokens_seen": 2302368,
+      "step": 31
+    },
+    {
+      "epoch": 0.5316718587746625,
+      "grad_norm": 0.07434642314910889,
+      "learning_rate": 4.991232148123761e-05,
+      "loss": 0.435,
+      "num_input_tokens_seen": 2369984,
+      "step": 32
+    },
+    {
+      "epoch": 0.5482866043613707,
+      "grad_norm": 0.05302443355321884,
+      "learning_rate": 4.990675941232353e-05,
+      "loss": 0.3981,
+      "num_input_tokens_seen": 2453032,
+      "step": 33
+    },
+    {
+      "epoch": 0.564901349948079,
+      "grad_norm": 0.053745292127132416,
+      "learning_rate": 4.990102663526924e-05,
+      "loss": 0.3755,
+      "num_input_tokens_seen": 2527464,
+      "step": 34
+    },
+    {
+      "epoch": 0.5815160955347871,
+      "grad_norm": 0.06717613339424133,
+      "learning_rate": 4.989512318936655e-05,
+      "loss": 0.3699,
+      "num_input_tokens_seen": 2597032,
+      "step": 35
+    },
+    {
+      "epoch": 0.5981308411214953,
+      "grad_norm": 0.071847103536129,
+      "learning_rate": 4.9889049115077005e-05,
+      "loss": 0.3705,
+      "num_input_tokens_seen": 2671704,
+      "step": 36
+    },
+    {
+      "epoch": 0.6147455867082036,
+      "grad_norm": 0.0460306741297245,
+      "learning_rate": 4.988280445403164e-05,
+      "loss": 0.3797,
+      "num_input_tokens_seen": 2767640,
+      "step": 37
+    },
+    {
+      "epoch": 0.6313603322949117,
+      "grad_norm": 0.053273387253284454,
+      "learning_rate": 4.987638924903067e-05,
+      "loss": 0.3799,
+      "num_input_tokens_seen": 2843720,
+      "step": 38
+    },
+    {
+      "epoch": 0.6479750778816199,
+      "grad_norm": 0.05600422993302345,
+      "learning_rate": 4.9869803544043166e-05,
+      "loss": 0.2866,
+      "num_input_tokens_seen": 2921472,
+      "step": 39
+    },
+    {
+      "epoch": 0.6645898234683282,
+      "grad_norm": 0.06414052098989487,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.4115,
+      "num_input_tokens_seen": 2998400,
+      "step": 40
+    },
+    {
+      "epoch": 0.6812045690550363,
+      "grad_norm": 0.09214208275079727,
+      "learning_rate": 4.985612081582764e-05,
+      "loss": 0.3804,
+      "num_input_tokens_seen": 3059648,
+      "step": 41
+    },
+    {
+      "epoch": 0.6978193146417445,
+      "grad_norm": 0.0555964931845665,
+      "learning_rate": 4.98490238863795e-05,
+      "loss": 0.3121,
+      "num_input_tokens_seen": 3140184,
+      "step": 42
+    },
+    {
+      "epoch": 0.7144340602284528,
+      "grad_norm": 0.06256969273090363,
+      "learning_rate": 4.984175664450397e-05,
+      "loss": 0.3271,
+      "num_input_tokens_seen": 3207184,
+      "step": 43
+    },
+    {
+      "epoch": 0.731048805815161,
+      "grad_norm": 0.0543232187628746,
+      "learning_rate": 4.983431914000991e-05,
+      "loss": 0.364,
+      "num_input_tokens_seen": 3292344,
+      "step": 44
+    },
+    {
+      "epoch": 0.7476635514018691,
+      "grad_norm": 0.06077824532985687,
+      "learning_rate": 4.982671142387316e-05,
+      "loss": 0.3894,
+      "num_input_tokens_seen": 3365384,
+      "step": 45
+    },
+    {
+      "epoch": 0.7642782969885774,
+      "grad_norm": 0.06091070920228958,
+      "learning_rate": 4.981893354823614e-05,
+      "loss": 0.3354,
+      "num_input_tokens_seen": 3440720,
+      "step": 46
+    },
+    {
+      "epoch": 0.7808930425752856,
+      "grad_norm": 0.054153311997652054,
+      "learning_rate": 4.9810985566407544e-05,
+      "loss": 0.3058,
+      "num_input_tokens_seen": 3533576,
+      "step": 47
+    },
+    {
+      "epoch": 0.7975077881619937,
+      "grad_norm": 0.06662417948246002,
+      "learning_rate": 4.980286753286195e-05,
+      "loss": 0.4658,
+      "num_input_tokens_seen": 3599744,
+      "step": 48
+    },
+    {
+      "epoch": 0.814122533748702,
+      "grad_norm": 0.05790851265192032,
+      "learning_rate": 4.979457950323945e-05,
+      "loss": 0.3647,
+      "num_input_tokens_seen": 3689520,
+      "step": 49
+    },
+    {
+      "epoch": 0.8307372793354102,
+      "grad_norm": 0.10742159187793732,
+      "learning_rate": 4.9786121534345265e-05,
+      "loss": 0.343,
+      "num_input_tokens_seen": 3751808,
+      "step": 50
+    },
+    {
+      "epoch": 0.8473520249221184,
+      "grad_norm": 0.05565556138753891,
+      "learning_rate": 4.9777493684149375e-05,
+      "loss": 0.3317,
+      "num_input_tokens_seen": 3839096,
+      "step": 51
+    },
+    {
+      "epoch": 0.8639667705088265,
+      "grad_norm": 0.05752381682395935,
+      "learning_rate": 4.976869601178609e-05,
+      "loss": 0.38,
+      "num_input_tokens_seen": 3919824,
+      "step": 52
+    },
+    {
+      "epoch": 0.8805815160955348,
+      "grad_norm": 0.06406434625387192,
+      "learning_rate": 4.975972857755369e-05,
+      "loss": 0.2676,
+      "num_input_tokens_seen": 3989312,
+      "step": 53
+    },
+    {
+      "epoch": 0.897196261682243,
+      "grad_norm": 0.0653691440820694,
+      "learning_rate": 4.975059144291394e-05,
+      "loss": 0.3516,
+      "num_input_tokens_seen": 4060528,
+      "step": 54
+    },
+    {
+      "epoch": 0.9138110072689511,
+      "grad_norm": 0.06272953748703003,
+      "learning_rate": 4.974128467049176e-05,
+      "loss": 0.3004,
+      "num_input_tokens_seen": 4129368,
+      "step": 55
+    },
+    {
+      "epoch": 0.9304257528556594,
+      "grad_norm": 0.08054930716753006,
+      "learning_rate": 4.9731808324074717e-05,
+      "loss": 0.3009,
+      "num_input_tokens_seen": 4175208,
+      "step": 56
+    },
+    {
+      "epoch": 0.9470404984423676,
+      "grad_norm": 0.07523038238286972,
+      "learning_rate": 4.972216246861262e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 4218096,
+      "step": 57
+    },
+    {
+      "epoch": 0.9636552440290758,
+      "grad_norm": 0.07347433269023895,
+      "learning_rate": 4.971234717021709e-05,
+      "loss": 0.3321,
+      "num_input_tokens_seen": 4275968,
+      "step": 58
+    },
+    {
+      "epoch": 0.980269989615784,
+      "grad_norm": 0.05830248445272446,
+      "learning_rate": 4.9702362496161085e-05,
+      "loss": 0.2881,
+      "num_input_tokens_seen": 4346616,
+      "step": 59
+    },
+    {
+      "epoch": 0.9968847352024922,
+      "grad_norm": 0.061629410833120346,
+      "learning_rate": 4.9692208514878444e-05,
+      "loss": 0.2993,
+      "num_input_tokens_seen": 4425064,
+      "step": 60
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.13380740582942963,
+      "learning_rate": 4.968188529596342e-05,
+      "loss": 0.2511,
+      "num_input_tokens_seen": 4435328,
+      "step": 61
+    },
+    {
+      "epoch": 1.0166147455867083,
+      "grad_norm": 0.0726238414645195,
+      "learning_rate": 4.9671392910170185e-05,
+      "loss": 0.3127,
+      "num_input_tokens_seen": 4500104,
+      "step": 62
+    },
+    {
+      "epoch": 1.0332294911734163,
+      "grad_norm": 0.05980083718895912,
+      "learning_rate": 4.966073142941239e-05,
+      "loss": 0.3601,
+      "num_input_tokens_seen": 4581976,
+      "step": 63
+    },
+    {
+      "epoch": 1.0498442367601246,
+      "grad_norm": 0.06445376574993134,
+      "learning_rate": 4.964990092676263e-05,
+      "loss": 0.3049,
+      "num_input_tokens_seen": 4652160,
+      "step": 64
+    },
+    {
+      "epoch": 1.066458982346833,
+      "grad_norm": 0.07824505120515823,
+      "learning_rate": 4.9638901476451946e-05,
+      "loss": 0.3099,
+      "num_input_tokens_seen": 4709368,
+      "step": 65
+    },
+    {
+      "epoch": 1.083073727933541,
+      "grad_norm": 0.058268457651138306,
+      "learning_rate": 4.962773315386935e-05,
+      "loss": 0.3273,
+      "num_input_tokens_seen": 4798256,
+      "step": 66
+    },
+    {
+      "epoch": 1.0996884735202492,
+      "grad_norm": 0.07069691270589828,
+      "learning_rate": 4.961639603556127e-05,
+      "loss": 0.282,
+      "num_input_tokens_seen": 4859200,
+      "step": 67
+    },
+    {
+      "epoch": 1.1163032191069575,
+      "grad_norm": 0.0775996670126915,
+      "learning_rate": 4.960489019923105e-05,
+      "loss": 0.3642,
+      "num_input_tokens_seen": 4925992,
+      "step": 68
+    },
+    {
+      "epoch": 1.1329179646936656,
+      "grad_norm": 0.07044171541929245,
+      "learning_rate": 4.9593215723738404e-05,
+      "loss": 0.2896,
+      "num_input_tokens_seen": 4998808,
+      "step": 69
+    },
+    {
+      "epoch": 1.1495327102803738,
+      "grad_norm": 0.05971802771091461,
+      "learning_rate": 4.958137268909887e-05,
+      "loss": 0.2578,
+      "num_input_tokens_seen": 5089672,
+      "step": 70
+    },
+    {
+      "epoch": 1.1661474558670821,
+      "grad_norm": 0.07145556062459946,
+      "learning_rate": 4.9569361176483286e-05,
+      "loss": 0.3243,
+      "num_input_tokens_seen": 5166744,
+      "step": 71
+    },
+    {
+      "epoch": 1.1827622014537902,
+      "grad_norm": 0.07455787807703018,
+      "learning_rate": 4.9557181268217227e-05,
+      "loss": 0.3949,
+      "num_input_tokens_seen": 5228264,
+      "step": 72
+    },
+    {
+      "epoch": 1.1993769470404985,
+      "grad_norm": 0.055582575500011444,
+      "learning_rate": 4.9544833047780394e-05,
+      "loss": 0.2877,
+      "num_input_tokens_seen": 5338224,
+      "step": 73
+    },
+    {
+      "epoch": 1.2159916926272065,
+      "grad_norm": 0.07675391435623169,
+      "learning_rate": 4.9532316599806124e-05,
+      "loss": 0.3152,
+      "num_input_tokens_seen": 5399848,
+      "step": 74
+    },
+    {
+      "epoch": 1.2326064382139148,
+      "grad_norm": 0.08048644661903381,
+      "learning_rate": 4.951963201008076e-05,
+      "loss": 0.2976,
+      "num_input_tokens_seen": 5468624,
+      "step": 75
+    },
+    {
+      "epoch": 1.249221183800623,
+      "grad_norm": 0.07579060643911362,
+      "learning_rate": 4.9506779365543046e-05,
+      "loss": 0.2982,
+      "num_input_tokens_seen": 5536776,
+      "step": 76
+    },
+    {
+      "epoch": 1.2658359293873311,
+      "grad_norm": 0.07828006893396378,
+      "learning_rate": 4.949375875428357e-05,
+      "loss": 0.3272,
+      "num_input_tokens_seen": 5609296,
+      "step": 77
+    },
+    {
+      "epoch": 1.2824506749740394,
+      "grad_norm": 0.08079098165035248,
+      "learning_rate": 4.9480570265544144e-05,
+      "loss": 0.2768,
+      "num_input_tokens_seen": 5663824,
+      "step": 78
+    },
+    {
+      "epoch": 1.2990654205607477,
+      "grad_norm": 0.07579358667135239,
+      "learning_rate": 4.94672139897172e-05,
+      "loss": 0.318,
+      "num_input_tokens_seen": 5742032,
+      "step": 79
+    },
+    {
+      "epoch": 1.3156801661474558,
+      "grad_norm": 0.07588379085063934,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.3007,
+      "num_input_tokens_seen": 5816864,
+      "step": 80
+    },
+    {
+      "epoch": 1.332294911734164,
+      "grad_norm": 0.08709035068750381,
+      "learning_rate": 4.943999844411977e-05,
+      "loss": 0.2797,
+      "num_input_tokens_seen": 5881624,
+      "step": 81
+    },
+    {
+      "epoch": 1.3489096573208723,
+      "grad_norm": 0.05975884944200516,
+      "learning_rate": 4.94261393608816e-05,
+      "loss": 0.2591,
+      "num_input_tokens_seen": 5970272,
+      "step": 82
+    },
+    {
+      "epoch": 1.3655244029075804,
+      "grad_norm": 0.07372818142175674,
+      "learning_rate": 4.941211286361922e-05,
+      "loss": 0.2687,
+      "num_input_tokens_seen": 6058752,
+      "step": 83
+    },
+    {
+      "epoch": 1.3821391484942886,
+      "grad_norm": 0.09071576595306396,
+      "learning_rate": 4.939791904846869e-05,
+      "loss": 0.2979,
+      "num_input_tokens_seen": 6120064,
+      "step": 84
+    },
+    {
+      "epoch": 1.398753894080997,
+      "grad_norm": 0.0849960595369339,
+      "learning_rate": 4.938355801271282e-05,
+      "loss": 0.2927,
+      "num_input_tokens_seen": 6182072,
+      "step": 85
+    },
+    {
+      "epoch": 1.415368639667705,
+      "grad_norm": 0.08258760720491409,
+      "learning_rate": 4.936902985478055e-05,
+      "loss": 0.295,
+      "num_input_tokens_seen": 6269680,
+      "step": 86
+    },
+    {
+      "epoch": 1.4319833852544133,
+      "grad_norm": 0.0851503536105156,
+      "learning_rate": 4.935433467424624e-05,
+      "loss": 0.2925,
+      "num_input_tokens_seen": 6347424,
+      "step": 87
+    },
+    {
+      "epoch": 1.4485981308411215,
+      "grad_norm": 0.08852345496416092,
+      "learning_rate": 4.933947257182901e-05,
+      "loss": 0.3153,
+      "num_input_tokens_seen": 6412584,
+      "step": 88
+    },
+    {
+      "epoch": 1.4652128764278296,
+      "grad_norm": 0.08184897154569626,
+      "learning_rate": 4.932444364939205e-05,
+      "loss": 0.292,
+      "num_input_tokens_seen": 6482728,
+      "step": 89
+    },
+    {
+      "epoch": 1.4818276220145379,
+      "grad_norm": 0.08270515501499176,
+      "learning_rate": 4.9309248009941914e-05,
+      "loss": 0.3472,
+      "num_input_tokens_seen": 6562104,
+      "step": 90
+    },
+    {
+      "epoch": 1.4984423676012462,
+      "grad_norm": 0.07407747954130173,
+      "learning_rate": 4.929388575762782e-05,
+      "loss": 0.2995,
+      "num_input_tokens_seen": 6656552,
+      "step": 91
+    },
+    {
+      "epoch": 1.5150571131879542,
+      "grad_norm": 0.08710360527038574,
+      "learning_rate": 4.9278356997740904e-05,
+      "loss": 0.2549,
+      "num_input_tokens_seen": 6714184,
+      "step": 92
+    },
+    {
+      "epoch": 1.5316718587746625,
+      "grad_norm": 0.0773790255188942,
+      "learning_rate": 4.9262661836713564e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 6793552,
+      "step": 93
+    },
+    {
+      "epoch": 1.5482866043613708,
+      "grad_norm": 0.1002134457230568,
+      "learning_rate": 4.924680038211867e-05,
+      "loss": 0.2876,
+      "num_input_tokens_seen": 6865256,
+      "step": 94
+    },
+    {
+      "epoch": 1.5649013499480788,
+      "grad_norm": 0.09670394659042358,
+      "learning_rate": 4.9230772742668866e-05,
+      "loss": 0.2846,
+      "num_input_tokens_seen": 6931152,
+      "step": 95
+    },
+    {
+      "epoch": 1.5815160955347871,
+      "grad_norm": 0.08910100907087326,
+      "learning_rate": 4.9214579028215776e-05,
+      "loss": 0.2944,
+      "num_input_tokens_seen": 6998408,
+      "step": 96
+    },
+    {
+      "epoch": 1.5981308411214954,
+      "grad_norm": 0.09202459454536438,
+      "learning_rate": 4.919821934974933e-05,
+      "loss": 0.251,
+      "num_input_tokens_seen": 7053008,
+      "step": 97
+    },
+    {
+      "epoch": 1.6147455867082035,
+      "grad_norm": 0.10218881815671921,
+      "learning_rate": 4.918169381939692e-05,
+      "loss": 0.2851,
+      "num_input_tokens_seen": 7106440,
+      "step": 98
+    },
+    {
+      "epoch": 1.6313603322949117,
+      "grad_norm": 0.09290914982557297,
+      "learning_rate": 4.916500255042268e-05,
+      "loss": 0.2959,
+      "num_input_tokens_seen": 7167032,
+      "step": 99
+    },
+    {
+      "epoch": 1.64797507788162,
+      "grad_norm": 0.07791033387184143,
+      "learning_rate": 4.914814565722671e-05,
+      "loss": 0.2481,
+      "num_input_tokens_seen": 7245720,
+      "step": 100
+    },
+    {
+      "epoch": 1.664589823468328,
+      "grad_norm": 0.08885534107685089,
+      "learning_rate": 4.913112325534426e-05,
+      "loss": 0.3168,
+      "num_input_tokens_seen": 7326320,
+      "step": 101
+    },
+    {
+      "epoch": 1.6812045690550363,
+      "grad_norm": 0.08569750934839249,
+      "learning_rate": 4.9113935461444955e-05,
+      "loss": 0.2805,
+      "num_input_tokens_seen": 7442232,
+      "step": 102
+    },
+    {
+      "epoch": 1.6978193146417446,
+      "grad_norm": 0.1112508773803711,
+      "learning_rate": 4.9096582393332025e-05,
+      "loss": 0.2675,
+      "num_input_tokens_seen": 7502496,
+      "step": 103
+    },
+    {
+      "epoch": 1.7144340602284527,
+      "grad_norm": 0.09654372185468674,
+      "learning_rate": 4.907906416994146e-05,
+      "loss": 0.3038,
+      "num_input_tokens_seen": 7566496,
+      "step": 104
+    },
+    {
+      "epoch": 1.731048805815161,
+      "grad_norm": 0.10022995620965958,
+      "learning_rate": 4.906138091134118e-05,
+      "loss": 0.3639,
+      "num_input_tokens_seen": 7629056,
+      "step": 105
+    },
+    {
+      "epoch": 1.7476635514018692,
+      "grad_norm": 0.08336564153432846,
+      "learning_rate": 4.9043532738730284e-05,
+      "loss": 0.2944,
+      "num_input_tokens_seen": 7706096,
+      "step": 106
+    },
+    {
+      "epoch": 1.7642782969885773,
+      "grad_norm": 0.08539658784866333,
+      "learning_rate": 4.9025519774438136e-05,
+      "loss": 0.2392,
+      "num_input_tokens_seen": 7780072,
+      "step": 107
+    },
+    {
+      "epoch": 1.7808930425752856,
+      "grad_norm": 0.09139693528413773,
+      "learning_rate": 4.900734214192358e-05,
+      "loss": 0.2685,
+      "num_input_tokens_seen": 7857712,
+      "step": 108
+    },
+    {
+      "epoch": 1.7975077881619939,
+      "grad_norm": 0.1043916717171669,
+      "learning_rate": 4.898899996577407e-05,
+      "loss": 0.2513,
+      "num_input_tokens_seen": 7916832,
+      "step": 109
+    },
+    {
+      "epoch": 1.814122533748702,
+      "grad_norm": 0.09203662723302841,
+      "learning_rate": 4.8970493371704826e-05,
+      "loss": 0.2974,
+      "num_input_tokens_seen": 7993056,
+      "step": 110
+    },
+    {
+      "epoch": 1.8307372793354102,
+      "grad_norm": 0.09319474548101425,
+      "learning_rate": 4.8951822486557986e-05,
+      "loss": 0.3096,
+      "num_input_tokens_seen": 8090056,
+      "step": 111
+    },
+    {
+      "epoch": 1.8473520249221185,
+      "grad_norm": 0.10193445533514023,
+      "learning_rate": 4.893298743830168e-05,
+      "loss": 0.2633,
+      "num_input_tokens_seen": 8164808,
+      "step": 112
+    },
+    {
+      "epoch": 1.8639667705088265,
+      "grad_norm": 0.11407948285341263,
+      "learning_rate": 4.891398835602925e-05,
+      "loss": 0.2584,
+      "num_input_tokens_seen": 8223568,
+      "step": 113
+    },
+    {
+      "epoch": 1.8805815160955348,
+      "grad_norm": 0.11977085471153259,
+      "learning_rate": 4.8894825369958255e-05,
+      "loss": 0.2619,
+      "num_input_tokens_seen": 8276160,
+      "step": 114
+    },
+    {
+      "epoch": 1.897196261682243,
+      "grad_norm": 0.10925433784723282,
+      "learning_rate": 4.8875498611429674e-05,
+      "loss": 0.2762,
+      "num_input_tokens_seen": 8354904,
+      "step": 115
+    },
+    {
+      "epoch": 1.9138110072689511,
+      "grad_norm": 0.09673939645290375,
+      "learning_rate": 4.8856008212906925e-05,
+      "loss": 0.3152,
+      "num_input_tokens_seen": 8442584,
+      "step": 116
+    },
+    {
+      "epoch": 1.9304257528556594,
+      "grad_norm": 0.10827789455652237,
+      "learning_rate": 4.8836354307975026e-05,
+      "loss": 0.2759,
+      "num_input_tokens_seen": 8506688,
+      "step": 117
+    },
+    {
+      "epoch": 1.9470404984423677,
+      "grad_norm": 0.08390220254659653,
+      "learning_rate": 4.881653703133966e-05,
+      "loss": 0.2192,
+      "num_input_tokens_seen": 8610712,
+      "step": 118
+    },
+    {
+      "epoch": 1.9636552440290758,
+      "grad_norm": 0.09252211451530457,
+      "learning_rate": 4.87965565188262e-05,
+      "loss": 0.2618,
+      "num_input_tokens_seen": 8692624,
+      "step": 119
+    },
+    {
+      "epoch": 1.980269989615784,
+      "grad_norm": 0.1107102632522583,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.2666,
+      "num_input_tokens_seen": 8772208,
+      "step": 120
+    },
+    {
+      "epoch": 1.9968847352024923,
+      "grad_norm": 0.0917077362537384,
+      "learning_rate": 4.8756106335059646e-05,
+      "loss": 0.253,
+      "num_input_tokens_seen": 8854904,
+      "step": 121
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.2606711685657501,
+      "learning_rate": 4.87356369410476e-05,
+      "loss": 0.235,
+      "num_input_tokens_seen": 8872656,
+      "step": 122
+    },
+    {
+      "epoch": 2.016614745586708,
+      "grad_norm": 0.10363993793725967,
+      "learning_rate": 4.8715004865637614e-05,
+      "loss": 0.266,
+      "num_input_tokens_seen": 8946480,
+      "step": 123
+    },
+    {
+      "epoch": 2.0332294911734166,
+      "grad_norm": 0.09997844696044922,
+      "learning_rate": 4.869421025023965e-05,
+      "loss": 0.2696,
+      "num_input_tokens_seen": 9023328,
+      "step": 124
+    },
+    {
+      "epoch": 2.0498442367601246,
+      "grad_norm": 0.13349319994449615,
+      "learning_rate": 4.867325323737765e-05,
+      "loss": 0.2552,
+      "num_input_tokens_seen": 9074320,
+      "step": 125
+    },
+    {
+      "epoch": 2.0664589823468327,
+      "grad_norm": 0.11201464384794235,
+      "learning_rate": 4.8652133970688636e-05,
+      "loss": 0.2486,
+      "num_input_tokens_seen": 9148784,
+      "step": 126
+    },
+    {
+      "epoch": 2.083073727933541,
+      "grad_norm": 0.10193142294883728,
+      "learning_rate": 4.8630852594921706e-05,
+      "loss": 0.2814,
+      "num_input_tokens_seen": 9246624,
+      "step": 127
+    },
+    {
+      "epoch": 2.0996884735202492,
+      "grad_norm": 0.1305130422115326,
+      "learning_rate": 4.860940925593703e-05,
+      "loss": 0.304,
+      "num_input_tokens_seen": 9328176,
+      "step": 128
+    },
+    {
+      "epoch": 2.1163032191069573,
+      "grad_norm": 0.1137692779302597,
+      "learning_rate": 4.8587804100704845e-05,
+      "loss": 0.2427,
+      "num_input_tokens_seen": 9388936,
+      "step": 129
+    },
+    {
+      "epoch": 2.132917964693666,
+      "grad_norm": 0.12126237154006958,
+      "learning_rate": 4.856603727730447e-05,
+      "loss": 0.2485,
+      "num_input_tokens_seen": 9461664,
+      "step": 130
+    },
+    {
+      "epoch": 2.149532710280374,
+      "grad_norm": 0.11567176878452301,
+      "learning_rate": 4.854410893492326e-05,
+      "loss": 0.2628,
+      "num_input_tokens_seen": 9535000,
+      "step": 131
+    },
+    {
+      "epoch": 2.166147455867082,
+      "grad_norm": 0.1399552971124649,
+      "learning_rate": 4.852201922385564e-05,
+      "loss": 0.2518,
+      "num_input_tokens_seen": 9600296,
+      "step": 132
+    },
+    {
+      "epoch": 2.1827622014537904,
+      "grad_norm": 0.13912151753902435,
+      "learning_rate": 4.8499768295502004e-05,
+      "loss": 0.2429,
+      "num_input_tokens_seen": 9686784,
+      "step": 133
+    },
+    {
+      "epoch": 2.1993769470404985,
+      "grad_norm": 0.11130474507808685,
+      "learning_rate": 4.847735630236773e-05,
+      "loss": 0.2775,
+      "num_input_tokens_seen": 9781112,
+      "step": 134
+    },
+    {
+      "epoch": 2.2159916926272065,
+      "grad_norm": 0.12169156968593597,
+      "learning_rate": 4.8454783398062106e-05,
+      "loss": 0.2439,
+      "num_input_tokens_seen": 9849528,
+      "step": 135
+    },
+    {
+      "epoch": 2.232606438213915,
+      "grad_norm": 0.11766713112592697,
+      "learning_rate": 4.843204973729729e-05,
+      "loss": 0.2538,
+      "num_input_tokens_seen": 9931080,
+      "step": 136
+    },
+    {
+      "epoch": 2.249221183800623,
+      "grad_norm": 0.11854218691587448,
+      "learning_rate": 4.840915547588725e-05,
+      "loss": 0.2782,
+      "num_input_tokens_seen": 10011176,
+      "step": 137
+    },
+    {
+      "epoch": 2.265835929387331,
+      "grad_norm": 0.1340581178665161,
+      "learning_rate": 4.838610077074669e-05,
+      "loss": 0.248,
+      "num_input_tokens_seen": 10084128,
+      "step": 138
+    },
+    {
+      "epoch": 2.2824506749740396,
+      "grad_norm": 0.12075436115264893,
+      "learning_rate": 4.836288577988996e-05,
+      "loss": 0.2582,
+      "num_input_tokens_seen": 10155536,
+      "step": 139
+    },
+    {
+      "epoch": 2.2990654205607477,
+      "grad_norm": 0.10599923878908157,
+      "learning_rate": 4.8339510662430046e-05,
+      "loss": 0.2199,
+      "num_input_tokens_seen": 10251160,
+      "step": 140
+    },
+    {
+      "epoch": 2.3156801661474558,
+      "grad_norm": 0.1117846742272377,
+      "learning_rate": 4.8315975578577355e-05,
+      "loss": 0.2324,
+      "num_input_tokens_seen": 10345864,
+      "step": 141
+    },
+    {
+      "epoch": 2.3322949117341643,
+      "grad_norm": 0.13972057402133942,
+      "learning_rate": 4.8292280689638725e-05,
+      "loss": 0.4072,
+      "num_input_tokens_seen": 10417616,
+      "step": 142
+    },
+    {
+      "epoch": 2.3489096573208723,
+      "grad_norm": 0.13837860524654388,
+      "learning_rate": 4.826842615801628e-05,
+      "loss": 0.2607,
+      "num_input_tokens_seen": 10481816,
+      "step": 143
+    },
+    {
+      "epoch": 2.3655244029075804,
+      "grad_norm": 0.14040137827396393,
+      "learning_rate": 4.8244412147206284e-05,
+      "loss": 0.3094,
+      "num_input_tokens_seen": 10562056,
+      "step": 144
+    },
+    {
+      "epoch": 2.382139148494289,
+      "grad_norm": 0.1393299251794815,
+      "learning_rate": 4.822023882179811e-05,
+      "loss": 0.2407,
+      "num_input_tokens_seen": 10612808,
+      "step": 145
+    },
+    {
+      "epoch": 2.398753894080997,
+      "grad_norm": 0.13878698647022247,
+      "learning_rate": 4.8195906347473e-05,
+      "loss": 0.2481,
+      "num_input_tokens_seen": 10682328,
+      "step": 146
+    },
+    {
+      "epoch": 2.415368639667705,
+      "grad_norm": 0.10430227965116501,
+      "learning_rate": 4.817141489100302e-05,
+      "loss": 0.2528,
+      "num_input_tokens_seen": 10771912,
+      "step": 147
+    },
+    {
+      "epoch": 2.431983385254413,
+      "grad_norm": 0.12963703274726868,
+      "learning_rate": 4.814676462024988e-05,
+      "loss": 0.2739,
+      "num_input_tokens_seen": 10842232,
+      "step": 148
+    },
+    {
+      "epoch": 2.4485981308411215,
+      "grad_norm": 0.13274963200092316,
+      "learning_rate": 4.8121955704163745e-05,
+      "loss": 0.2407,
+      "num_input_tokens_seen": 10902264,
+      "step": 149
+    },
+    {
+      "epoch": 2.4652128764278296,
+      "grad_norm": 0.11079717427492142,
+      "learning_rate": 4.8096988312782174e-05,
+      "loss": 0.2142,
+      "num_input_tokens_seen": 10992744,
+      "step": 150
+    },
+    {
+      "epoch": 2.4818276220145377,
+      "grad_norm": 0.08429212868213654,
+      "learning_rate": 4.8071862617228855e-05,
+      "loss": 0.1428,
+      "num_input_tokens_seen": 11090064,
+      "step": 151
+    },
+    {
+      "epoch": 2.498442367601246,
+      "grad_norm": 0.12903761863708496,
+      "learning_rate": 4.8046578789712515e-05,
+      "loss": 0.2268,
+      "num_input_tokens_seen": 11162864,
+      "step": 152
+    },
+    {
+      "epoch": 2.515057113187954,
+      "grad_norm": 0.14638672769069672,
+      "learning_rate": 4.8021137003525664e-05,
+      "loss": 0.2388,
+      "num_input_tokens_seen": 11224368,
+      "step": 153
+    },
+    {
+      "epoch": 2.5316718587746623,
+      "grad_norm": 0.1372838169336319,
+      "learning_rate": 4.7995537433043446e-05,
+      "loss": 0.2588,
+      "num_input_tokens_seen": 11291056,
+      "step": 154
+    },
+    {
+      "epoch": 2.5482866043613708,
+      "grad_norm": 0.15665481984615326,
+      "learning_rate": 4.796978025372246e-05,
+      "loss": 0.2225,
+      "num_input_tokens_seen": 11345464,
+      "step": 155
+    },
+    {
+      "epoch": 2.564901349948079,
+      "grad_norm": 0.13234855234622955,
+      "learning_rate": 4.794386564209953e-05,
+      "loss": 0.275,
+      "num_input_tokens_seen": 11418912,
+      "step": 156
+    },
+    {
+      "epoch": 2.581516095534787,
+      "grad_norm": 0.13585953414440155,
+      "learning_rate": 4.79177937757905e-05,
+      "loss": 0.2407,
+      "num_input_tokens_seen": 11491216,
+      "step": 157
+    },
+    {
+      "epoch": 2.5981308411214954,
+      "grad_norm": 0.1423913538455963,
+      "learning_rate": 4.7891564833489035e-05,
+      "loss": 0.1971,
+      "num_input_tokens_seen": 11558016,
+      "step": 158
+    },
+    {
+      "epoch": 2.6147455867082035,
+      "grad_norm": 0.13013511896133423,
+      "learning_rate": 4.7865178994965344e-05,
+      "loss": 0.2362,
+      "num_input_tokens_seen": 11630432,
+      "step": 159
+    },
+    {
+      "epoch": 2.6313603322949115,
+      "grad_norm": 0.1587141752243042,
+      "learning_rate": 4.783863644106502e-05,
+      "loss": 0.2252,
+      "num_input_tokens_seen": 11684624,
+      "step": 160
+    },
+    {
+      "epoch": 2.64797507788162,
+      "grad_norm": 0.12592960894107819,
+      "learning_rate": 4.781193735370777e-05,
+      "loss": 0.2506,
+      "num_input_tokens_seen": 11770232,
+      "step": 161
+    },
+    {
+      "epoch": 2.664589823468328,
+      "grad_norm": 0.1583249419927597,
+      "learning_rate": 4.7785081915886134e-05,
+      "loss": 0.2352,
+      "num_input_tokens_seen": 11828360,
+      "step": 162
+    },
+    {
+      "epoch": 2.681204569055036,
+      "grad_norm": 0.14881783723831177,
+      "learning_rate": 4.775807031166428e-05,
+      "loss": 0.2308,
+      "num_input_tokens_seen": 11915944,
+      "step": 163
+    },
+    {
+      "epoch": 2.6978193146417446,
+      "grad_norm": 0.1607823222875595,
+      "learning_rate": 4.773090272617672e-05,
+      "loss": 0.2238,
+      "num_input_tokens_seen": 11981792,
+      "step": 164
+    },
+    {
+      "epoch": 2.7144340602284527,
+      "grad_norm": 0.13583113253116608,
+      "learning_rate": 4.7703579345627035e-05,
+      "loss": 0.3196,
+      "num_input_tokens_seen": 12044024,
+      "step": 165
+    },
+    {
+      "epoch": 2.7310488058151607,
+      "grad_norm": 0.19167298078536987,
+      "learning_rate": 4.7676100357286624e-05,
+      "loss": 0.2745,
+      "num_input_tokens_seen": 12093424,
+      "step": 166
+    },
+    {
+      "epoch": 2.7476635514018692,
+      "grad_norm": 0.130703404545784,
+      "learning_rate": 4.76484659494934e-05,
+      "loss": 0.2285,
+      "num_input_tokens_seen": 12167792,
+      "step": 167
+    },
+    {
+      "epoch": 2.7642782969885773,
+      "grad_norm": 0.14331185817718506,
+      "learning_rate": 4.762067631165049e-05,
+      "loss": 0.2506,
+      "num_input_tokens_seen": 12233712,
+      "step": 168
+    },
+    {
+      "epoch": 2.7808930425752854,
+      "grad_norm": 0.12700341641902924,
+      "learning_rate": 4.7592731634224966e-05,
+      "loss": 0.2052,
+      "num_input_tokens_seen": 12310544,
+      "step": 169
+    },
+    {
+      "epoch": 2.797507788161994,
+      "grad_norm": 0.15118420124053955,
+      "learning_rate": 4.756463210874652e-05,
+      "loss": 0.2309,
+      "num_input_tokens_seen": 12400160,
+      "step": 170
+    },
+    {
+      "epoch": 2.814122533748702,
+      "grad_norm": 0.14001020789146423,
+      "learning_rate": 4.753637792780614e-05,
+      "loss": 0.2544,
+      "num_input_tokens_seen": 12480432,
+      "step": 171
+    },
+    {
+      "epoch": 2.83073727933541,
+      "grad_norm": 0.12076311558485031,
+      "learning_rate": 4.7507969285054845e-05,
+      "loss": 0.2434,
+      "num_input_tokens_seen": 12568064,
+      "step": 172
+    },
+    {
+      "epoch": 2.8473520249221185,
+      "grad_norm": 0.16462342441082,
+      "learning_rate": 4.7479406375202264e-05,
+      "loss": 0.2417,
+      "num_input_tokens_seen": 12647400,
+      "step": 173
+    },
+    {
+      "epoch": 2.8639667705088265,
+      "grad_norm": 0.17294971644878387,
+      "learning_rate": 4.745068939401539e-05,
+      "loss": 0.2121,
+      "num_input_tokens_seen": 12698208,
+      "step": 174
+    },
+    {
+      "epoch": 2.8805815160955346,
+      "grad_norm": 0.16743803024291992,
+      "learning_rate": 4.742181853831721e-05,
+      "loss": 0.2238,
+      "num_input_tokens_seen": 12758528,
+      "step": 175
+    },
+    {
+      "epoch": 2.897196261682243,
+      "grad_norm": 0.14583320915699005,
+      "learning_rate": 4.7392794005985326e-05,
+      "loss": 0.2333,
+      "num_input_tokens_seen": 12837264,
+      "step": 176
+    },
+    {
+      "epoch": 2.913811007268951,
+      "grad_norm": 0.1509270817041397,
+      "learning_rate": 4.7363615995950626e-05,
+      "loss": 0.2179,
+      "num_input_tokens_seen": 12902368,
+      "step": 177
+    },
+    {
+      "epoch": 2.930425752855659,
+      "grad_norm": 0.12910738587379456,
+      "learning_rate": 4.733428470819594e-05,
+      "loss": 0.2144,
+      "num_input_tokens_seen": 12974296,
+      "step": 178
+    },
+    {
+      "epoch": 2.9470404984423677,
+      "grad_norm": 0.142000213265419,
+      "learning_rate": 4.730480034375462e-05,
+      "loss": 0.2413,
+      "num_input_tokens_seen": 13057280,
+      "step": 179
+    },
+    {
+      "epoch": 2.9636552440290758,
+      "grad_norm": 0.131468266248703,
+      "learning_rate": 4.72751631047092e-05,
+      "loss": 0.294,
+      "num_input_tokens_seen": 13158232,
+      "step": 180
+    },
+    {
+      "epoch": 2.980269989615784,
+      "grad_norm": 0.1529342085123062,
+      "learning_rate": 4.7245373194189994e-05,
+      "loss": 0.216,
+      "num_input_tokens_seen": 13229840,
+      "step": 181
+    },
+    {
+      "epoch": 2.9968847352024923,
+      "grad_norm": 0.1573815941810608,
+      "learning_rate": 4.7215430816373726e-05,
+      "loss": 0.2384,
+      "num_input_tokens_seen": 13296520,
+      "step": 182
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.2532118558883667,
+      "learning_rate": 4.718533617648209e-05,
+      "loss": 0.1459,
+      "num_input_tokens_seen": 13309672,
+      "step": 183
+    },
+    {
+      "epoch": 3.016614745586708,
+      "grad_norm": 0.16963432729244232,
+      "learning_rate": 4.715508948078037e-05,
+      "loss": 0.1985,
+      "num_input_tokens_seen": 13371544,
+      "step": 184
+    },
+    {
+      "epoch": 3.0332294911734166,
+      "grad_norm": 0.18877384066581726,
+      "learning_rate": 4.712469093657605e-05,
+      "loss": 0.1856,
+      "num_input_tokens_seen": 13432984,
+      "step": 185
+    },
+    {
+      "epoch": 3.0498442367601246,
+      "grad_norm": 0.14922884106636047,
+      "learning_rate": 4.709414075221734e-05,
+      "loss": 0.2385,
+      "num_input_tokens_seen": 13500016,
+      "step": 186
+    },
+    {
+      "epoch": 3.0664589823468327,
+      "grad_norm": 0.2028326541185379,
+      "learning_rate": 4.706343913709178e-05,
+      "loss": 0.2227,
+      "num_input_tokens_seen": 13579672,
+      "step": 187
+    },
+    {
+      "epoch": 3.083073727933541,
+      "grad_norm": 0.19964616000652313,
+      "learning_rate": 4.70325863016248e-05,
+      "loss": 0.2045,
+      "num_input_tokens_seen": 13630704,
+      "step": 188
+    },
+    {
+      "epoch": 3.0996884735202492,
+      "grad_norm": 0.1594657599925995,
+      "learning_rate": 4.7001582457278304e-05,
+      "loss": 0.2648,
+      "num_input_tokens_seen": 13695472,
+      "step": 189
+    },
+    {
+      "epoch": 3.1163032191069573,
+      "grad_norm": 0.16952532529830933,
+      "learning_rate": 4.697042781654913e-05,
+      "loss": 0.22,
+      "num_input_tokens_seen": 13767792,
+      "step": 190
+    },
+    {
+      "epoch": 3.132917964693666,
+      "grad_norm": 0.16775831580162048,
+      "learning_rate": 4.693912259296773e-05,
+      "loss": 0.2667,
+      "num_input_tokens_seen": 13857352,
+      "step": 191
+    },
+    {
+      "epoch": 3.149532710280374,
+      "grad_norm": 0.15529580414295197,
+      "learning_rate": 4.690766700109659e-05,
+      "loss": 0.2154,
+      "num_input_tokens_seen": 13939928,
+      "step": 192
+    },
+    {
+      "epoch": 3.166147455867082,
+      "grad_norm": 0.1619848757982254,
+      "learning_rate": 4.687606125652882e-05,
+      "loss": 0.1963,
+      "num_input_tokens_seen": 14017936,
+      "step": 193
+    },
+    {
+      "epoch": 3.1827622014537904,
+      "grad_norm": 0.18066684901714325,
+      "learning_rate": 4.684430557588664e-05,
+      "loss": 0.1862,
+      "num_input_tokens_seen": 14074176,
+      "step": 194
+    },
+    {
+      "epoch": 3.1993769470404985,
+      "grad_norm": 0.16520777344703674,
+      "learning_rate": 4.681240017681993e-05,
+      "loss": 0.2576,
+      "num_input_tokens_seen": 14167656,
+      "step": 195
+    },
+    {
+      "epoch": 3.2159916926272065,
+      "grad_norm": 0.15385325253009796,
+      "learning_rate": 4.678034527800474e-05,
+      "loss": 0.1813,
+      "num_input_tokens_seen": 14235800,
+      "step": 196
+    },
+    {
+      "epoch": 3.232606438213915,
+      "grad_norm": 0.16897696256637573,
+      "learning_rate": 4.674814109914174e-05,
+      "loss": 0.1741,
+      "num_input_tokens_seen": 14301272,
+      "step": 197
+    },
+    {
+      "epoch": 3.249221183800623,
+      "grad_norm": 0.19556447863578796,
+      "learning_rate": 4.671578786095478e-05,
+      "loss": 0.2186,
+      "num_input_tokens_seen": 14347352,
+      "step": 198
+    },
+    {
+      "epoch": 3.265835929387331,
+      "grad_norm": 0.17333142459392548,
+      "learning_rate": 4.668328578518933e-05,
+      "loss": 0.2892,
+      "num_input_tokens_seen": 14434600,
+      "step": 199
+    },
+    {
+      "epoch": 3.2824506749740396,
+      "grad_norm": 0.20295488834381104,
+      "learning_rate": 4.665063509461097e-05,
+      "loss": 0.2014,
+      "num_input_tokens_seen": 14484104,
+      "step": 200
+    },
+    {
+      "epoch": 3.2990654205607477,
+      "grad_norm": 0.1597638726234436,
+      "learning_rate": 4.661783601300388e-05,
+      "loss": 0.2158,
+      "num_input_tokens_seen": 14567152,
+      "step": 201
+    },
+    {
+      "epoch": 3.3156801661474558,
+      "grad_norm": 0.19849488139152527,
+      "learning_rate": 4.6584888765169296e-05,
+      "loss": 0.2578,
+      "num_input_tokens_seen": 14647040,
+      "step": 202
+    },
+    {
+      "epoch": 3.3322949117341643,
+      "grad_norm": 0.1508200466632843,
+      "learning_rate": 4.6551793576923964e-05,
+      "loss": 0.2213,
+      "num_input_tokens_seen": 14738216,
+      "step": 203
+    },
+    {
+      "epoch": 3.3489096573208723,
+      "grad_norm": 0.1687687337398529,
+      "learning_rate": 4.65185506750986e-05,
+      "loss": 0.1828,
+      "num_input_tokens_seen": 14811216,
+      "step": 204
+    },
+    {
+      "epoch": 3.3655244029075804,
+      "grad_norm": 0.16587376594543457,
+      "learning_rate": 4.648516028753632e-05,
+      "loss": 0.1619,
+      "num_input_tokens_seen": 14885992,
+      "step": 205
+    },
+    {
+      "epoch": 3.382139148494289,
+      "grad_norm": 0.16600169241428375,
+      "learning_rate": 4.645162264309112e-05,
+      "loss": 0.2438,
+      "num_input_tokens_seen": 14961984,
+      "step": 206
+    },
+    {
+      "epoch": 3.398753894080997,
+      "grad_norm": 0.1877703070640564,
+      "learning_rate": 4.6417937971626245e-05,
+      "loss": 0.1771,
+      "num_input_tokens_seen": 15021240,
+      "step": 207
+    },
+    {
+      "epoch": 3.415368639667705,
+      "grad_norm": 0.20105206966400146,
+      "learning_rate": 4.638410650401267e-05,
+      "loss": 0.1742,
+      "num_input_tokens_seen": 15092016,
+      "step": 208
+    },
+    {
+      "epoch": 3.431983385254413,
+      "grad_norm": 0.12934140861034393,
+      "learning_rate": 4.635012847212748e-05,
+      "loss": 0.1725,
+      "num_input_tokens_seen": 15198192,
+      "step": 209
+    },
+    {
+      "epoch": 3.4485981308411215,
+      "grad_norm": 0.18388882279396057,
+      "learning_rate": 4.6316004108852305e-05,
+      "loss": 0.186,
+      "num_input_tokens_seen": 15258432,
+      "step": 210
+    },
+    {
+      "epoch": 3.4652128764278296,
+      "grad_norm": 0.1766858547925949,
+      "learning_rate": 4.628173364807171e-05,
+      "loss": 0.2166,
+      "num_input_tokens_seen": 15329600,
+      "step": 211
+    },
+    {
+      "epoch": 3.4818276220145377,
+      "grad_norm": 0.16214998066425323,
+      "learning_rate": 4.6247317324671605e-05,
+      "loss": 0.2038,
+      "num_input_tokens_seen": 15407920,
+      "step": 212
+    },
+    {
+      "epoch": 3.498442367601246,
+      "grad_norm": 0.16933797299861908,
+      "learning_rate": 4.6212755374537596e-05,
+      "loss": 0.2017,
+      "num_input_tokens_seen": 15479640,
+      "step": 213
+    },
+    {
+      "epoch": 3.515057113187954,
+      "grad_norm": 0.19472749531269073,
+      "learning_rate": 4.617804803455344e-05,
+      "loss": 0.2048,
+      "num_input_tokens_seen": 15561960,
+      "step": 214
+    },
+    {
+      "epoch": 3.5316718587746623,
+      "grad_norm": 0.33335182070732117,
+      "learning_rate": 4.614319554259934e-05,
+      "loss": 0.2358,
+      "num_input_tokens_seen": 15641440,
+      "step": 215
+    },
+    {
+      "epoch": 3.5482866043613708,
+      "grad_norm": 0.19587557017803192,
+      "learning_rate": 4.610819813755038e-05,
+      "loss": 0.2374,
+      "num_input_tokens_seen": 15728872,
+      "step": 216
+    },
+    {
+      "epoch": 3.564901349948079,
+      "grad_norm": 0.19063518941402435,
+      "learning_rate": 4.607305605927487e-05,
+      "loss": 0.1919,
+      "num_input_tokens_seen": 15798112,
+      "step": 217
+    },
+    {
+      "epoch": 3.581516095534787,
+      "grad_norm": 0.19598323106765747,
+      "learning_rate": 4.6037769548632656e-05,
+      "loss": 0.2583,
+      "num_input_tokens_seen": 15865936,
+      "step": 218
+    },
+    {
+      "epoch": 3.5981308411214954,
+      "grad_norm": 0.18066690862178802,
+      "learning_rate": 4.600233884747355e-05,
+      "loss": 0.2337,
+      "num_input_tokens_seen": 15941368,
+      "step": 219
+    },
+    {
+      "epoch": 3.6147455867082035,
+      "grad_norm": 0.16981899738311768,
+      "learning_rate": 4.5966764198635606e-05,
+      "loss": 0.1818,
+      "num_input_tokens_seen": 16028208,
+      "step": 220
+    },
+    {
+      "epoch": 3.6313603322949115,
+      "grad_norm": 0.180410236120224,
+      "learning_rate": 4.5931045845943474e-05,
+      "loss": 0.1646,
+      "num_input_tokens_seen": 16104408,
+      "step": 221
+    },
+    {
+      "epoch": 3.64797507788162,
+      "grad_norm": 0.19180680811405182,
+      "learning_rate": 4.5895184034206765e-05,
+      "loss": 0.3263,
+      "num_input_tokens_seen": 16156800,
+      "step": 222
+    },
+    {
+      "epoch": 3.664589823468328,
+      "grad_norm": 0.16119280457496643,
+      "learning_rate": 4.585917900921829e-05,
+      "loss": 0.2636,
+      "num_input_tokens_seen": 16256712,
+      "step": 223
+    },
+    {
+      "epoch": 3.681204569055036,
+      "grad_norm": 0.18559172749519348,
+      "learning_rate": 4.5823031017752485e-05,
+      "loss": 0.1759,
+      "num_input_tokens_seen": 16330344,
+      "step": 224
+    },
+    {
+      "epoch": 3.6978193146417446,
+      "grad_norm": 0.17767880856990814,
+      "learning_rate": 4.5786740307563636e-05,
+      "loss": 0.196,
+      "num_input_tokens_seen": 16399792,
+      "step": 225
+    },
+    {
+      "epoch": 3.7144340602284527,
+      "grad_norm": 0.17806987464427948,
+      "learning_rate": 4.575030712738419e-05,
+      "loss": 0.186,
+      "num_input_tokens_seen": 16466368,
+      "step": 226
+    },
+    {
+      "epoch": 3.7310488058151607,
+      "grad_norm": 0.1952792853116989,
+      "learning_rate": 4.571373172692309e-05,
+      "loss": 0.1789,
+      "num_input_tokens_seen": 16530976,
+      "step": 227
+    },
+    {
+      "epoch": 3.7476635514018692,
+      "grad_norm": 0.1774374544620514,
+      "learning_rate": 4.567701435686404e-05,
+      "loss": 0.1929,
+      "num_input_tokens_seen": 16600216,
+      "step": 228
+    },
+    {
+      "epoch": 3.7642782969885773,
+      "grad_norm": 0.18798600137233734,
+      "learning_rate": 4.5640155268863796e-05,
+      "loss": 0.2268,
+      "num_input_tokens_seen": 16673192,
+      "step": 229
+    },
+    {
+      "epoch": 3.7808930425752854,
+      "grad_norm": 0.2022520750761032,
+      "learning_rate": 4.5603154715550386e-05,
+      "loss": 0.1716,
+      "num_input_tokens_seen": 16739912,
+      "step": 230
+    },
+    {
+      "epoch": 3.797507788161994,
+      "grad_norm": 0.15170948207378387,
+      "learning_rate": 4.55660129505215e-05,
+      "loss": 0.1844,
+      "num_input_tokens_seen": 16834632,
+      "step": 231
+    },
+    {
+      "epoch": 3.814122533748702,
+      "grad_norm": 0.16655084490776062,
+      "learning_rate": 4.5528730228342605e-05,
+      "loss": 0.1899,
+      "num_input_tokens_seen": 16914728,
+      "step": 232
+    },
+    {
+      "epoch": 3.83073727933541,
+      "grad_norm": 0.19025221467018127,
+      "learning_rate": 4.549130680454532e-05,
+      "loss": 0.2214,
+      "num_input_tokens_seen": 17014304,
+      "step": 233
+    },
+    {
+      "epoch": 3.8473520249221185,
+      "grad_norm": 0.17126557230949402,
+      "learning_rate": 4.545374293562559e-05,
+      "loss": 0.2062,
+      "num_input_tokens_seen": 17106664,
+      "step": 234
+    },
+    {
+      "epoch": 3.8639667705088265,
+      "grad_norm": 0.16162410378456116,
+      "learning_rate": 4.541603887904198e-05,
+      "loss": 0.2016,
+      "num_input_tokens_seen": 17193744,
+      "step": 235
+    },
+    {
+      "epoch": 3.8805815160955346,
+      "grad_norm": 0.2067136913537979,
+      "learning_rate": 4.537819489321386e-05,
+      "loss": 0.1992,
+      "num_input_tokens_seen": 17254656,
+      "step": 236
+    },
+    {
+      "epoch": 3.897196261682243,
+      "grad_norm": 0.200433611869812,
+      "learning_rate": 4.534021123751968e-05,
+      "loss": 0.1961,
+      "num_input_tokens_seen": 17325896,
+      "step": 237
+    },
+    {
+      "epoch": 3.913811007268951,
+      "grad_norm": 0.2062034010887146,
+      "learning_rate": 4.5302088172295156e-05,
+      "loss": 0.2302,
+      "num_input_tokens_seen": 17394424,
+      "step": 238
+    },
+    {
+      "epoch": 3.930425752855659,
+      "grad_norm": 0.1928798407316208,
+      "learning_rate": 4.526382595883152e-05,
+      "loss": 0.1846,
+      "num_input_tokens_seen": 17456352,
+      "step": 239
+    },
+    {
+      "epoch": 3.9470404984423677,
+      "grad_norm": 0.2011859118938446,
+      "learning_rate": 4.522542485937369e-05,
+      "loss": 0.1879,
+      "num_input_tokens_seen": 17519168,
+      "step": 240
+    },
+    {
+      "epoch": 3.9636552440290758,
+      "grad_norm": 0.20441657304763794,
+      "learning_rate": 4.51868851371185e-05,
+      "loss": 0.206,
+      "num_input_tokens_seen": 17585144,
+      "step": 241
+    },
+    {
+      "epoch": 3.980269989615784,
+      "grad_norm": 0.18314018845558167,
+      "learning_rate": 4.5148207056212896e-05,
+      "loss": 0.1676,
+      "num_input_tokens_seen": 17662024,
+      "step": 242
+    },
+    {
+      "epoch": 3.9968847352024923,
+      "grad_norm": 0.21530692279338837,
+      "learning_rate": 4.5109390881752114e-05,
+      "loss": 0.1961,
+      "num_input_tokens_seen": 17724360,
+      "step": 243
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.38912200927734375,
+      "learning_rate": 4.5070436879777865e-05,
+      "loss": 0.185,
+      "num_input_tokens_seen": 17746200,
+      "step": 244
+    },
+    {
+      "epoch": 4.0166147455867085,
+      "grad_norm": 0.15166164934635162,
+      "learning_rate": 4.503134531727652e-05,
+      "loss": 0.1674,
+      "num_input_tokens_seen": 17830760,
+      "step": 245
+    },
+    {
+      "epoch": 4.033229491173416,
+      "grad_norm": 0.1999833583831787,
+      "learning_rate": 4.499211646217727e-05,
+      "loss": 0.1739,
+      "num_input_tokens_seen": 17903840,
+      "step": 246
+    },
+    {
+      "epoch": 4.049844236760125,
+      "grad_norm": 0.2024000585079193,
+      "learning_rate": 4.495275058335029e-05,
+      "loss": 0.1753,
+      "num_input_tokens_seen": 17990448,
+      "step": 247
+    },
+    {
+      "epoch": 4.066458982346833,
+      "grad_norm": 0.22637376189231873,
+      "learning_rate": 4.491324795060491e-05,
+      "loss": 0.1896,
+      "num_input_tokens_seen": 18069520,
+      "step": 248
+    },
+    {
+      "epoch": 4.083073727933541,
+      "grad_norm": 0.24361123144626617,
+      "learning_rate": 4.487360883468775e-05,
+      "loss": 0.1688,
+      "num_input_tokens_seen": 18129128,
+      "step": 249
+    },
+    {
+      "epoch": 4.099688473520249,
+      "grad_norm": 0.21949416399002075,
+      "learning_rate": 4.4833833507280884e-05,
+      "loss": 0.1928,
+      "num_input_tokens_seen": 18202472,
+      "step": 250
+    },
+    {
+      "epoch": 4.116303219106958,
+      "grad_norm": 0.22039519250392914,
+      "learning_rate": 4.4793922240999933e-05,
+      "loss": 0.1737,
+      "num_input_tokens_seen": 18267232,
+      "step": 251
+    },
+    {
+      "epoch": 4.132917964693665,
+      "grad_norm": 0.23173294961452484,
+      "learning_rate": 4.4753875309392266e-05,
+      "loss": 0.1883,
+      "num_input_tokens_seen": 18325216,
+      "step": 252
+    },
+    {
+      "epoch": 4.149532710280374,
+      "grad_norm": 0.24100351333618164,
+      "learning_rate": 4.471369298693505e-05,
+      "loss": 0.2042,
+      "num_input_tokens_seen": 18406184,
+      "step": 253
+    },
+    {
+      "epoch": 4.166147455867082,
+      "grad_norm": 0.1888919323682785,
+      "learning_rate": 4.467337554903344e-05,
+      "loss": 0.1656,
+      "num_input_tokens_seen": 18481056,
+      "step": 254
+    },
+    {
+      "epoch": 4.18276220145379,
+      "grad_norm": 0.17849119007587433,
+      "learning_rate": 4.463292327201862e-05,
+      "loss": 0.1454,
+      "num_input_tokens_seen": 18554864,
+      "step": 255
+    },
+    {
+      "epoch": 4.1993769470404985,
+      "grad_norm": 0.24600732326507568,
+      "learning_rate": 4.4592336433146e-05,
+      "loss": 0.2039,
+      "num_input_tokens_seen": 18612120,
+      "step": 256
+    },
+    {
+      "epoch": 4.215991692627207,
+      "grad_norm": 0.23695628345012665,
+      "learning_rate": 4.4551615310593195e-05,
+      "loss": 0.2112,
+      "num_input_tokens_seen": 18710408,
+      "step": 257
+    },
+    {
+      "epoch": 4.232606438213915,
+      "grad_norm": 0.2511826753616333,
+      "learning_rate": 4.451076018345825e-05,
+      "loss": 0.1831,
+      "num_input_tokens_seen": 18769400,
+      "step": 258
+    },
+    {
+      "epoch": 4.249221183800623,
+      "grad_norm": 0.1971820890903473,
+      "learning_rate": 4.4469771331757604e-05,
+      "loss": 0.1722,
+      "num_input_tokens_seen": 18849704,
+      "step": 259
+    },
+    {
+      "epoch": 4.265835929387332,
+      "grad_norm": 0.23203876614570618,
+      "learning_rate": 4.442864903642428e-05,
+      "loss": 0.1981,
+      "num_input_tokens_seen": 18943328,
+      "step": 260
+    },
+    {
+      "epoch": 4.282450674974039,
+      "grad_norm": 0.23434185981750488,
+      "learning_rate": 4.4387393579305865e-05,
+      "loss": 0.2014,
+      "num_input_tokens_seen": 19022536,
+      "step": 261
+    },
+    {
+      "epoch": 4.299065420560748,
+      "grad_norm": 0.2373885214328766,
+      "learning_rate": 4.434600524316266e-05,
+      "loss": 0.167,
+      "num_input_tokens_seen": 19089200,
+      "step": 262
+    },
+    {
+      "epoch": 4.315680166147456,
+      "grad_norm": 0.20130722224712372,
+      "learning_rate": 4.430448431166567e-05,
+      "loss": 0.2747,
+      "num_input_tokens_seen": 19171216,
+      "step": 263
+    },
+    {
+      "epoch": 4.332294911734164,
+      "grad_norm": 0.17947593331336975,
+      "learning_rate": 4.426283106939474e-05,
+      "loss": 0.1508,
+      "num_input_tokens_seen": 19271872,
+      "step": 264
+    },
+    {
+      "epoch": 4.348909657320872,
+      "grad_norm": 0.23321041464805603,
+      "learning_rate": 4.4221045801836494e-05,
+      "loss": 0.2585,
+      "num_input_tokens_seen": 19342984,
+      "step": 265
+    },
+    {
+      "epoch": 4.365524402907581,
+      "grad_norm": 0.26581740379333496,
+      "learning_rate": 4.41791287953825e-05,
+      "loss": 0.1789,
+      "num_input_tokens_seen": 19391640,
+      "step": 266
+    },
+    {
+      "epoch": 4.382139148494288,
+      "grad_norm": 0.20715415477752686,
+      "learning_rate": 4.4137080337327205e-05,
+      "loss": 0.1953,
+      "num_input_tokens_seen": 19463232,
+      "step": 267
+    },
+    {
+      "epoch": 4.398753894080997,
+      "grad_norm": 0.20641866326332092,
+      "learning_rate": 4.4094900715866064e-05,
+      "loss": 0.1752,
+      "num_input_tokens_seen": 19523728,
+      "step": 268
+    },
+    {
+      "epoch": 4.415368639667705,
+      "grad_norm": 0.23343385756015778,
+      "learning_rate": 4.4052590220093446e-05,
+      "loss": 0.1904,
+      "num_input_tokens_seen": 19598960,
+      "step": 269
+    },
+    {
+      "epoch": 4.431983385254413,
+      "grad_norm": 0.20117436349391937,
+      "learning_rate": 4.401014914000078e-05,
+      "loss": 0.1801,
+      "num_input_tokens_seen": 19666136,
+      "step": 270
+    },
+    {
+      "epoch": 4.4485981308411215,
+      "grad_norm": 0.24009813368320465,
+      "learning_rate": 4.3967577766474455e-05,
+      "loss": 0.1798,
+      "num_input_tokens_seen": 19728600,
+      "step": 271
+    },
+    {
+      "epoch": 4.46521287642783,
+      "grad_norm": 0.2242031991481781,
+      "learning_rate": 4.3924876391293915e-05,
+      "loss": 0.2221,
+      "num_input_tokens_seen": 19801032,
+      "step": 272
+    },
+    {
+      "epoch": 4.481827622014538,
+      "grad_norm": 0.22890391945838928,
+      "learning_rate": 4.3882045307129594e-05,
+      "loss": 0.1906,
+      "num_input_tokens_seen": 19885496,
+      "step": 273
+    },
+    {
+      "epoch": 4.498442367601246,
+      "grad_norm": 0.21996937692165375,
+      "learning_rate": 4.383908480754095e-05,
+      "loss": 0.1775,
+      "num_input_tokens_seen": 19952072,
+      "step": 274
+    },
+    {
+      "epoch": 4.515057113187955,
+      "grad_norm": 0.1860388070344925,
+      "learning_rate": 4.379599518697444e-05,
+      "loss": 0.1593,
+      "num_input_tokens_seen": 20026536,
+      "step": 275
+    },
+    {
+      "epoch": 4.531671858774662,
+      "grad_norm": 0.20987707376480103,
+      "learning_rate": 4.375277674076149e-05,
+      "loss": 0.1409,
+      "num_input_tokens_seen": 20079112,
+      "step": 276
+    },
+    {
+      "epoch": 4.548286604361371,
+      "grad_norm": 0.21347324550151825,
+      "learning_rate": 4.3709429765116504e-05,
+      "loss": 0.2701,
+      "num_input_tokens_seen": 20144264,
+      "step": 277
+    },
+    {
+      "epoch": 4.564901349948079,
+      "grad_norm": 0.27563896775245667,
+      "learning_rate": 4.366595455713479e-05,
+      "loss": 0.1856,
+      "num_input_tokens_seen": 20207568,
+      "step": 278
+    },
+    {
+      "epoch": 4.581516095534787,
+      "grad_norm": 0.21850791573524475,
+      "learning_rate": 4.3622351414790554e-05,
+      "loss": 0.2204,
+      "num_input_tokens_seen": 20292376,
+      "step": 279
+    },
+    {
+      "epoch": 4.598130841121495,
+      "grad_norm": 0.19672711193561554,
+      "learning_rate": 4.357862063693486e-05,
+      "loss": 0.1397,
+      "num_input_tokens_seen": 20383048,
+      "step": 280
+    },
+    {
+      "epoch": 4.614745586708204,
+      "grad_norm": 0.23507343232631683,
+      "learning_rate": 4.353476252329356e-05,
+      "loss": 0.1655,
+      "num_input_tokens_seen": 20463376,
+      "step": 281
+    },
+    {
+      "epoch": 4.6313603322949115,
+      "grad_norm": 0.23900464177131653,
+      "learning_rate": 4.349077737446525e-05,
+      "loss": 0.1511,
+      "num_input_tokens_seen": 20537808,
+      "step": 282
+    },
+    {
+      "epoch": 4.64797507788162,
+      "grad_norm": 0.2231033891439438,
+      "learning_rate": 4.344666549191921e-05,
+      "loss": 0.1783,
+      "num_input_tokens_seen": 20605496,
+      "step": 283
+    },
+    {
+      "epoch": 4.6645898234683285,
+      "grad_norm": 0.20772908627986908,
+      "learning_rate": 4.3402427177993366e-05,
+      "loss": 0.2135,
+      "num_input_tokens_seen": 20692096,
+      "step": 284
+    },
+    {
+      "epoch": 4.681204569055036,
+      "grad_norm": 0.24927115440368652,
+      "learning_rate": 4.335806273589214e-05,
+      "loss": 0.1919,
+      "num_input_tokens_seen": 20762800,
+      "step": 285
+    },
+    {
+      "epoch": 4.697819314641745,
+      "grad_norm": 0.2112305462360382,
+      "learning_rate": 4.3313572469684474e-05,
+      "loss": 0.1546,
+      "num_input_tokens_seen": 20831584,
+      "step": 286
+    },
+    {
+      "epoch": 4.714434060228453,
+      "grad_norm": 0.22639551758766174,
+      "learning_rate": 4.326895668430166e-05,
+      "loss": 0.124,
+      "num_input_tokens_seen": 20897320,
+      "step": 287
+    },
+    {
+      "epoch": 4.731048805815161,
+      "grad_norm": 0.2295934110879898,
+      "learning_rate": 4.3224215685535294e-05,
+      "loss": 0.1639,
+      "num_input_tokens_seen": 20966136,
+      "step": 288
+    },
+    {
+      "epoch": 4.747663551401869,
+      "grad_norm": 0.2341577112674713,
+      "learning_rate": 4.317934978003517e-05,
+      "loss": 0.1584,
+      "num_input_tokens_seen": 21034800,
+      "step": 289
+    },
+    {
+      "epoch": 4.764278296988578,
+      "grad_norm": 0.2542404234409332,
+      "learning_rate": 4.313435927530719e-05,
+      "loss": 0.1918,
+      "num_input_tokens_seen": 21098672,
+      "step": 290
+    },
+    {
+      "epoch": 4.780893042575285,
+      "grad_norm": 0.23311223089694977,
+      "learning_rate": 4.3089244479711236e-05,
+      "loss": 0.1597,
+      "num_input_tokens_seen": 21177632,
+      "step": 291
+    },
+    {
+      "epoch": 4.797507788161994,
+      "grad_norm": 0.2642923593521118,
+      "learning_rate": 4.304400570245906e-05,
+      "loss": 0.1847,
+      "num_input_tokens_seen": 21240896,
+      "step": 292
+    },
+    {
+      "epoch": 4.814122533748702,
+      "grad_norm": 0.18841278553009033,
+      "learning_rate": 4.299864325361217e-05,
+      "loss": 0.1472,
+      "num_input_tokens_seen": 21322984,
+      "step": 293
+    },
+    {
+      "epoch": 4.83073727933541,
+      "grad_norm": 0.22440434992313385,
+      "learning_rate": 4.295315744407972e-05,
+      "loss": 0.1607,
+      "num_input_tokens_seen": 21389128,
+      "step": 294
+    },
+    {
+      "epoch": 4.8473520249221185,
+      "grad_norm": 0.22145289182662964,
+      "learning_rate": 4.290754858561637e-05,
+      "loss": 0.1851,
+      "num_input_tokens_seen": 21469912,
+      "step": 295
+    },
+    {
+      "epoch": 4.863966770508826,
+      "grad_norm": 0.22817087173461914,
+      "learning_rate": 4.2861816990820084e-05,
+      "loss": 0.1531,
+      "num_input_tokens_seen": 21540320,
+      "step": 296
+    },
+    {
+      "epoch": 4.880581516095535,
+      "grad_norm": 0.22014038264751434,
+      "learning_rate": 4.281596297313013e-05,
+      "loss": 0.1815,
+      "num_input_tokens_seen": 21626312,
+      "step": 297
+    },
+    {
+      "epoch": 4.897196261682243,
+      "grad_norm": 0.2234148383140564,
+      "learning_rate": 4.2769986846824815e-05,
+      "loss": 0.1667,
+      "num_input_tokens_seen": 21702792,
+      "step": 298
+    },
+    {
+      "epoch": 4.913811007268951,
+      "grad_norm": 0.2851375341415405,
+      "learning_rate": 4.272388892701934e-05,
+      "loss": 0.1805,
+      "num_input_tokens_seen": 21771880,
+      "step": 299
+    },
+    {
+      "epoch": 4.930425752855659,
+      "grad_norm": 0.2221265286207199,
+      "learning_rate": 4.267766952966369e-05,
+      "loss": 0.1653,
+      "num_input_tokens_seen": 21844024,
+      "step": 300
+    },
+    {
+      "epoch": 4.947040498442368,
+      "grad_norm": 0.20688939094543457,
+      "learning_rate": 4.2631328971540444e-05,
+      "loss": 0.1654,
+      "num_input_tokens_seen": 21925632,
+      "step": 301
+    },
+    {
+      "epoch": 4.963655244029075,
+      "grad_norm": 0.2270977795124054,
+      "learning_rate": 4.2584867570262597e-05,
+      "loss": 0.1774,
+      "num_input_tokens_seen": 21981952,
+      "step": 302
+    },
+    {
+      "epoch": 4.980269989615784,
+      "grad_norm": 0.18652501702308655,
+      "learning_rate": 4.25382856442714e-05,
+      "loss": 0.1452,
+      "num_input_tokens_seen": 22070440,
+      "step": 303
+    },
+    {
+      "epoch": 4.996884735202492,
+      "grad_norm": 0.19792407751083374,
+      "learning_rate": 4.249158351283414e-05,
+      "loss": 0.1806,
+      "num_input_tokens_seen": 22170184,
+      "step": 304
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.5933757424354553,
+      "learning_rate": 4.244476149604201e-05,
+      "loss": 0.2115,
+      "num_input_tokens_seen": 22181856,
+      "step": 305
+    },
+    {
+      "epoch": 5.0166147455867085,
+      "grad_norm": 0.22399979829788208,
+      "learning_rate": 4.2397819914807856e-05,
+      "loss": 0.1614,
+      "num_input_tokens_seen": 22256808,
+      "step": 306
+    },
+    {
+      "epoch": 5.033229491173416,
+      "grad_norm": 0.2595834732055664,
+      "learning_rate": 4.2350759090864046e-05,
+      "loss": 0.1838,
+      "num_input_tokens_seen": 22325224,
+      "step": 307
+    },
+    {
+      "epoch": 5.049844236760125,
+      "grad_norm": 0.188430517911911,
+      "learning_rate": 4.230357934676017e-05,
+      "loss": 0.1421,
+      "num_input_tokens_seen": 22389624,
+      "step": 308
+    },
+    {
+      "epoch": 5.066458982346833,
+      "grad_norm": 0.325431764125824,
+      "learning_rate": 4.225628100586093e-05,
+      "loss": 0.1852,
+      "num_input_tokens_seen": 22463872,
+      "step": 309
+    },
+    {
+      "epoch": 5.083073727933541,
+      "grad_norm": 0.27097174525260925,
+      "learning_rate": 4.220886439234385e-05,
+      "loss": 0.1526,
+      "num_input_tokens_seen": 22515824,
+      "step": 310
+    },
+    {
+      "epoch": 5.099688473520249,
+      "grad_norm": 0.21379193663597107,
+      "learning_rate": 4.2161329831197095e-05,
+      "loss": 0.151,
+      "num_input_tokens_seen": 22602336,
+      "step": 311
+    },
+    {
+      "epoch": 5.116303219106958,
+      "grad_norm": 0.24827998876571655,
+      "learning_rate": 4.211367764821722e-05,
+      "loss": 0.142,
+      "num_input_tokens_seen": 22655176,
+      "step": 312
+    },
+    {
+      "epoch": 5.132917964693665,
+      "grad_norm": 0.26246964931488037,
+      "learning_rate": 4.2065908170006955e-05,
+      "loss": 0.1589,
+      "num_input_tokens_seen": 22728680,
+      "step": 313
+    },
+    {
+      "epoch": 5.149532710280374,
+      "grad_norm": 0.24459198117256165,
+      "learning_rate": 4.201802172397295e-05,
+      "loss": 0.1435,
+      "num_input_tokens_seen": 22806784,
+      "step": 314
+    },
+    {
+      "epoch": 5.166147455867082,
+      "grad_norm": 0.26540517807006836,
+      "learning_rate": 4.197001863832355e-05,
+      "loss": 0.1447,
+      "num_input_tokens_seen": 22880648,
+      "step": 315
+    },
+    {
+      "epoch": 5.18276220145379,
+      "grad_norm": 0.25646644830703735,
+      "learning_rate": 4.192189924206652e-05,
+      "loss": 0.1418,
+      "num_input_tokens_seen": 22953184,
+      "step": 316
+    },
+    {
+      "epoch": 5.1993769470404985,
+      "grad_norm": 0.2358384132385254,
+      "learning_rate": 4.187366386500683e-05,
+      "loss": 0.1845,
+      "num_input_tokens_seen": 23037392,
+      "step": 317
+    },
+    {
+      "epoch": 5.215991692627207,
+      "grad_norm": 0.2270258218050003,
+      "learning_rate": 4.182531283774434e-05,
+      "loss": 0.2668,
+      "num_input_tokens_seen": 23086552,
+      "step": 318
+    },
+    {
+      "epoch": 5.232606438213915,
+      "grad_norm": 0.24396558105945587,
+      "learning_rate": 4.177684649167158e-05,
+      "loss": 0.1567,
+      "num_input_tokens_seen": 23153152,
+      "step": 319
+    },
+    {
+      "epoch": 5.249221183800623,
+      "grad_norm": 0.2542375922203064,
+      "learning_rate": 4.172826515897146e-05,
+      "loss": 0.1617,
+      "num_input_tokens_seen": 23240928,
+      "step": 320
+    },
+    {
+      "epoch": 5.265835929387332,
+      "grad_norm": 0.2268146276473999,
+      "learning_rate": 4.1679569172614996e-05,
+      "loss": 0.1573,
+      "num_input_tokens_seen": 23325912,
+      "step": 321
+    },
+    {
+      "epoch": 5.282450674974039,
+      "grad_norm": 0.26405712962150574,
+      "learning_rate": 4.163075886635902e-05,
+      "loss": 0.1738,
+      "num_input_tokens_seen": 23401952,
+      "step": 322
+    },
+    {
+      "epoch": 5.299065420560748,
+      "grad_norm": 0.2599943280220032,
+      "learning_rate": 4.1581834574743915e-05,
+      "loss": 0.1433,
+      "num_input_tokens_seen": 23463760,
+      "step": 323
+    },
+    {
+      "epoch": 5.315680166147456,
+      "grad_norm": 0.24650180339813232,
+      "learning_rate": 4.1532796633091296e-05,
+      "loss": 0.1511,
+      "num_input_tokens_seen": 23535272,
+      "step": 324
+    },
+    {
+      "epoch": 5.332294911734164,
+      "grad_norm": 0.22445832192897797,
+      "learning_rate": 4.148364537750172e-05,
+      "loss": 0.1296,
+      "num_input_tokens_seen": 23607752,
+      "step": 325
+    },
+    {
+      "epoch": 5.348909657320872,
+      "grad_norm": 0.20715995132923126,
+      "learning_rate": 4.14343811448524e-05,
+      "loss": 0.142,
+      "num_input_tokens_seen": 23674872,
+      "step": 326
+    },
+    {
+      "epoch": 5.365524402907581,
+      "grad_norm": 0.2707969546318054,
+      "learning_rate": 4.138500427279485e-05,
+      "loss": 0.1679,
+      "num_input_tokens_seen": 23736384,
+      "step": 327
+    },
+    {
+      "epoch": 5.382139148494288,
+      "grad_norm": 0.2017841637134552,
+      "learning_rate": 4.133551509975264e-05,
+      "loss": 0.1346,
+      "num_input_tokens_seen": 23835000,
+      "step": 328
+    },
+    {
+      "epoch": 5.398753894080997,
+      "grad_norm": 0.21116195619106293,
+      "learning_rate": 4.128591396491901e-05,
+      "loss": 0.1364,
+      "num_input_tokens_seen": 23912552,
+      "step": 329
+    },
+    {
+      "epoch": 5.415368639667705,
+      "grad_norm": 0.2331131547689438,
+      "learning_rate": 4.123620120825459e-05,
+      "loss": 0.1719,
+      "num_input_tokens_seen": 23987368,
+      "step": 330
+    },
+    {
+      "epoch": 5.431983385254413,
+      "grad_norm": 0.27115845680236816,
+      "learning_rate": 4.118637717048506e-05,
+      "loss": 0.1468,
+      "num_input_tokens_seen": 24050848,
+      "step": 331
+    },
+    {
+      "epoch": 5.4485981308411215,
+      "grad_norm": 0.21654783189296722,
+      "learning_rate": 4.113644219309877e-05,
+      "loss": 0.1418,
+      "num_input_tokens_seen": 24146104,
+      "step": 332
+    },
+    {
+      "epoch": 5.46521287642783,
+      "grad_norm": 0.2783348560333252,
+      "learning_rate": 4.1086396618344476e-05,
+      "loss": 0.1502,
+      "num_input_tokens_seen": 24194184,
+      "step": 333
+    },
+    {
+      "epoch": 5.481827622014538,
+      "grad_norm": 0.23255467414855957,
+      "learning_rate": 4.1036240789228954e-05,
+      "loss": 0.1571,
+      "num_input_tokens_seen": 24275368,
+      "step": 334
+    },
+    {
+      "epoch": 5.498442367601246,
+      "grad_norm": 0.2655453681945801,
+      "learning_rate": 4.098597504951462e-05,
+      "loss": 0.1607,
+      "num_input_tokens_seen": 24329192,
+      "step": 335
+    },
+    {
+      "epoch": 5.515057113187955,
+      "grad_norm": 0.23245719075202942,
+      "learning_rate": 4.093559974371725e-05,
+      "loss": 0.1453,
+      "num_input_tokens_seen": 24426696,
+      "step": 336
+    },
+    {
+      "epoch": 5.531671858774662,
+      "grad_norm": 0.20036327838897705,
+      "learning_rate": 4.088511521710352e-05,
+      "loss": 0.1411,
+      "num_input_tokens_seen": 24514344,
+      "step": 337
+    },
+    {
+      "epoch": 5.548286604361371,
+      "grad_norm": 0.32294195890426636,
+      "learning_rate": 4.083452181568875e-05,
+      "loss": 0.1467,
+      "num_input_tokens_seen": 24584464,
+      "step": 338
+    },
+    {
+      "epoch": 5.564901349948079,
+      "grad_norm": 0.23969624936580658,
+      "learning_rate": 4.0783819886234445e-05,
+      "loss": 0.1304,
+      "num_input_tokens_seen": 24660600,
+      "step": 339
+    },
+    {
+      "epoch": 5.581516095534787,
+      "grad_norm": 0.2539989948272705,
+      "learning_rate": 4.073300977624594e-05,
+      "loss": 0.1374,
+      "num_input_tokens_seen": 24717088,
+      "step": 340
+    },
+    {
+      "epoch": 5.598130841121495,
+      "grad_norm": 0.26608580350875854,
+      "learning_rate": 4.068209183397004e-05,
+      "loss": 0.1519,
+      "num_input_tokens_seen": 24775352,
+      "step": 341
+    },
+    {
+      "epoch": 5.614745586708204,
+      "grad_norm": 0.2161550372838974,
+      "learning_rate": 4.063106640839264e-05,
+      "loss": 0.1409,
+      "num_input_tokens_seen": 24860072,
+      "step": 342
+    },
+    {
+      "epoch": 5.6313603322949115,
+      "grad_norm": 0.22482600808143616,
+      "learning_rate": 4.057993384923626e-05,
+      "loss": 0.1393,
+      "num_input_tokens_seen": 24947856,
+      "step": 343
+    },
+    {
+      "epoch": 5.64797507788162,
+      "grad_norm": 0.2367829829454422,
+      "learning_rate": 4.052869450695776e-05,
+      "loss": 0.1539,
+      "num_input_tokens_seen": 25024992,
+      "step": 344
+    },
+    {
+      "epoch": 5.6645898234683285,
+      "grad_norm": 0.29229775071144104,
+      "learning_rate": 4.047734873274586e-05,
+      "loss": 0.1522,
+      "num_input_tokens_seen": 25092248,
+      "step": 345
+    },
+    {
+      "epoch": 5.681204569055036,
+      "grad_norm": 0.2589828670024872,
+      "learning_rate": 4.042589687851872e-05,
+      "loss": 0.1493,
+      "num_input_tokens_seen": 25170496,
+      "step": 346
+    },
+    {
+      "epoch": 5.697819314641745,
+      "grad_norm": 0.23003339767456055,
+      "learning_rate": 4.037433929692161e-05,
+      "loss": 0.1529,
+      "num_input_tokens_seen": 25268720,
+      "step": 347
+    },
+    {
+      "epoch": 5.714434060228453,
+      "grad_norm": 0.26932114362716675,
+      "learning_rate": 4.0322676341324415e-05,
+      "loss": 0.1499,
+      "num_input_tokens_seen": 25332688,
+      "step": 348
+    },
+    {
+      "epoch": 5.731048805815161,
+      "grad_norm": 0.27059391140937805,
+      "learning_rate": 4.027090836581925e-05,
+      "loss": 0.173,
+      "num_input_tokens_seen": 25413904,
+      "step": 349
+    },
+    {
+      "epoch": 5.747663551401869,
+      "grad_norm": 0.24265804886817932,
+      "learning_rate": 4.021903572521802e-05,
+      "loss": 0.1531,
+      "num_input_tokens_seen": 25503720,
+      "step": 350
+    },
+    {
+      "epoch": 5.764278296988578,
+      "grad_norm": 0.28688696026802063,
+      "learning_rate": 4.0167058775049996e-05,
+      "loss": 0.1615,
+      "num_input_tokens_seen": 25568560,
+      "step": 351
+    },
+    {
+      "epoch": 5.780893042575285,
+      "grad_norm": 0.26201075315475464,
+      "learning_rate": 4.011497787155938e-05,
+      "loss": 0.1452,
+      "num_input_tokens_seen": 25635184,
+      "step": 352
+    },
+    {
+      "epoch": 5.797507788161994,
+      "grad_norm": 0.22841767966747284,
+      "learning_rate": 4.006279337170283e-05,
+      "loss": 0.148,
+      "num_input_tokens_seen": 25719768,
+      "step": 353
+    },
+    {
+      "epoch": 5.814122533748702,
+      "grad_norm": 0.28246188163757324,
+      "learning_rate": 4.0010505633147106e-05,
+      "loss": 0.1446,
+      "num_input_tokens_seen": 25795016,
+      "step": 354
+    },
+    {
+      "epoch": 5.83073727933541,
+      "grad_norm": 0.2533949315547943,
+      "learning_rate": 3.995811501426648e-05,
+      "loss": 0.1297,
+      "num_input_tokens_seen": 25863184,
+      "step": 355
+    },
+    {
+      "epoch": 5.8473520249221185,
+      "grad_norm": 0.25117793679237366,
+      "learning_rate": 3.99056218741404e-05,
+      "loss": 0.1479,
+      "num_input_tokens_seen": 25935752,
+      "step": 356
+    },
+    {
+      "epoch": 5.863966770508826,
+      "grad_norm": 0.2759612202644348,
+      "learning_rate": 3.985302657255097e-05,
+      "loss": 0.1466,
+      "num_input_tokens_seen": 25995760,
+      "step": 357
+    },
+    {
+      "epoch": 5.880581516095535,
+      "grad_norm": 0.24218028783798218,
+      "learning_rate": 3.980032946998049e-05,
+      "loss": 0.1297,
+      "num_input_tokens_seen": 26061240,
+      "step": 358
+    },
+    {
+      "epoch": 5.897196261682243,
+      "grad_norm": 0.2628185749053955,
+      "learning_rate": 3.974753092760901e-05,
+      "loss": 0.1421,
+      "num_input_tokens_seen": 26131024,
+      "step": 359
+    },
+    {
+      "epoch": 5.913811007268951,
+      "grad_norm": 0.18742328882217407,
+      "learning_rate": 3.969463130731183e-05,
+      "loss": 0.1737,
+      "num_input_tokens_seen": 26233672,
+      "step": 360
+    },
+    {
+      "epoch": 5.930425752855659,
+      "grad_norm": 0.22187161445617676,
+      "learning_rate": 3.964163097165702e-05,
+      "loss": 0.1153,
+      "num_input_tokens_seen": 26303488,
+      "step": 361
+    },
+    {
+      "epoch": 5.947040498442368,
+      "grad_norm": 0.25965312123298645,
+      "learning_rate": 3.958853028390294e-05,
+      "loss": 0.1347,
+      "num_input_tokens_seen": 26377768,
+      "step": 362
+    },
+    {
+      "epoch": 5.963655244029075,
+      "grad_norm": 0.2957039475440979,
+      "learning_rate": 3.953532960799577e-05,
+      "loss": 0.2736,
+      "num_input_tokens_seen": 26435984,
+      "step": 363
+    },
+    {
+      "epoch": 5.980269989615784,
+      "grad_norm": 0.3063158392906189,
+      "learning_rate": 3.948202930856697e-05,
+      "loss": 0.1563,
+      "num_input_tokens_seen": 26513960,
+      "step": 364
+    },
+    {
+      "epoch": 5.996884735202492,
+      "grad_norm": 0.21576139330863953,
+      "learning_rate": 3.942862975093085e-05,
+      "loss": 0.1361,
+      "num_input_tokens_seen": 26599064,
+      "step": 365
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 0.49579551815986633,
+      "learning_rate": 3.937513130108197e-05,
+      "loss": 0.1849,
+      "num_input_tokens_seen": 26617264,
+      "step": 366
+    },
+    {
+      "epoch": 6.0166147455867085,
+      "grad_norm": 0.25051239132881165,
+      "learning_rate": 3.9321534325692726e-05,
+      "loss": 0.1452,
+      "num_input_tokens_seen": 26717024,
+      "step": 367
+    },
+    {
+      "epoch": 6.033229491173416,
+      "grad_norm": 0.2842366099357605,
+      "learning_rate": 3.92678391921108e-05,
+      "loss": 0.1361,
+      "num_input_tokens_seen": 26788120,
+      "step": 368
+    },
+    {
+      "epoch": 6.049844236760125,
+      "grad_norm": 0.32458341121673584,
+      "learning_rate": 3.92140462683566e-05,
+      "loss": 0.1537,
+      "num_input_tokens_seen": 26853760,
+      "step": 369
+    },
+    {
+      "epoch": 6.066458982346833,
+      "grad_norm": 0.27452871203422546,
+      "learning_rate": 3.916015592312082e-05,
+      "loss": 0.1354,
+      "num_input_tokens_seen": 26923848,
+      "step": 370
+    },
+    {
+      "epoch": 6.083073727933541,
+      "grad_norm": 0.2542634904384613,
+      "learning_rate": 3.9106168525761855e-05,
+      "loss": 0.1233,
+      "num_input_tokens_seen": 26976184,
+      "step": 371
+    },
+    {
+      "epoch": 6.099688473520249,
+      "grad_norm": 0.2786627411842346,
+      "learning_rate": 3.905208444630327e-05,
+      "loss": 0.125,
+      "num_input_tokens_seen": 27065712,
+      "step": 372
+    },
+    {
+      "epoch": 6.116303219106958,
+      "grad_norm": 0.19119420647621155,
+      "learning_rate": 3.899790405543129e-05,
+      "loss": 0.1475,
+      "num_input_tokens_seen": 27145472,
+      "step": 373
+    },
+    {
+      "epoch": 6.132917964693665,
+      "grad_norm": 0.25163477659225464,
+      "learning_rate": 3.894362772449226e-05,
+      "loss": 0.1235,
+      "num_input_tokens_seen": 27233904,
+      "step": 374
+    },
+    {
+      "epoch": 6.149532710280374,
+      "grad_norm": 0.2418321669101715,
+      "learning_rate": 3.888925582549006e-05,
+      "loss": 0.1117,
+      "num_input_tokens_seen": 27311512,
+      "step": 375
+    },
+    {
+      "epoch": 6.166147455867082,
+      "grad_norm": 0.26851242780685425,
+      "learning_rate": 3.883478873108361e-05,
+      "loss": 0.1226,
+      "num_input_tokens_seen": 27387400,
+      "step": 376
+    },
+    {
+      "epoch": 6.18276220145379,
+      "grad_norm": 0.280127614736557,
+      "learning_rate": 3.878022681458426e-05,
+      "loss": 0.1295,
+      "num_input_tokens_seen": 27461280,
+      "step": 377
+    },
+    {
+      "epoch": 6.1993769470404985,
+      "grad_norm": 0.255588173866272,
+      "learning_rate": 3.87255704499533e-05,
+      "loss": 0.1197,
+      "num_input_tokens_seen": 27556400,
+      "step": 378
+    },
+    {
+      "epoch": 6.215991692627207,
+      "grad_norm": 0.2888232171535492,
+      "learning_rate": 3.8670820011799315e-05,
+      "loss": 0.1417,
+      "num_input_tokens_seen": 27613664,
+      "step": 379
+    },
+    {
+      "epoch": 6.232606438213915,
+      "grad_norm": 0.28014957904815674,
+      "learning_rate": 3.861597587537568e-05,
+      "loss": 0.1565,
+      "num_input_tokens_seen": 27681024,
+      "step": 380
+    },
+    {
+      "epoch": 6.249221183800623,
+      "grad_norm": 0.2911868095397949,
+      "learning_rate": 3.856103841657797e-05,
+      "loss": 0.1293,
+      "num_input_tokens_seen": 27759536,
+      "step": 381
+    },
+    {
+      "epoch": 6.265835929387332,
+      "grad_norm": 0.19534167647361755,
+      "learning_rate": 3.850600801194138e-05,
+      "loss": 0.0967,
+      "num_input_tokens_seen": 27857288,
+      "step": 382
+    },
+    {
+      "epoch": 6.282450674974039,
+      "grad_norm": 0.2868412733078003,
+      "learning_rate": 3.8450885038638127e-05,
+      "loss": 0.1395,
+      "num_input_tokens_seen": 27940528,
+      "step": 383
+    },
+    {
+      "epoch": 6.299065420560748,
+      "grad_norm": 0.3421030342578888,
+      "learning_rate": 3.8395669874474915e-05,
+      "loss": 0.1402,
+      "num_input_tokens_seen": 28033824,
+      "step": 384
+    },
+    {
+      "epoch": 6.315680166147456,
+      "grad_norm": 0.2819141447544098,
+      "learning_rate": 3.834036289789029e-05,
+      "loss": 0.1137,
+      "num_input_tokens_seen": 28096192,
+      "step": 385
+    },
+    {
+      "epoch": 6.332294911734164,
+      "grad_norm": 0.22595854103565216,
+      "learning_rate": 3.828496448795207e-05,
+      "loss": 0.1143,
+      "num_input_tokens_seen": 28181256,
+      "step": 386
+    },
+    {
+      "epoch": 6.348909657320872,
+      "grad_norm": 0.2450927495956421,
+      "learning_rate": 3.822947502435477e-05,
+      "loss": 0.121,
+      "num_input_tokens_seen": 28245480,
+      "step": 387
+    },
+    {
+      "epoch": 6.365524402907581,
+      "grad_norm": 0.2552040219306946,
+      "learning_rate": 3.8173894887416945e-05,
+      "loss": 0.1183,
+      "num_input_tokens_seen": 28307200,
+      "step": 388
+    },
+    {
+      "epoch": 6.382139148494288,
+      "grad_norm": 0.3974437415599823,
+      "learning_rate": 3.811822445807863e-05,
+      "loss": 0.1423,
+      "num_input_tokens_seen": 28384640,
+      "step": 389
+    },
+    {
+      "epoch": 6.398753894080997,
+      "grad_norm": 0.2750483453273773,
+      "learning_rate": 3.8062464117898724e-05,
+      "loss": 0.1068,
+      "num_input_tokens_seen": 28447992,
+      "step": 390
+    },
+    {
+      "epoch": 6.415368639667705,
+      "grad_norm": 0.3021928071975708,
+      "learning_rate": 3.800661424905235e-05,
+      "loss": 0.1233,
+      "num_input_tokens_seen": 28513856,
+      "step": 391
+    },
+    {
+      "epoch": 6.431983385254413,
+      "grad_norm": 0.4087376296520233,
+      "learning_rate": 3.795067523432826e-05,
+      "loss": 0.1354,
+      "num_input_tokens_seen": 28596584,
+      "step": 392
+    },
+    {
+      "epoch": 6.4485981308411215,
+      "grad_norm": 0.32668039202690125,
+      "learning_rate": 3.789464745712619e-05,
+      "loss": 0.1483,
+      "num_input_tokens_seen": 28664560,
+      "step": 393
+    },
+    {
+      "epoch": 6.46521287642783,
+      "grad_norm": 0.24187126755714417,
+      "learning_rate": 3.7838531301454254e-05,
+      "loss": 0.1225,
+      "num_input_tokens_seen": 28739512,
+      "step": 394
+    },
+    {
+      "epoch": 6.481827622014538,
+      "grad_norm": 0.2861412465572357,
+      "learning_rate": 3.77823271519263e-05,
+      "loss": 0.1318,
+      "num_input_tokens_seen": 28831848,
+      "step": 395
+    },
+    {
+      "epoch": 6.498442367601246,
+      "grad_norm": 0.30458521842956543,
+      "learning_rate": 3.7726035393759285e-05,
+      "loss": 0.1337,
+      "num_input_tokens_seen": 28898408,
+      "step": 396
+    },
+    {
+      "epoch": 6.515057113187955,
+      "grad_norm": 0.2644833028316498,
+      "learning_rate": 3.76696564127706e-05,
+      "loss": 0.1283,
+      "num_input_tokens_seen": 28960224,
+      "step": 397
+    },
+    {
+      "epoch": 6.531671858774662,
+      "grad_norm": 0.3309299349784851,
+      "learning_rate": 3.761319059537548e-05,
+      "loss": 0.1548,
+      "num_input_tokens_seen": 29020568,
+      "step": 398
+    },
+    {
+      "epoch": 6.548286604361371,
+      "grad_norm": 0.2788560092449188,
+      "learning_rate": 3.755663832858432e-05,
+      "loss": 0.1154,
+      "num_input_tokens_seen": 29095448,
+      "step": 399
+    },
+    {
+      "epoch": 6.564901349948079,
+      "grad_norm": 0.26450732350349426,
+      "learning_rate": 3.7500000000000003e-05,
+      "loss": 0.1276,
+      "num_input_tokens_seen": 29186600,
+      "step": 400
+    },
+    {
+      "epoch": 6.581516095534787,
+      "grad_norm": 0.2650800049304962,
+      "learning_rate": 3.744327599781531e-05,
+      "loss": 0.2184,
+      "num_input_tokens_seen": 29258552,
+      "step": 401
+    },
+    {
+      "epoch": 6.598130841121495,
+      "grad_norm": 0.2814129590988159,
+      "learning_rate": 3.7386466710810194e-05,
+      "loss": 0.1508,
+      "num_input_tokens_seen": 29344848,
+      "step": 402
+    },
+    {
+      "epoch": 6.614745586708204,
+      "grad_norm": 0.2940891981124878,
+      "learning_rate": 3.7329572528349146e-05,
+      "loss": 0.2035,
+      "num_input_tokens_seen": 29410184,
+      "step": 403
+    },
+    {
+      "epoch": 6.6313603322949115,
+      "grad_norm": 0.25937849283218384,
+      "learning_rate": 3.727259384037852e-05,
+      "loss": 0.1308,
+      "num_input_tokens_seen": 29484928,
+      "step": 404
+    },
+    {
+      "epoch": 6.64797507788162,
+      "grad_norm": 0.26631370186805725,
+      "learning_rate": 3.721553103742388e-05,
+      "loss": 0.1349,
+      "num_input_tokens_seen": 29566432,
+      "step": 405
+    },
+    {
+      "epoch": 6.6645898234683285,
+      "grad_norm": 0.2721930146217346,
+      "learning_rate": 3.715838451058726e-05,
+      "loss": 0.1246,
+      "num_input_tokens_seen": 29634032,
+      "step": 406
+    },
+    {
+      "epoch": 6.681204569055036,
+      "grad_norm": 0.3131358325481415,
+      "learning_rate": 3.7101154651544584e-05,
+      "loss": 0.142,
+      "num_input_tokens_seen": 29681424,
+      "step": 407
+    },
+    {
+      "epoch": 6.697819314641745,
+      "grad_norm": 0.266184002161026,
+      "learning_rate": 3.704384185254288e-05,
+      "loss": 0.118,
+      "num_input_tokens_seen": 29762208,
+      "step": 408
+    },
+    {
+      "epoch": 6.714434060228453,
+      "grad_norm": 0.29790574312210083,
+      "learning_rate": 3.6986446506397666e-05,
+      "loss": 0.1286,
+      "num_input_tokens_seen": 29816280,
+      "step": 409
+    },
+    {
+      "epoch": 6.731048805815161,
+      "grad_norm": 0.2710191309452057,
+      "learning_rate": 3.692896900649021e-05,
+      "loss": 0.1211,
+      "num_input_tokens_seen": 29893040,
+      "step": 410
+    },
+    {
+      "epoch": 6.747663551401869,
+      "grad_norm": 0.46238377690315247,
+      "learning_rate": 3.6871409746764865e-05,
+      "loss": 0.1324,
+      "num_input_tokens_seen": 29971688,
+      "step": 411
+    },
+    {
+      "epoch": 6.764278296988578,
+      "grad_norm": 0.24678252637386322,
+      "learning_rate": 3.681376912172636e-05,
+      "loss": 0.1175,
+      "num_input_tokens_seen": 30051784,
+      "step": 412
+    },
+    {
+      "epoch": 6.780893042575285,
+      "grad_norm": 0.2699359655380249,
+      "learning_rate": 3.675604752643706e-05,
+      "loss": 0.1279,
+      "num_input_tokens_seen": 30146048,
+      "step": 413
+    },
+    {
+      "epoch": 6.797507788161994,
+      "grad_norm": 0.3923390805721283,
+      "learning_rate": 3.6698245356514335e-05,
+      "loss": 0.1204,
+      "num_input_tokens_seen": 30221296,
+      "step": 414
+    },
+    {
+      "epoch": 6.814122533748702,
+      "grad_norm": 0.26230940222740173,
+      "learning_rate": 3.6640363008127784e-05,
+      "loss": 0.1311,
+      "num_input_tokens_seen": 30287664,
+      "step": 415
+    },
+    {
+      "epoch": 6.83073727933541,
+      "grad_norm": 0.31548890471458435,
+      "learning_rate": 3.6582400877996546e-05,
+      "loss": 0.1417,
+      "num_input_tokens_seen": 30352816,
+      "step": 416
+    },
+    {
+      "epoch": 6.8473520249221185,
+      "grad_norm": 0.25795799493789673,
+      "learning_rate": 3.652435936338656e-05,
+      "loss": 0.1316,
+      "num_input_tokens_seen": 30439688,
+      "step": 417
+    },
+    {
+      "epoch": 6.863966770508826,
+      "grad_norm": 0.3696773648262024,
+      "learning_rate": 3.646623886210788e-05,
+      "loss": 0.1422,
+      "num_input_tokens_seen": 30506856,
+      "step": 418
+    },
+    {
+      "epoch": 6.880581516095535,
+      "grad_norm": 0.28359490633010864,
+      "learning_rate": 3.64080397725119e-05,
+      "loss": 0.1148,
+      "num_input_tokens_seen": 30565848,
+      "step": 419
+    },
+    {
+      "epoch": 6.897196261682243,
+      "grad_norm": 0.3696165978908539,
+      "learning_rate": 3.634976249348867e-05,
+      "loss": 0.1472,
+      "num_input_tokens_seen": 30633944,
+      "step": 420
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 1200,
+  "num_input_tokens_seen": 30633944,
+  "num_train_epochs": 20,
+  "save_steps": 60,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.5843307538945147e+18,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-480/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-480/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-540/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 17500,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-60/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-60/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-60/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-60/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e481e2c5f00842111d2e7b8a7a74708af234360fb1162f5d56df2a2fe7e117b8
+size 14512