Add files using upload-large-folder tool

checkpoint-108/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-108/trainer_state.json

@@ -0,0 +1,897 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.8847926267281108,
+  "eval_steps": 500,
+  "global_step": 108,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03686635944700461,
+      "grad_norm": 0.06150835379958153,
+      "learning_rate": 4.999957692144361e-05,
+      "loss": 0.663,
+      "num_input_tokens_seen": 23408,
+      "step": 1
+    },
+    {
+      "epoch": 0.07373271889400922,
+      "grad_norm": 0.05766688659787178,
+      "learning_rate": 4.999830770009406e-05,
+      "loss": 0.5847,
+      "num_input_tokens_seen": 45520,
+      "step": 2
+    },
+    {
+      "epoch": 0.11059907834101383,
+      "grad_norm": 0.0780782401561737,
+      "learning_rate": 4.9996192378909786e-05,
+      "loss": 0.5545,
+      "num_input_tokens_seen": 68904,
+      "step": 3
+    },
+    {
+      "epoch": 0.14746543778801843,
+      "grad_norm": 0.07575194537639618,
+      "learning_rate": 4.9993231029486544e-05,
+      "loss": 0.7025,
+      "num_input_tokens_seen": 89696,
+      "step": 4
+    },
+    {
+      "epoch": 0.18433179723502305,
+      "grad_norm": 0.0708586797118187,
+      "learning_rate": 4.998942375205502e-05,
+      "loss": 0.6063,
+      "num_input_tokens_seen": 118440,
+      "step": 5
+    },
+    {
+      "epoch": 0.22119815668202766,
+      "grad_norm": 0.0794093981385231,
+      "learning_rate": 4.99847706754774e-05,
+      "loss": 0.593,
+      "num_input_tokens_seen": 145072,
+      "step": 6
+    },
+    {
+      "epoch": 0.25806451612903225,
+      "grad_norm": 0.04954744130373001,
+      "learning_rate": 4.997927195724303e-05,
+      "loss": 0.4857,
+      "num_input_tokens_seen": 182112,
+      "step": 7
+    },
+    {
+      "epoch": 0.29493087557603687,
+      "grad_norm": 0.06549246609210968,
+      "learning_rate": 4.997292778346312e-05,
+      "loss": 0.5393,
+      "num_input_tokens_seen": 212192,
+      "step": 8
+    },
+    {
+      "epoch": 0.3317972350230415,
+      "grad_norm": 0.07614441215991974,
+      "learning_rate": 4.996573836886435e-05,
+      "loss": 0.7044,
+      "num_input_tokens_seen": 233728,
+      "step": 9
+    },
+    {
+      "epoch": 0.3686635944700461,
+      "grad_norm": 0.07068340480327606,
+      "learning_rate": 4.995770395678171e-05,
+      "loss": 0.6086,
+      "num_input_tokens_seen": 259776,
+      "step": 10
+    },
+    {
+      "epoch": 0.4055299539170507,
+      "grad_norm": 0.07266585528850555,
+      "learning_rate": 4.9948824819150185e-05,
+      "loss": 0.6247,
+      "num_input_tokens_seen": 284816,
+      "step": 11
+    },
+    {
+      "epoch": 0.4423963133640553,
+      "grad_norm": 0.06145177036523819,
+      "learning_rate": 4.993910125649561e-05,
+      "loss": 0.4918,
+      "num_input_tokens_seen": 309776,
+      "step": 12
+    },
+    {
+      "epoch": 0.4792626728110599,
+      "grad_norm": 0.06720694154500961,
+      "learning_rate": 4.992853359792444e-05,
+      "loss": 0.5017,
+      "num_input_tokens_seen": 339064,
+      "step": 13
+    },
+    {
+      "epoch": 0.5161290322580645,
+      "grad_norm": 0.07309022545814514,
+      "learning_rate": 4.9917122201112656e-05,
+      "loss": 0.5447,
+      "num_input_tokens_seen": 369864,
+      "step": 14
+    },
+    {
+      "epoch": 0.5529953917050692,
+      "grad_norm": 0.0778132826089859,
+      "learning_rate": 4.990486745229364e-05,
+      "loss": 0.5034,
+      "num_input_tokens_seen": 397184,
+      "step": 15
+    },
+    {
+      "epoch": 0.5898617511520737,
+      "grad_norm": 0.07727856189012527,
+      "learning_rate": 4.989176976624511e-05,
+      "loss": 0.5277,
+      "num_input_tokens_seen": 426296,
+      "step": 16
+    },
+    {
+      "epoch": 0.6267281105990783,
+      "grad_norm": 0.0700518786907196,
+      "learning_rate": 4.987782958627508e-05,
+      "loss": 0.5356,
+      "num_input_tokens_seen": 459544,
+      "step": 17
+    },
+    {
+      "epoch": 0.663594470046083,
+      "grad_norm": 0.07351912558078766,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.5505,
+      "num_input_tokens_seen": 488616,
+      "step": 18
+    },
+    {
+      "epoch": 0.7004608294930875,
+      "grad_norm": 0.08268705755472183,
+      "learning_rate": 4.9847423660363e-05,
+      "loss": 0.5557,
+      "num_input_tokens_seen": 512176,
+      "step": 19
+    },
+    {
+      "epoch": 0.7373271889400922,
+      "grad_norm": 0.08081424236297607,
+      "learning_rate": 4.983095894354858e-05,
+      "loss": 0.4576,
+      "num_input_tokens_seen": 539456,
+      "step": 20
+    },
+    {
+      "epoch": 0.7741935483870968,
+      "grad_norm": 0.07748426496982574,
+      "learning_rate": 4.9813653791033057e-05,
+      "loss": 0.3918,
+      "num_input_tokens_seen": 561384,
+      "step": 21
+    },
+    {
+      "epoch": 0.8110599078341014,
+      "grad_norm": 0.10133849829435349,
+      "learning_rate": 4.979550878853154e-05,
+      "loss": 0.531,
+      "num_input_tokens_seen": 589792,
+      "step": 22
+    },
+    {
+      "epoch": 0.847926267281106,
+      "grad_norm": 0.09528470784425735,
+      "learning_rate": 4.9776524550184965e-05,
+      "loss": 0.4159,
+      "num_input_tokens_seen": 613944,
+      "step": 23
+    },
+    {
+      "epoch": 0.8847926267281107,
+      "grad_norm": 0.08829868584871292,
+      "learning_rate": 4.975670171853926e-05,
+      "loss": 0.403,
+      "num_input_tokens_seen": 641432,
+      "step": 24
+    },
+    {
+      "epoch": 0.9216589861751152,
+      "grad_norm": 0.10020875930786133,
+      "learning_rate": 4.973604096452361e-05,
+      "loss": 0.4238,
+      "num_input_tokens_seen": 665232,
+      "step": 25
+    },
+    {
+      "epoch": 0.9585253456221198,
+      "grad_norm": 0.1256554126739502,
+      "learning_rate": 4.971454298742779e-05,
+      "loss": 0.5033,
+      "num_input_tokens_seen": 687216,
+      "step": 26
+    },
+    {
+      "epoch": 0.9953917050691244,
+      "grad_norm": 0.17123407125473022,
+      "learning_rate": 4.9692208514878444e-05,
+      "loss": 0.3842,
+      "num_input_tokens_seen": 728880,
+      "step": 27
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.27383726835250854,
+      "learning_rate": 4.966903830281449e-05,
+      "loss": 0.4988,
+      "num_input_tokens_seen": 731976,
+      "step": 28
+    },
+    {
+      "epoch": 1.0368663594470047,
+      "grad_norm": 0.148647278547287,
+      "learning_rate": 4.9645033135461494e-05,
+      "loss": 0.4489,
+      "num_input_tokens_seen": 756184,
+      "step": 29
+    },
+    {
+      "epoch": 1.0737327188940091,
+      "grad_norm": 0.11988866329193115,
+      "learning_rate": 4.962019382530521e-05,
+      "loss": 0.4999,
+      "num_input_tokens_seen": 779072,
+      "step": 30
+    },
+    {
+      "epoch": 1.1105990783410138,
+      "grad_norm": 0.10590679943561554,
+      "learning_rate": 4.9594521213063974e-05,
+      "loss": 0.3711,
+      "num_input_tokens_seen": 805672,
+      "step": 31
+    },
+    {
+      "epoch": 1.1474654377880185,
+      "grad_norm": 0.11934591829776764,
+      "learning_rate": 4.9568016167660334e-05,
+      "loss": 0.3385,
+      "num_input_tokens_seen": 828840,
+      "step": 32
+    },
+    {
+      "epoch": 1.1843317972350231,
+      "grad_norm": 0.11900116503238678,
+      "learning_rate": 4.9540679586191605e-05,
+      "loss": 0.3812,
+      "num_input_tokens_seen": 850792,
+      "step": 33
+    },
+    {
+      "epoch": 1.2211981566820276,
+      "grad_norm": 0.11279546469449997,
+      "learning_rate": 4.951251239389948e-05,
+      "loss": 0.2924,
+      "num_input_tokens_seen": 885608,
+      "step": 34
+    },
+    {
+      "epoch": 1.2580645161290323,
+      "grad_norm": 0.08340126276016235,
+      "learning_rate": 4.948351554413879e-05,
+      "loss": 0.3489,
+      "num_input_tokens_seen": 921912,
+      "step": 35
+    },
+    {
+      "epoch": 1.294930875576037,
+      "grad_norm": 0.11327336728572845,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.2921,
+      "num_input_tokens_seen": 945672,
+      "step": 36
+    },
+    {
+      "epoch": 1.3317972350230414,
+      "grad_norm": 0.10909265279769897,
+      "learning_rate": 4.942303682600178e-05,
+      "loss": 0.3421,
+      "num_input_tokens_seen": 970824,
+      "step": 37
+    },
+    {
+      "epoch": 1.368663594470046,
+      "grad_norm": 0.11239592730998993,
+      "learning_rate": 4.939155700460536e-05,
+      "loss": 0.3848,
+      "num_input_tokens_seen": 995840,
+      "step": 38
+    },
+    {
+      "epoch": 1.4055299539170507,
+      "grad_norm": 0.1034158244729042,
+      "learning_rate": 4.9359251619630886e-05,
+      "loss": 0.3434,
+      "num_input_tokens_seen": 1035880,
+      "step": 39
+    },
+    {
+      "epoch": 1.4423963133640554,
+      "grad_norm": 0.12086673080921173,
+      "learning_rate": 4.9326121764495596e-05,
+      "loss": 0.3588,
+      "num_input_tokens_seen": 1056488,
+      "step": 40
+    },
+    {
+      "epoch": 1.4792626728110598,
+      "grad_norm": 0.11408794671297073,
+      "learning_rate": 4.9292168560522014e-05,
+      "loss": 0.4021,
+      "num_input_tokens_seen": 1080080,
+      "step": 41
+    },
+    {
+      "epoch": 1.5161290322580645,
+      "grad_norm": 0.1417722851037979,
+      "learning_rate": 4.925739315689991e-05,
+      "loss": 0.3343,
+      "num_input_tokens_seen": 1095632,
+      "step": 42
+    },
+    {
+      "epoch": 1.5529953917050692,
+      "grad_norm": 0.2315663844347,
+      "learning_rate": 4.9221796730647516e-05,
+      "loss": 0.4025,
+      "num_input_tokens_seen": 1126040,
+      "step": 43
+    },
+    {
+      "epoch": 1.5898617511520738,
+      "grad_norm": 0.07856712490320206,
+      "learning_rate": 4.9185380486571595e-05,
+      "loss": 0.3337,
+      "num_input_tokens_seen": 1156656,
+      "step": 44
+    },
+    {
+      "epoch": 1.6267281105990783,
+      "grad_norm": 0.10064279288053513,
+      "learning_rate": 4.914814565722671e-05,
+      "loss": 0.3758,
+      "num_input_tokens_seen": 1183400,
+      "step": 45
+    },
+    {
+      "epoch": 1.663594470046083,
+      "grad_norm": 0.106510229408741,
+      "learning_rate": 4.9110093502873476e-05,
+      "loss": 0.3363,
+      "num_input_tokens_seen": 1209680,
+      "step": 46
+    },
+    {
+      "epoch": 1.7004608294930876,
+      "grad_norm": 0.10446175187826157,
+      "learning_rate": 4.907122531143594e-05,
+      "loss": 0.3853,
+      "num_input_tokens_seen": 1237864,
+      "step": 47
+    },
+    {
+      "epoch": 1.737327188940092,
+      "grad_norm": 0.2570022940635681,
+      "learning_rate": 4.9031542398457974e-05,
+      "loss": 0.3194,
+      "num_input_tokens_seen": 1273080,
+      "step": 48
+    },
+    {
+      "epoch": 1.7741935483870968,
+      "grad_norm": 0.0968039259314537,
+      "learning_rate": 4.8991046107058735e-05,
+      "loss": 0.2955,
+      "num_input_tokens_seen": 1300256,
+      "step": 49
+    },
+    {
+      "epoch": 1.8110599078341014,
+      "grad_norm": 0.11750692129135132,
+      "learning_rate": 4.894973780788722e-05,
+      "loss": 0.3314,
+      "num_input_tokens_seen": 1322616,
+      "step": 50
+    },
+    {
+      "epoch": 1.8479262672811059,
+      "grad_norm": 0.09568070620298386,
+      "learning_rate": 4.890761889907589e-05,
+      "loss": 0.3341,
+      "num_input_tokens_seen": 1353552,
+      "step": 51
+    },
+    {
+      "epoch": 1.8847926267281108,
+      "grad_norm": 0.11816362291574478,
+      "learning_rate": 4.88646908061933e-05,
+      "loss": 0.3202,
+      "num_input_tokens_seen": 1376456,
+      "step": 52
+    },
+    {
+      "epoch": 1.9216589861751152,
+      "grad_norm": 0.133891761302948,
+      "learning_rate": 4.8820954982195905e-05,
+      "loss": 0.2852,
+      "num_input_tokens_seen": 1398344,
+      "step": 53
+    },
+    {
+      "epoch": 1.9585253456221197,
+      "grad_norm": 0.11985364556312561,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.3413,
+      "num_input_tokens_seen": 1424264,
+      "step": 54
+    },
+    {
+      "epoch": 1.9953917050691246,
+      "grad_norm": 0.09828098118305206,
+      "learning_rate": 4.873106608932585e-05,
+      "loss": 0.2726,
+      "num_input_tokens_seen": 1460688,
+      "step": 55
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.31861889362335205,
+      "learning_rate": 4.868491606285823e-05,
+      "loss": 0.3512,
+      "num_input_tokens_seen": 1463952,
+      "step": 56
+    },
+    {
+      "epoch": 2.0368663594470044,
+      "grad_norm": 0.11089354753494263,
+      "learning_rate": 4.8637964389982926e-05,
+      "loss": 0.2953,
+      "num_input_tokens_seen": 1492600,
+      "step": 57
+    },
+    {
+      "epoch": 2.0737327188940093,
+      "grad_norm": 0.12034712731838226,
+      "learning_rate": 4.859021265983959e-05,
+      "loss": 0.2875,
+      "num_input_tokens_seen": 1514496,
+      "step": 58
+    },
+    {
+      "epoch": 2.110599078341014,
+      "grad_norm": 0.1372830867767334,
+      "learning_rate": 4.854166248864689e-05,
+      "loss": 0.3535,
+      "num_input_tokens_seen": 1541656,
+      "step": 59
+    },
+    {
+      "epoch": 2.1474654377880182,
+      "grad_norm": 0.10605650395154953,
+      "learning_rate": 4.849231551964771e-05,
+      "loss": 0.3013,
+      "num_input_tokens_seen": 1569672,
+      "step": 60
+    },
+    {
+      "epoch": 2.184331797235023,
+      "grad_norm": 0.12524856626987457,
+      "learning_rate": 4.844217342305363e-05,
+      "loss": 0.2558,
+      "num_input_tokens_seen": 1599304,
+      "step": 61
+    },
+    {
+      "epoch": 2.2211981566820276,
+      "grad_norm": 0.1151009052991867,
+      "learning_rate": 4.839123789598829e-05,
+      "loss": 0.2666,
+      "num_input_tokens_seen": 1628288,
+      "step": 62
+    },
+    {
+      "epoch": 2.258064516129032,
+      "grad_norm": 0.10297228395938873,
+      "learning_rate": 4.8339510662430046e-05,
+      "loss": 0.2982,
+      "num_input_tokens_seen": 1656384,
+      "step": 63
+    },
+    {
+      "epoch": 2.294930875576037,
+      "grad_norm": 0.11971966177225113,
+      "learning_rate": 4.828699347315356e-05,
+      "loss": 0.3746,
+      "num_input_tokens_seen": 1679344,
+      "step": 64
+    },
+    {
+      "epoch": 2.3317972350230414,
+      "grad_norm": 0.12384694814682007,
+      "learning_rate": 4.823368810567056e-05,
+      "loss": 0.2605,
+      "num_input_tokens_seen": 1703904,
+      "step": 65
+    },
+    {
+      "epoch": 2.3686635944700463,
+      "grad_norm": 0.14720793068408966,
+      "learning_rate": 4.817959636416969e-05,
+      "loss": 0.2816,
+      "num_input_tokens_seen": 1722360,
+      "step": 66
+    },
+    {
+      "epoch": 2.4055299539170507,
+      "grad_norm": 0.10553745925426483,
+      "learning_rate": 4.81247200794554e-05,
+      "loss": 0.2502,
+      "num_input_tokens_seen": 1752464,
+      "step": 67
+    },
+    {
+      "epoch": 2.442396313364055,
+      "grad_norm": 0.11162563413381577,
+      "learning_rate": 4.806906110888606e-05,
+      "loss": 0.2286,
+      "num_input_tokens_seen": 1782744,
+      "step": 68
+    },
+    {
+      "epoch": 2.47926267281106,
+      "grad_norm": 0.12341686338186264,
+      "learning_rate": 4.8012621336311016e-05,
+      "loss": 0.299,
+      "num_input_tokens_seen": 1815120,
+      "step": 69
+    },
+    {
+      "epoch": 2.5161290322580645,
+      "grad_norm": 0.12033814936876297,
+      "learning_rate": 4.7955402672006854e-05,
+      "loss": 0.2609,
+      "num_input_tokens_seen": 1840408,
+      "step": 70
+    },
+    {
+      "epoch": 2.5529953917050694,
+      "grad_norm": 0.11412363499403,
+      "learning_rate": 4.789740705261278e-05,
+      "loss": 0.3316,
+      "num_input_tokens_seen": 1873000,
+      "step": 71
+    },
+    {
+      "epoch": 2.589861751152074,
+      "grad_norm": 0.18193566799163818,
+      "learning_rate": 4.783863644106502e-05,
+      "loss": 0.2773,
+      "num_input_tokens_seen": 1900920,
+      "step": 72
+    },
+    {
+      "epoch": 2.6267281105990783,
+      "grad_norm": 0.14569184184074402,
+      "learning_rate": 4.777909282653042e-05,
+      "loss": 0.2644,
+      "num_input_tokens_seen": 1919272,
+      "step": 73
+    },
+    {
+      "epoch": 2.6635944700460827,
+      "grad_norm": 0.12456360459327698,
+      "learning_rate": 4.771877822433911e-05,
+      "loss": 0.2974,
+      "num_input_tokens_seen": 1959152,
+      "step": 74
+    },
+    {
+      "epoch": 2.7004608294930876,
+      "grad_norm": 0.1362839788198471,
+      "learning_rate": 4.765769467591625e-05,
+      "loss": 0.2743,
+      "num_input_tokens_seen": 1982960,
+      "step": 75
+    },
+    {
+      "epoch": 2.737327188940092,
+      "grad_norm": 0.15120629966259003,
+      "learning_rate": 4.759584424871302e-05,
+      "loss": 0.2425,
+      "num_input_tokens_seen": 2006736,
+      "step": 76
+    },
+    {
+      "epoch": 2.774193548387097,
+      "grad_norm": 0.15145516395568848,
+      "learning_rate": 4.7533229036136553e-05,
+      "loss": 0.2723,
+      "num_input_tokens_seen": 2031536,
+      "step": 77
+    },
+    {
+      "epoch": 2.8110599078341014,
+      "grad_norm": 0.1607382595539093,
+      "learning_rate": 4.7469851157479177e-05,
+      "loss": 0.2525,
+      "num_input_tokens_seen": 2053168,
+      "step": 78
+    },
+    {
+      "epoch": 2.847926267281106,
+      "grad_norm": 0.14976347982883453,
+      "learning_rate": 4.740571275784659e-05,
+      "loss": 0.2792,
+      "num_input_tokens_seen": 2078208,
+      "step": 79
+    },
+    {
+      "epoch": 2.8847926267281108,
+      "grad_norm": 0.12997782230377197,
+      "learning_rate": 4.734081600808531e-05,
+      "loss": 0.291,
+      "num_input_tokens_seen": 2101560,
+      "step": 80
+    },
+    {
+      "epoch": 2.921658986175115,
+      "grad_norm": 0.1369074434041977,
+      "learning_rate": 4.72751631047092e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2127824,
+      "step": 81
+    },
+    {
+      "epoch": 2.9585253456221197,
+      "grad_norm": 0.11734048277139664,
+      "learning_rate": 4.7208756269825104e-05,
+      "loss": 0.2442,
+      "num_input_tokens_seen": 2171536,
+      "step": 82
+    },
+    {
+      "epoch": 2.9953917050691246,
+      "grad_norm": 0.14909496903419495,
+      "learning_rate": 4.714159775105765e-05,
+      "loss": 0.2344,
+      "num_input_tokens_seen": 2193016,
+      "step": 83
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.40910854935646057,
+      "learning_rate": 4.707368982147318e-05,
+      "loss": 0.2645,
+      "num_input_tokens_seen": 2195928,
+      "step": 84
+    },
+    {
+      "epoch": 3.0368663594470044,
+      "grad_norm": 0.1433151513338089,
+      "learning_rate": 4.700503477950278e-05,
+      "loss": 0.2185,
+      "num_input_tokens_seen": 2219472,
+      "step": 85
+    },
+    {
+      "epoch": 3.0737327188940093,
+      "grad_norm": 0.13229826092720032,
+      "learning_rate": 4.693563494886455e-05,
+      "loss": 0.2364,
+      "num_input_tokens_seen": 2246504,
+      "step": 86
+    },
+    {
+      "epoch": 3.110599078341014,
+      "grad_norm": 0.1702108383178711,
+      "learning_rate": 4.6865492678484895e-05,
+      "loss": 0.2703,
+      "num_input_tokens_seen": 2270304,
+      "step": 87
+    },
+    {
+      "epoch": 3.1474654377880182,
+      "grad_norm": 0.1411813497543335,
+      "learning_rate": 4.679461034241906e-05,
+      "loss": 0.2807,
+      "num_input_tokens_seen": 2299968,
+      "step": 88
+    },
+    {
+      "epoch": 3.184331797235023,
+      "grad_norm": 0.12941044569015503,
+      "learning_rate": 4.672299033977076e-05,
+      "loss": 0.2182,
+      "num_input_tokens_seen": 2333432,
+      "step": 89
+    },
+    {
+      "epoch": 3.2211981566820276,
+      "grad_norm": 0.1649884432554245,
+      "learning_rate": 4.665063509461097e-05,
+      "loss": 0.2572,
+      "num_input_tokens_seen": 2355320,
+      "step": 90
+    },
+    {
+      "epoch": 3.258064516129032,
+      "grad_norm": 0.14386451244354248,
+      "learning_rate": 4.657754705589591e-05,
+      "loss": 0.2412,
+      "num_input_tokens_seen": 2378416,
+      "step": 91
+    },
+    {
+      "epoch": 3.294930875576037,
+      "grad_norm": 0.15350477397441864,
+      "learning_rate": 4.650372869738414e-05,
+      "loss": 0.242,
+      "num_input_tokens_seen": 2405432,
+      "step": 92
+    },
+    {
+      "epoch": 3.3317972350230414,
+      "grad_norm": 0.16243231296539307,
+      "learning_rate": 4.642918251755281e-05,
+      "loss": 0.2436,
+      "num_input_tokens_seen": 2429992,
+      "step": 93
+    },
+    {
+      "epoch": 3.3686635944700463,
+      "grad_norm": 0.18401454389095306,
+      "learning_rate": 4.6353911039513145e-05,
+      "loss": 0.2538,
+      "num_input_tokens_seen": 2456264,
+      "step": 94
+    },
+    {
+      "epoch": 3.4055299539170507,
+      "grad_norm": 0.13898858428001404,
+      "learning_rate": 4.627791681092499e-05,
+      "loss": 0.2065,
+      "num_input_tokens_seen": 2486032,
+      "step": 95
+    },
+    {
+      "epoch": 3.442396313364055,
+      "grad_norm": 0.16943225264549255,
+      "learning_rate": 4.620120240391065e-05,
+      "loss": 0.2086,
+      "num_input_tokens_seen": 2509272,
+      "step": 96
+    },
+    {
+      "epoch": 3.47926267281106,
+      "grad_norm": 0.15286582708358765,
+      "learning_rate": 4.612377041496776e-05,
+      "loss": 0.2517,
+      "num_input_tokens_seen": 2538392,
+      "step": 97
+    },
+    {
+      "epoch": 3.5161290322580645,
+      "grad_norm": 0.17737072706222534,
+      "learning_rate": 4.604562346488144e-05,
+      "loss": 0.2413,
+      "num_input_tokens_seen": 2562624,
+      "step": 98
+    },
+    {
+      "epoch": 3.5529953917050694,
+      "grad_norm": 0.13630996644496918,
+      "learning_rate": 4.5966764198635606e-05,
+      "loss": 0.2157,
+      "num_input_tokens_seen": 2595448,
+      "step": 99
+    },
+    {
+      "epoch": 3.589861751152074,
+      "grad_norm": 0.13918158411979675,
+      "learning_rate": 4.588719528532342e-05,
+      "loss": 0.2319,
+      "num_input_tokens_seen": 2633568,
+      "step": 100
+    },
+    {
+      "epoch": 3.6267281105990783,
+      "grad_norm": 0.16548505425453186,
+      "learning_rate": 4.580691941805695e-05,
+      "loss": 0.2473,
+      "num_input_tokens_seen": 2659408,
+      "step": 101
+    },
+    {
+      "epoch": 3.6635944700460827,
+      "grad_norm": 0.17397943139076233,
+      "learning_rate": 4.572593931387604e-05,
+      "loss": 0.2527,
+      "num_input_tokens_seen": 2687272,
+      "step": 102
+    },
+    {
+      "epoch": 3.7004608294930876,
+      "grad_norm": 0.15970823168754578,
+      "learning_rate": 4.5644257713656356e-05,
+      "loss": 0.22,
+      "num_input_tokens_seen": 2718232,
+      "step": 103
+    },
+    {
+      "epoch": 3.737327188940092,
+      "grad_norm": 0.16440241038799286,
+      "learning_rate": 4.556187738201656e-05,
+      "loss": 0.2374,
+      "num_input_tokens_seen": 2743336,
+      "step": 104
+    },
+    {
+      "epoch": 3.774193548387097,
+      "grad_norm": 0.15761925280094147,
+      "learning_rate": 4.54788011072248e-05,
+      "loss": 0.2023,
+      "num_input_tokens_seen": 2771384,
+      "step": 105
+    },
+    {
+      "epoch": 3.8110599078341014,
+      "grad_norm": 0.1528206616640091,
+      "learning_rate": 4.539503170110431e-05,
+      "loss": 0.2289,
+      "num_input_tokens_seen": 2797488,
+      "step": 106
+    },
+    {
+      "epoch": 3.847926267281106,
+      "grad_norm": 0.16380636394023895,
+      "learning_rate": 4.531057199893824e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2825408,
+      "step": 107
+    },
+    {
+      "epoch": 3.8847926267281108,
+      "grad_norm": 0.19076082110404968,
+      "learning_rate": 4.522542485937369e-05,
+      "loss": 0.2467,
+      "num_input_tokens_seen": 2848856,
+      "step": 108
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 540,
+  "num_input_tokens_seen": 2848856,
+  "num_train_epochs": 20,
+  "save_steps": 27,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.4033425764017766e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-135/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-135/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-135/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-135/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-135/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-135/trainer_state.json

@@ -0,0 +1,1113 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 4.847926267281106,
+  "eval_steps": 500,
+  "global_step": 135,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03686635944700461,
+      "grad_norm": 0.06150835379958153,
+      "learning_rate": 4.999957692144361e-05,
+      "loss": 0.663,
+      "num_input_tokens_seen": 23408,
+      "step": 1
+    },
+    {
+      "epoch": 0.07373271889400922,
+      "grad_norm": 0.05766688659787178,
+      "learning_rate": 4.999830770009406e-05,
+      "loss": 0.5847,
+      "num_input_tokens_seen": 45520,
+      "step": 2
+    },
+    {
+      "epoch": 0.11059907834101383,
+      "grad_norm": 0.0780782401561737,
+      "learning_rate": 4.9996192378909786e-05,
+      "loss": 0.5545,
+      "num_input_tokens_seen": 68904,
+      "step": 3
+    },
+    {
+      "epoch": 0.14746543778801843,
+      "grad_norm": 0.07575194537639618,
+      "learning_rate": 4.9993231029486544e-05,
+      "loss": 0.7025,
+      "num_input_tokens_seen": 89696,
+      "step": 4
+    },
+    {
+      "epoch": 0.18433179723502305,
+      "grad_norm": 0.0708586797118187,
+      "learning_rate": 4.998942375205502e-05,
+      "loss": 0.6063,
+      "num_input_tokens_seen": 118440,
+      "step": 5
+    },
+    {
+      "epoch": 0.22119815668202766,
+      "grad_norm": 0.0794093981385231,
+      "learning_rate": 4.99847706754774e-05,
+      "loss": 0.593,
+      "num_input_tokens_seen": 145072,
+      "step": 6
+    },
+    {
+      "epoch": 0.25806451612903225,
+      "grad_norm": 0.04954744130373001,
+      "learning_rate": 4.997927195724303e-05,
+      "loss": 0.4857,
+      "num_input_tokens_seen": 182112,
+      "step": 7
+    },
+    {
+      "epoch": 0.29493087557603687,
+      "grad_norm": 0.06549246609210968,
+      "learning_rate": 4.997292778346312e-05,
+      "loss": 0.5393,
+      "num_input_tokens_seen": 212192,
+      "step": 8
+    },
+    {
+      "epoch": 0.3317972350230415,
+      "grad_norm": 0.07614441215991974,
+      "learning_rate": 4.996573836886435e-05,
+      "loss": 0.7044,
+      "num_input_tokens_seen": 233728,
+      "step": 9
+    },
+    {
+      "epoch": 0.3686635944700461,
+      "grad_norm": 0.07068340480327606,
+      "learning_rate": 4.995770395678171e-05,
+      "loss": 0.6086,
+      "num_input_tokens_seen": 259776,
+      "step": 10
+    },
+    {
+      "epoch": 0.4055299539170507,
+      "grad_norm": 0.07266585528850555,
+      "learning_rate": 4.9948824819150185e-05,
+      "loss": 0.6247,
+      "num_input_tokens_seen": 284816,
+      "step": 11
+    },
+    {
+      "epoch": 0.4423963133640553,
+      "grad_norm": 0.06145177036523819,
+      "learning_rate": 4.993910125649561e-05,
+      "loss": 0.4918,
+      "num_input_tokens_seen": 309776,
+      "step": 12
+    },
+    {
+      "epoch": 0.4792626728110599,
+      "grad_norm": 0.06720694154500961,
+      "learning_rate": 4.992853359792444e-05,
+      "loss": 0.5017,
+      "num_input_tokens_seen": 339064,
+      "step": 13
+    },
+    {
+      "epoch": 0.5161290322580645,
+      "grad_norm": 0.07309022545814514,
+      "learning_rate": 4.9917122201112656e-05,
+      "loss": 0.5447,
+      "num_input_tokens_seen": 369864,
+      "step": 14
+    },
+    {
+      "epoch": 0.5529953917050692,
+      "grad_norm": 0.0778132826089859,
+      "learning_rate": 4.990486745229364e-05,
+      "loss": 0.5034,
+      "num_input_tokens_seen": 397184,
+      "step": 15
+    },
+    {
+      "epoch": 0.5898617511520737,
+      "grad_norm": 0.07727856189012527,
+      "learning_rate": 4.989176976624511e-05,
+      "loss": 0.5277,
+      "num_input_tokens_seen": 426296,
+      "step": 16
+    },
+    {
+      "epoch": 0.6267281105990783,
+      "grad_norm": 0.0700518786907196,
+      "learning_rate": 4.987782958627508e-05,
+      "loss": 0.5356,
+      "num_input_tokens_seen": 459544,
+      "step": 17
+    },
+    {
+      "epoch": 0.663594470046083,
+      "grad_norm": 0.07351912558078766,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.5505,
+      "num_input_tokens_seen": 488616,
+      "step": 18
+    },
+    {
+      "epoch": 0.7004608294930875,
+      "grad_norm": 0.08268705755472183,
+      "learning_rate": 4.9847423660363e-05,
+      "loss": 0.5557,
+      "num_input_tokens_seen": 512176,
+      "step": 19
+    },
+    {
+      "epoch": 0.7373271889400922,
+      "grad_norm": 0.08081424236297607,
+      "learning_rate": 4.983095894354858e-05,
+      "loss": 0.4576,
+      "num_input_tokens_seen": 539456,
+      "step": 20
+    },
+    {
+      "epoch": 0.7741935483870968,
+      "grad_norm": 0.07748426496982574,
+      "learning_rate": 4.9813653791033057e-05,
+      "loss": 0.3918,
+      "num_input_tokens_seen": 561384,
+      "step": 21
+    },
+    {
+      "epoch": 0.8110599078341014,
+      "grad_norm": 0.10133849829435349,
+      "learning_rate": 4.979550878853154e-05,
+      "loss": 0.531,
+      "num_input_tokens_seen": 589792,
+      "step": 22
+    },
+    {
+      "epoch": 0.847926267281106,
+      "grad_norm": 0.09528470784425735,
+      "learning_rate": 4.9776524550184965e-05,
+      "loss": 0.4159,
+      "num_input_tokens_seen": 613944,
+      "step": 23
+    },
+    {
+      "epoch": 0.8847926267281107,
+      "grad_norm": 0.08829868584871292,
+      "learning_rate": 4.975670171853926e-05,
+      "loss": 0.403,
+      "num_input_tokens_seen": 641432,
+      "step": 24
+    },
+    {
+      "epoch": 0.9216589861751152,
+      "grad_norm": 0.10020875930786133,
+      "learning_rate": 4.973604096452361e-05,
+      "loss": 0.4238,
+      "num_input_tokens_seen": 665232,
+      "step": 25
+    },
+    {
+      "epoch": 0.9585253456221198,
+      "grad_norm": 0.1256554126739502,
+      "learning_rate": 4.971454298742779e-05,
+      "loss": 0.5033,
+      "num_input_tokens_seen": 687216,
+      "step": 26
+    },
+    {
+      "epoch": 0.9953917050691244,
+      "grad_norm": 0.17123407125473022,
+      "learning_rate": 4.9692208514878444e-05,
+      "loss": 0.3842,
+      "num_input_tokens_seen": 728880,
+      "step": 27
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.27383726835250854,
+      "learning_rate": 4.966903830281449e-05,
+      "loss": 0.4988,
+      "num_input_tokens_seen": 731976,
+      "step": 28
+    },
+    {
+      "epoch": 1.0368663594470047,
+      "grad_norm": 0.148647278547287,
+      "learning_rate": 4.9645033135461494e-05,
+      "loss": 0.4489,
+      "num_input_tokens_seen": 756184,
+      "step": 29
+    },
+    {
+      "epoch": 1.0737327188940091,
+      "grad_norm": 0.11988866329193115,
+      "learning_rate": 4.962019382530521e-05,
+      "loss": 0.4999,
+      "num_input_tokens_seen": 779072,
+      "step": 30
+    },
+    {
+      "epoch": 1.1105990783410138,
+      "grad_norm": 0.10590679943561554,
+      "learning_rate": 4.9594521213063974e-05,
+      "loss": 0.3711,
+      "num_input_tokens_seen": 805672,
+      "step": 31
+    },
+    {
+      "epoch": 1.1474654377880185,
+      "grad_norm": 0.11934591829776764,
+      "learning_rate": 4.9568016167660334e-05,
+      "loss": 0.3385,
+      "num_input_tokens_seen": 828840,
+      "step": 32
+    },
+    {
+      "epoch": 1.1843317972350231,
+      "grad_norm": 0.11900116503238678,
+      "learning_rate": 4.9540679586191605e-05,
+      "loss": 0.3812,
+      "num_input_tokens_seen": 850792,
+      "step": 33
+    },
+    {
+      "epoch": 1.2211981566820276,
+      "grad_norm": 0.11279546469449997,
+      "learning_rate": 4.951251239389948e-05,
+      "loss": 0.2924,
+      "num_input_tokens_seen": 885608,
+      "step": 34
+    },
+    {
+      "epoch": 1.2580645161290323,
+      "grad_norm": 0.08340126276016235,
+      "learning_rate": 4.948351554413879e-05,
+      "loss": 0.3489,
+      "num_input_tokens_seen": 921912,
+      "step": 35
+    },
+    {
+      "epoch": 1.294930875576037,
+      "grad_norm": 0.11327336728572845,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.2921,
+      "num_input_tokens_seen": 945672,
+      "step": 36
+    },
+    {
+      "epoch": 1.3317972350230414,
+      "grad_norm": 0.10909265279769897,
+      "learning_rate": 4.942303682600178e-05,
+      "loss": 0.3421,
+      "num_input_tokens_seen": 970824,
+      "step": 37
+    },
+    {
+      "epoch": 1.368663594470046,
+      "grad_norm": 0.11239592730998993,
+      "learning_rate": 4.939155700460536e-05,
+      "loss": 0.3848,
+      "num_input_tokens_seen": 995840,
+      "step": 38
+    },
+    {
+      "epoch": 1.4055299539170507,
+      "grad_norm": 0.1034158244729042,
+      "learning_rate": 4.9359251619630886e-05,
+      "loss": 0.3434,
+      "num_input_tokens_seen": 1035880,
+      "step": 39
+    },
+    {
+      "epoch": 1.4423963133640554,
+      "grad_norm": 0.12086673080921173,
+      "learning_rate": 4.9326121764495596e-05,
+      "loss": 0.3588,
+      "num_input_tokens_seen": 1056488,
+      "step": 40
+    },
+    {
+      "epoch": 1.4792626728110598,
+      "grad_norm": 0.11408794671297073,
+      "learning_rate": 4.9292168560522014e-05,
+      "loss": 0.4021,
+      "num_input_tokens_seen": 1080080,
+      "step": 41
+    },
+    {
+      "epoch": 1.5161290322580645,
+      "grad_norm": 0.1417722851037979,
+      "learning_rate": 4.925739315689991e-05,
+      "loss": 0.3343,
+      "num_input_tokens_seen": 1095632,
+      "step": 42
+    },
+    {
+      "epoch": 1.5529953917050692,
+      "grad_norm": 0.2315663844347,
+      "learning_rate": 4.9221796730647516e-05,
+      "loss": 0.4025,
+      "num_input_tokens_seen": 1126040,
+      "step": 43
+    },
+    {
+      "epoch": 1.5898617511520738,
+      "grad_norm": 0.07856712490320206,
+      "learning_rate": 4.9185380486571595e-05,
+      "loss": 0.3337,
+      "num_input_tokens_seen": 1156656,
+      "step": 44
+    },
+    {
+      "epoch": 1.6267281105990783,
+      "grad_norm": 0.10064279288053513,
+      "learning_rate": 4.914814565722671e-05,
+      "loss": 0.3758,
+      "num_input_tokens_seen": 1183400,
+      "step": 45
+    },
+    {
+      "epoch": 1.663594470046083,
+      "grad_norm": 0.106510229408741,
+      "learning_rate": 4.9110093502873476e-05,
+      "loss": 0.3363,
+      "num_input_tokens_seen": 1209680,
+      "step": 46
+    },
+    {
+      "epoch": 1.7004608294930876,
+      "grad_norm": 0.10446175187826157,
+      "learning_rate": 4.907122531143594e-05,
+      "loss": 0.3853,
+      "num_input_tokens_seen": 1237864,
+      "step": 47
+    },
+    {
+      "epoch": 1.737327188940092,
+      "grad_norm": 0.2570022940635681,
+      "learning_rate": 4.9031542398457974e-05,
+      "loss": 0.3194,
+      "num_input_tokens_seen": 1273080,
+      "step": 48
+    },
+    {
+      "epoch": 1.7741935483870968,
+      "grad_norm": 0.0968039259314537,
+      "learning_rate": 4.8991046107058735e-05,
+      "loss": 0.2955,
+      "num_input_tokens_seen": 1300256,
+      "step": 49
+    },
+    {
+      "epoch": 1.8110599078341014,
+      "grad_norm": 0.11750692129135132,
+      "learning_rate": 4.894973780788722e-05,
+      "loss": 0.3314,
+      "num_input_tokens_seen": 1322616,
+      "step": 50
+    },
+    {
+      "epoch": 1.8479262672811059,
+      "grad_norm": 0.09568070620298386,
+      "learning_rate": 4.890761889907589e-05,
+      "loss": 0.3341,
+      "num_input_tokens_seen": 1353552,
+      "step": 51
+    },
+    {
+      "epoch": 1.8847926267281108,
+      "grad_norm": 0.11816362291574478,
+      "learning_rate": 4.88646908061933e-05,
+      "loss": 0.3202,
+      "num_input_tokens_seen": 1376456,
+      "step": 52
+    },
+    {
+      "epoch": 1.9216589861751152,
+      "grad_norm": 0.133891761302948,
+      "learning_rate": 4.8820954982195905e-05,
+      "loss": 0.2852,
+      "num_input_tokens_seen": 1398344,
+      "step": 53
+    },
+    {
+      "epoch": 1.9585253456221197,
+      "grad_norm": 0.11985364556312561,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.3413,
+      "num_input_tokens_seen": 1424264,
+      "step": 54
+    },
+    {
+      "epoch": 1.9953917050691246,
+      "grad_norm": 0.09828098118305206,
+      "learning_rate": 4.873106608932585e-05,
+      "loss": 0.2726,
+      "num_input_tokens_seen": 1460688,
+      "step": 55
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.31861889362335205,
+      "learning_rate": 4.868491606285823e-05,
+      "loss": 0.3512,
+      "num_input_tokens_seen": 1463952,
+      "step": 56
+    },
+    {
+      "epoch": 2.0368663594470044,
+      "grad_norm": 0.11089354753494263,
+      "learning_rate": 4.8637964389982926e-05,
+      "loss": 0.2953,
+      "num_input_tokens_seen": 1492600,
+      "step": 57
+    },
+    {
+      "epoch": 2.0737327188940093,
+      "grad_norm": 0.12034712731838226,
+      "learning_rate": 4.859021265983959e-05,
+      "loss": 0.2875,
+      "num_input_tokens_seen": 1514496,
+      "step": 58
+    },
+    {
+      "epoch": 2.110599078341014,
+      "grad_norm": 0.1372830867767334,
+      "learning_rate": 4.854166248864689e-05,
+      "loss": 0.3535,
+      "num_input_tokens_seen": 1541656,
+      "step": 59
+    },
+    {
+      "epoch": 2.1474654377880182,
+      "grad_norm": 0.10605650395154953,
+      "learning_rate": 4.849231551964771e-05,
+      "loss": 0.3013,
+      "num_input_tokens_seen": 1569672,
+      "step": 60
+    },
+    {
+      "epoch": 2.184331797235023,
+      "grad_norm": 0.12524856626987457,
+      "learning_rate": 4.844217342305363e-05,
+      "loss": 0.2558,
+      "num_input_tokens_seen": 1599304,
+      "step": 61
+    },
+    {
+      "epoch": 2.2211981566820276,
+      "grad_norm": 0.1151009052991867,
+      "learning_rate": 4.839123789598829e-05,
+      "loss": 0.2666,
+      "num_input_tokens_seen": 1628288,
+      "step": 62
+    },
+    {
+      "epoch": 2.258064516129032,
+      "grad_norm": 0.10297228395938873,
+      "learning_rate": 4.8339510662430046e-05,
+      "loss": 0.2982,
+      "num_input_tokens_seen": 1656384,
+      "step": 63
+    },
+    {
+      "epoch": 2.294930875576037,
+      "grad_norm": 0.11971966177225113,
+      "learning_rate": 4.828699347315356e-05,
+      "loss": 0.3746,
+      "num_input_tokens_seen": 1679344,
+      "step": 64
+    },
+    {
+      "epoch": 2.3317972350230414,
+      "grad_norm": 0.12384694814682007,
+      "learning_rate": 4.823368810567056e-05,
+      "loss": 0.2605,
+      "num_input_tokens_seen": 1703904,
+      "step": 65
+    },
+    {
+      "epoch": 2.3686635944700463,
+      "grad_norm": 0.14720793068408966,
+      "learning_rate": 4.817959636416969e-05,
+      "loss": 0.2816,
+      "num_input_tokens_seen": 1722360,
+      "step": 66
+    },
+    {
+      "epoch": 2.4055299539170507,
+      "grad_norm": 0.10553745925426483,
+      "learning_rate": 4.81247200794554e-05,
+      "loss": 0.2502,
+      "num_input_tokens_seen": 1752464,
+      "step": 67
+    },
+    {
+      "epoch": 2.442396313364055,
+      "grad_norm": 0.11162563413381577,
+      "learning_rate": 4.806906110888606e-05,
+      "loss": 0.2286,
+      "num_input_tokens_seen": 1782744,
+      "step": 68
+    },
+    {
+      "epoch": 2.47926267281106,
+      "grad_norm": 0.12341686338186264,
+      "learning_rate": 4.8012621336311016e-05,
+      "loss": 0.299,
+      "num_input_tokens_seen": 1815120,
+      "step": 69
+    },
+    {
+      "epoch": 2.5161290322580645,
+      "grad_norm": 0.12033814936876297,
+      "learning_rate": 4.7955402672006854e-05,
+      "loss": 0.2609,
+      "num_input_tokens_seen": 1840408,
+      "step": 70
+    },
+    {
+      "epoch": 2.5529953917050694,
+      "grad_norm": 0.11412363499403,
+      "learning_rate": 4.789740705261278e-05,
+      "loss": 0.3316,
+      "num_input_tokens_seen": 1873000,
+      "step": 71
+    },
+    {
+      "epoch": 2.589861751152074,
+      "grad_norm": 0.18193566799163818,
+      "learning_rate": 4.783863644106502e-05,
+      "loss": 0.2773,
+      "num_input_tokens_seen": 1900920,
+      "step": 72
+    },
+    {
+      "epoch": 2.6267281105990783,
+      "grad_norm": 0.14569184184074402,
+      "learning_rate": 4.777909282653042e-05,
+      "loss": 0.2644,
+      "num_input_tokens_seen": 1919272,
+      "step": 73
+    },
+    {
+      "epoch": 2.6635944700460827,
+      "grad_norm": 0.12456360459327698,
+      "learning_rate": 4.771877822433911e-05,
+      "loss": 0.2974,
+      "num_input_tokens_seen": 1959152,
+      "step": 74
+    },
+    {
+      "epoch": 2.7004608294930876,
+      "grad_norm": 0.1362839788198471,
+      "learning_rate": 4.765769467591625e-05,
+      "loss": 0.2743,
+      "num_input_tokens_seen": 1982960,
+      "step": 75
+    },
+    {
+      "epoch": 2.737327188940092,
+      "grad_norm": 0.15120629966259003,
+      "learning_rate": 4.759584424871302e-05,
+      "loss": 0.2425,
+      "num_input_tokens_seen": 2006736,
+      "step": 76
+    },
+    {
+      "epoch": 2.774193548387097,
+      "grad_norm": 0.15145516395568848,
+      "learning_rate": 4.7533229036136553e-05,
+      "loss": 0.2723,
+      "num_input_tokens_seen": 2031536,
+      "step": 77
+    },
+    {
+      "epoch": 2.8110599078341014,
+      "grad_norm": 0.1607382595539093,
+      "learning_rate": 4.7469851157479177e-05,
+      "loss": 0.2525,
+      "num_input_tokens_seen": 2053168,
+      "step": 78
+    },
+    {
+      "epoch": 2.847926267281106,
+      "grad_norm": 0.14976347982883453,
+      "learning_rate": 4.740571275784659e-05,
+      "loss": 0.2792,
+      "num_input_tokens_seen": 2078208,
+      "step": 79
+    },
+    {
+      "epoch": 2.8847926267281108,
+      "grad_norm": 0.12997782230377197,
+      "learning_rate": 4.734081600808531e-05,
+      "loss": 0.291,
+      "num_input_tokens_seen": 2101560,
+      "step": 80
+    },
+    {
+      "epoch": 2.921658986175115,
+      "grad_norm": 0.1369074434041977,
+      "learning_rate": 4.72751631047092e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2127824,
+      "step": 81
+    },
+    {
+      "epoch": 2.9585253456221197,
+      "grad_norm": 0.11734048277139664,
+      "learning_rate": 4.7208756269825104e-05,
+      "loss": 0.2442,
+      "num_input_tokens_seen": 2171536,
+      "step": 82
+    },
+    {
+      "epoch": 2.9953917050691246,
+      "grad_norm": 0.14909496903419495,
+      "learning_rate": 4.714159775105765e-05,
+      "loss": 0.2344,
+      "num_input_tokens_seen": 2193016,
+      "step": 83
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.40910854935646057,
+      "learning_rate": 4.707368982147318e-05,
+      "loss": 0.2645,
+      "num_input_tokens_seen": 2195928,
+      "step": 84
+    },
+    {
+      "epoch": 3.0368663594470044,
+      "grad_norm": 0.1433151513338089,
+      "learning_rate": 4.700503477950278e-05,
+      "loss": 0.2185,
+      "num_input_tokens_seen": 2219472,
+      "step": 85
+    },
+    {
+      "epoch": 3.0737327188940093,
+      "grad_norm": 0.13229826092720032,
+      "learning_rate": 4.693563494886455e-05,
+      "loss": 0.2364,
+      "num_input_tokens_seen": 2246504,
+      "step": 86
+    },
+    {
+      "epoch": 3.110599078341014,
+      "grad_norm": 0.1702108383178711,
+      "learning_rate": 4.6865492678484895e-05,
+      "loss": 0.2703,
+      "num_input_tokens_seen": 2270304,
+      "step": 87
+    },
+    {
+      "epoch": 3.1474654377880182,
+      "grad_norm": 0.1411813497543335,
+      "learning_rate": 4.679461034241906e-05,
+      "loss": 0.2807,
+      "num_input_tokens_seen": 2299968,
+      "step": 88
+    },
+    {
+      "epoch": 3.184331797235023,
+      "grad_norm": 0.12941044569015503,
+      "learning_rate": 4.672299033977076e-05,
+      "loss": 0.2182,
+      "num_input_tokens_seen": 2333432,
+      "step": 89
+    },
+    {
+      "epoch": 3.2211981566820276,
+      "grad_norm": 0.1649884432554245,
+      "learning_rate": 4.665063509461097e-05,
+      "loss": 0.2572,
+      "num_input_tokens_seen": 2355320,
+      "step": 90
+    },
+    {
+      "epoch": 3.258064516129032,
+      "grad_norm": 0.14386451244354248,
+      "learning_rate": 4.657754705589591e-05,
+      "loss": 0.2412,
+      "num_input_tokens_seen": 2378416,
+      "step": 91
+    },
+    {
+      "epoch": 3.294930875576037,
+      "grad_norm": 0.15350477397441864,
+      "learning_rate": 4.650372869738414e-05,
+      "loss": 0.242,
+      "num_input_tokens_seen": 2405432,
+      "step": 92
+    },
+    {
+      "epoch": 3.3317972350230414,
+      "grad_norm": 0.16243231296539307,
+      "learning_rate": 4.642918251755281e-05,
+      "loss": 0.2436,
+      "num_input_tokens_seen": 2429992,
+      "step": 93
+    },
+    {
+      "epoch": 3.3686635944700463,
+      "grad_norm": 0.18401454389095306,
+      "learning_rate": 4.6353911039513145e-05,
+      "loss": 0.2538,
+      "num_input_tokens_seen": 2456264,
+      "step": 94
+    },
+    {
+      "epoch": 3.4055299539170507,
+      "grad_norm": 0.13898858428001404,
+      "learning_rate": 4.627791681092499e-05,
+      "loss": 0.2065,
+      "num_input_tokens_seen": 2486032,
+      "step": 95
+    },
+    {
+      "epoch": 3.442396313364055,
+      "grad_norm": 0.16943225264549255,
+      "learning_rate": 4.620120240391065e-05,
+      "loss": 0.2086,
+      "num_input_tokens_seen": 2509272,
+      "step": 96
+    },
+    {
+      "epoch": 3.47926267281106,
+      "grad_norm": 0.15286582708358765,
+      "learning_rate": 4.612377041496776e-05,
+      "loss": 0.2517,
+      "num_input_tokens_seen": 2538392,
+      "step": 97
+    },
+    {
+      "epoch": 3.5161290322580645,
+      "grad_norm": 0.17737072706222534,
+      "learning_rate": 4.604562346488144e-05,
+      "loss": 0.2413,
+      "num_input_tokens_seen": 2562624,
+      "step": 98
+    },
+    {
+      "epoch": 3.5529953917050694,
+      "grad_norm": 0.13630996644496918,
+      "learning_rate": 4.5966764198635606e-05,
+      "loss": 0.2157,
+      "num_input_tokens_seen": 2595448,
+      "step": 99
+    },
+    {
+      "epoch": 3.589861751152074,
+      "grad_norm": 0.13918158411979675,
+      "learning_rate": 4.588719528532342e-05,
+      "loss": 0.2319,
+      "num_input_tokens_seen": 2633568,
+      "step": 100
+    },
+    {
+      "epoch": 3.6267281105990783,
+      "grad_norm": 0.16548505425453186,
+      "learning_rate": 4.580691941805695e-05,
+      "loss": 0.2473,
+      "num_input_tokens_seen": 2659408,
+      "step": 101
+    },
+    {
+      "epoch": 3.6635944700460827,
+      "grad_norm": 0.17397943139076233,
+      "learning_rate": 4.572593931387604e-05,
+      "loss": 0.2527,
+      "num_input_tokens_seen": 2687272,
+      "step": 102
+    },
+    {
+      "epoch": 3.7004608294930876,
+      "grad_norm": 0.15970823168754578,
+      "learning_rate": 4.5644257713656356e-05,
+      "loss": 0.22,
+      "num_input_tokens_seen": 2718232,
+      "step": 103
+    },
+    {
+      "epoch": 3.737327188940092,
+      "grad_norm": 0.16440241038799286,
+      "learning_rate": 4.556187738201656e-05,
+      "loss": 0.2374,
+      "num_input_tokens_seen": 2743336,
+      "step": 104
+    },
+    {
+      "epoch": 3.774193548387097,
+      "grad_norm": 0.15761925280094147,
+      "learning_rate": 4.54788011072248e-05,
+      "loss": 0.2023,
+      "num_input_tokens_seen": 2771384,
+      "step": 105
+    },
+    {
+      "epoch": 3.8110599078341014,
+      "grad_norm": 0.1528206616640091,
+      "learning_rate": 4.539503170110431e-05,
+      "loss": 0.2289,
+      "num_input_tokens_seen": 2797488,
+      "step": 106
+    },
+    {
+      "epoch": 3.847926267281106,
+      "grad_norm": 0.16380636394023895,
+      "learning_rate": 4.531057199893824e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2825408,
+      "step": 107
+    },
+    {
+      "epoch": 3.8847926267281108,
+      "grad_norm": 0.19076082110404968,
+      "learning_rate": 4.522542485937369e-05,
+      "loss": 0.2467,
+      "num_input_tokens_seen": 2848856,
+      "step": 108
+    },
+    {
+      "epoch": 3.921658986175115,
+      "grad_norm": 0.1884787231683731,
+      "learning_rate": 4.5139593164324986e-05,
+      "loss": 0.2069,
+      "num_input_tokens_seen": 2867536,
+      "step": 109
+    },
+    {
+      "epoch": 3.9585253456221197,
+      "grad_norm": 0.19939164817333221,
+      "learning_rate": 4.50530798188761e-05,
+      "loss": 0.1843,
+      "num_input_tokens_seen": 2892032,
+      "step": 110
+    },
+    {
+      "epoch": 3.9953917050691246,
+      "grad_norm": 0.17019645869731903,
+      "learning_rate": 4.496588775118232e-05,
+      "loss": 0.2576,
+      "num_input_tokens_seen": 2924800,
+      "step": 111
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.41787323355674744,
+      "learning_rate": 4.48780199123712e-05,
+      "loss": 0.2734,
+      "num_input_tokens_seen": 2927904,
+      "step": 112
+    },
+    {
+      "epoch": 4.0368663594470044,
+      "grad_norm": 0.18536736071109772,
+      "learning_rate": 4.478947927644258e-05,
+      "loss": 0.2363,
+      "num_input_tokens_seen": 2954728,
+      "step": 113
+    },
+    {
+      "epoch": 4.073732718894009,
+      "grad_norm": 0.16922569274902344,
+      "learning_rate": 4.4700268840168045e-05,
+      "loss": 0.1888,
+      "num_input_tokens_seen": 2983656,
+      "step": 114
+    },
+    {
+      "epoch": 4.110599078341014,
+      "grad_norm": 0.17613869905471802,
+      "learning_rate": 4.4610391622989396e-05,
+      "loss": 0.2077,
+      "num_input_tokens_seen": 3014208,
+      "step": 115
+    },
+    {
+      "epoch": 4.147465437788019,
+      "grad_norm": 0.16198807954788208,
+      "learning_rate": 4.4519850666916484e-05,
+      "loss": 0.1858,
+      "num_input_tokens_seen": 3040976,
+      "step": 116
+    },
+    {
+      "epoch": 4.184331797235023,
+      "grad_norm": 0.19020891189575195,
+      "learning_rate": 4.442864903642428e-05,
+      "loss": 0.2284,
+      "num_input_tokens_seen": 3066744,
+      "step": 117
+    },
+    {
+      "epoch": 4.221198156682028,
+      "grad_norm": 0.21541331708431244,
+      "learning_rate": 4.43367898183491e-05,
+      "loss": 0.1856,
+      "num_input_tokens_seen": 3090320,
+      "step": 118
+    },
+    {
+      "epoch": 4.258064516129032,
+      "grad_norm": 0.19018961489200592,
+      "learning_rate": 4.4244276121784195e-05,
+      "loss": 0.1788,
+      "num_input_tokens_seen": 3114576,
+      "step": 119
+    },
+    {
+      "epoch": 4.2949308755760365,
+      "grad_norm": 0.24048063158988953,
+      "learning_rate": 4.415111107797445e-05,
+      "loss": 0.2155,
+      "num_input_tokens_seen": 3135552,
+      "step": 120
+    },
+    {
+      "epoch": 4.331797235023042,
+      "grad_norm": 0.17166386544704437,
+      "learning_rate": 4.405729784021046e-05,
+      "loss": 0.1831,
+      "num_input_tokens_seen": 3165168,
+      "step": 121
+    },
+    {
+      "epoch": 4.368663594470046,
+      "grad_norm": 0.18577668070793152,
+      "learning_rate": 4.396283958372173e-05,
+      "loss": 0.2168,
+      "num_input_tokens_seen": 3200240,
+      "step": 122
+    },
+    {
+      "epoch": 4.405529953917051,
+      "grad_norm": 0.20443667471408844,
+      "learning_rate": 4.386773950556931e-05,
+      "loss": 0.1978,
+      "num_input_tokens_seen": 3225024,
+      "step": 123
+    },
+    {
+      "epoch": 4.442396313364055,
+      "grad_norm": 0.17636562883853912,
+      "learning_rate": 4.377200082453749e-05,
+      "loss": 0.1727,
+      "num_input_tokens_seen": 3251696,
+      "step": 124
+    },
+    {
+      "epoch": 4.47926267281106,
+      "grad_norm": 0.2019859254360199,
+      "learning_rate": 4.36756267810249e-05,
+      "loss": 0.198,
+      "num_input_tokens_seen": 3275672,
+      "step": 125
+    },
+    {
+      "epoch": 4.516129032258064,
+      "grad_norm": 0.2633707523345947,
+      "learning_rate": 4.357862063693486e-05,
+      "loss": 0.2254,
+      "num_input_tokens_seen": 3305592,
+      "step": 126
+    },
+    {
+      "epoch": 4.552995391705069,
+      "grad_norm": 0.17379336059093475,
+      "learning_rate": 4.34809856755649e-05,
+      "loss": 0.2467,
+      "num_input_tokens_seen": 3345072,
+      "step": 127
+    },
+    {
+      "epoch": 4.589861751152074,
+      "grad_norm": 0.16954895853996277,
+      "learning_rate": 4.3382725201495723e-05,
+      "loss": 0.1616,
+      "num_input_tokens_seen": 3371832,
+      "step": 128
+    },
+    {
+      "epoch": 4.626728110599078,
+      "grad_norm": 0.21207046508789062,
+      "learning_rate": 4.3283842540479264e-05,
+      "loss": 0.189,
+      "num_input_tokens_seen": 3395912,
+      "step": 129
+    },
+    {
+      "epoch": 4.663594470046083,
+      "grad_norm": 0.40675088763237,
+      "learning_rate": 4.318434103932622e-05,
+      "loss": 0.1739,
+      "num_input_tokens_seen": 3423352,
+      "step": 130
+    },
+    {
+      "epoch": 4.700460829493087,
+      "grad_norm": 0.21301084756851196,
+      "learning_rate": 4.30842240657927e-05,
+      "loss": 0.187,
+      "num_input_tokens_seen": 3447992,
+      "step": 131
+    },
+    {
+      "epoch": 4.7373271889400925,
+      "grad_norm": 0.20403216779232025,
+      "learning_rate": 4.2983495008466276e-05,
+      "loss": 0.1786,
+      "num_input_tokens_seen": 3479752,
+      "step": 132
+    },
+    {
+      "epoch": 4.774193548387097,
+      "grad_norm": 0.1969008892774582,
+      "learning_rate": 4.288215727665129e-05,
+      "loss": 0.214,
+      "num_input_tokens_seen": 3509072,
+      "step": 133
+    },
+    {
+      "epoch": 4.811059907834101,
+      "grad_norm": 0.20199979841709137,
+      "learning_rate": 4.278021430025343e-05,
+      "loss": 0.171,
+      "num_input_tokens_seen": 3531848,
+      "step": 134
+    },
+    {
+      "epoch": 4.847926267281106,
+      "grad_norm": 0.2074364721775055,
+      "learning_rate": 4.267766952966369e-05,
+      "loss": 0.1786,
+      "num_input_tokens_seen": 3557144,
+      "step": 135
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 540,
+  "num_input_tokens_seen": 3557144,
+  "num_train_epochs": 20,
+  "save_steps": 27,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.000866178869494e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-162/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-162/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-162/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-162/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-162/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-189/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-189/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-216/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-216/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-243/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-243/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-243/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-243/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-243/trainer_state.json

@@ -0,0 +1,1977 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 8.700460829493087,
+  "eval_steps": 500,
+  "global_step": 243,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03686635944700461,
+      "grad_norm": 0.06150835379958153,
+      "learning_rate": 4.999957692144361e-05,
+      "loss": 0.663,
+      "num_input_tokens_seen": 23408,
+      "step": 1
+    },
+    {
+      "epoch": 0.07373271889400922,
+      "grad_norm": 0.05766688659787178,
+      "learning_rate": 4.999830770009406e-05,
+      "loss": 0.5847,
+      "num_input_tokens_seen": 45520,
+      "step": 2
+    },
+    {
+      "epoch": 0.11059907834101383,
+      "grad_norm": 0.0780782401561737,
+      "learning_rate": 4.9996192378909786e-05,
+      "loss": 0.5545,
+      "num_input_tokens_seen": 68904,
+      "step": 3
+    },
+    {
+      "epoch": 0.14746543778801843,
+      "grad_norm": 0.07575194537639618,
+      "learning_rate": 4.9993231029486544e-05,
+      "loss": 0.7025,
+      "num_input_tokens_seen": 89696,
+      "step": 4
+    },
+    {
+      "epoch": 0.18433179723502305,
+      "grad_norm": 0.0708586797118187,
+      "learning_rate": 4.998942375205502e-05,
+      "loss": 0.6063,
+      "num_input_tokens_seen": 118440,
+      "step": 5
+    },
+    {
+      "epoch": 0.22119815668202766,
+      "grad_norm": 0.0794093981385231,
+      "learning_rate": 4.99847706754774e-05,
+      "loss": 0.593,
+      "num_input_tokens_seen": 145072,
+      "step": 6
+    },
+    {
+      "epoch": 0.25806451612903225,
+      "grad_norm": 0.04954744130373001,
+      "learning_rate": 4.997927195724303e-05,
+      "loss": 0.4857,
+      "num_input_tokens_seen": 182112,
+      "step": 7
+    },
+    {
+      "epoch": 0.29493087557603687,
+      "grad_norm": 0.06549246609210968,
+      "learning_rate": 4.997292778346312e-05,
+      "loss": 0.5393,
+      "num_input_tokens_seen": 212192,
+      "step": 8
+    },
+    {
+      "epoch": 0.3317972350230415,
+      "grad_norm": 0.07614441215991974,
+      "learning_rate": 4.996573836886435e-05,
+      "loss": 0.7044,
+      "num_input_tokens_seen": 233728,
+      "step": 9
+    },
+    {
+      "epoch": 0.3686635944700461,
+      "grad_norm": 0.07068340480327606,
+      "learning_rate": 4.995770395678171e-05,
+      "loss": 0.6086,
+      "num_input_tokens_seen": 259776,
+      "step": 10
+    },
+    {
+      "epoch": 0.4055299539170507,
+      "grad_norm": 0.07266585528850555,
+      "learning_rate": 4.9948824819150185e-05,
+      "loss": 0.6247,
+      "num_input_tokens_seen": 284816,
+      "step": 11
+    },
+    {
+      "epoch": 0.4423963133640553,
+      "grad_norm": 0.06145177036523819,
+      "learning_rate": 4.993910125649561e-05,
+      "loss": 0.4918,
+      "num_input_tokens_seen": 309776,
+      "step": 12
+    },
+    {
+      "epoch": 0.4792626728110599,
+      "grad_norm": 0.06720694154500961,
+      "learning_rate": 4.992853359792444e-05,
+      "loss": 0.5017,
+      "num_input_tokens_seen": 339064,
+      "step": 13
+    },
+    {
+      "epoch": 0.5161290322580645,
+      "grad_norm": 0.07309022545814514,
+      "learning_rate": 4.9917122201112656e-05,
+      "loss": 0.5447,
+      "num_input_tokens_seen": 369864,
+      "step": 14
+    },
+    {
+      "epoch": 0.5529953917050692,
+      "grad_norm": 0.0778132826089859,
+      "learning_rate": 4.990486745229364e-05,
+      "loss": 0.5034,
+      "num_input_tokens_seen": 397184,
+      "step": 15
+    },
+    {
+      "epoch": 0.5898617511520737,
+      "grad_norm": 0.07727856189012527,
+      "learning_rate": 4.989176976624511e-05,
+      "loss": 0.5277,
+      "num_input_tokens_seen": 426296,
+      "step": 16
+    },
+    {
+      "epoch": 0.6267281105990783,
+      "grad_norm": 0.0700518786907196,
+      "learning_rate": 4.987782958627508e-05,
+      "loss": 0.5356,
+      "num_input_tokens_seen": 459544,
+      "step": 17
+    },
+    {
+      "epoch": 0.663594470046083,
+      "grad_norm": 0.07351912558078766,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.5505,
+      "num_input_tokens_seen": 488616,
+      "step": 18
+    },
+    {
+      "epoch": 0.7004608294930875,
+      "grad_norm": 0.08268705755472183,
+      "learning_rate": 4.9847423660363e-05,
+      "loss": 0.5557,
+      "num_input_tokens_seen": 512176,
+      "step": 19
+    },
+    {
+      "epoch": 0.7373271889400922,
+      "grad_norm": 0.08081424236297607,
+      "learning_rate": 4.983095894354858e-05,
+      "loss": 0.4576,
+      "num_input_tokens_seen": 539456,
+      "step": 20
+    },
+    {
+      "epoch": 0.7741935483870968,
+      "grad_norm": 0.07748426496982574,
+      "learning_rate": 4.9813653791033057e-05,
+      "loss": 0.3918,
+      "num_input_tokens_seen": 561384,
+      "step": 21
+    },
+    {
+      "epoch": 0.8110599078341014,
+      "grad_norm": 0.10133849829435349,
+      "learning_rate": 4.979550878853154e-05,
+      "loss": 0.531,
+      "num_input_tokens_seen": 589792,
+      "step": 22
+    },
+    {
+      "epoch": 0.847926267281106,
+      "grad_norm": 0.09528470784425735,
+      "learning_rate": 4.9776524550184965e-05,
+      "loss": 0.4159,
+      "num_input_tokens_seen": 613944,
+      "step": 23
+    },
+    {
+      "epoch": 0.8847926267281107,
+      "grad_norm": 0.08829868584871292,
+      "learning_rate": 4.975670171853926e-05,
+      "loss": 0.403,
+      "num_input_tokens_seen": 641432,
+      "step": 24
+    },
+    {
+      "epoch": 0.9216589861751152,
+      "grad_norm": 0.10020875930786133,
+      "learning_rate": 4.973604096452361e-05,
+      "loss": 0.4238,
+      "num_input_tokens_seen": 665232,
+      "step": 25
+    },
+    {
+      "epoch": 0.9585253456221198,
+      "grad_norm": 0.1256554126739502,
+      "learning_rate": 4.971454298742779e-05,
+      "loss": 0.5033,
+      "num_input_tokens_seen": 687216,
+      "step": 26
+    },
+    {
+      "epoch": 0.9953917050691244,
+      "grad_norm": 0.17123407125473022,
+      "learning_rate": 4.9692208514878444e-05,
+      "loss": 0.3842,
+      "num_input_tokens_seen": 728880,
+      "step": 27
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.27383726835250854,
+      "learning_rate": 4.966903830281449e-05,
+      "loss": 0.4988,
+      "num_input_tokens_seen": 731976,
+      "step": 28
+    },
+    {
+      "epoch": 1.0368663594470047,
+      "grad_norm": 0.148647278547287,
+      "learning_rate": 4.9645033135461494e-05,
+      "loss": 0.4489,
+      "num_input_tokens_seen": 756184,
+      "step": 29
+    },
+    {
+      "epoch": 1.0737327188940091,
+      "grad_norm": 0.11988866329193115,
+      "learning_rate": 4.962019382530521e-05,
+      "loss": 0.4999,
+      "num_input_tokens_seen": 779072,
+      "step": 30
+    },
+    {
+      "epoch": 1.1105990783410138,
+      "grad_norm": 0.10590679943561554,
+      "learning_rate": 4.9594521213063974e-05,
+      "loss": 0.3711,
+      "num_input_tokens_seen": 805672,
+      "step": 31
+    },
+    {
+      "epoch": 1.1474654377880185,
+      "grad_norm": 0.11934591829776764,
+      "learning_rate": 4.9568016167660334e-05,
+      "loss": 0.3385,
+      "num_input_tokens_seen": 828840,
+      "step": 32
+    },
+    {
+      "epoch": 1.1843317972350231,
+      "grad_norm": 0.11900116503238678,
+      "learning_rate": 4.9540679586191605e-05,
+      "loss": 0.3812,
+      "num_input_tokens_seen": 850792,
+      "step": 33
+    },
+    {
+      "epoch": 1.2211981566820276,
+      "grad_norm": 0.11279546469449997,
+      "learning_rate": 4.951251239389948e-05,
+      "loss": 0.2924,
+      "num_input_tokens_seen": 885608,
+      "step": 34
+    },
+    {
+      "epoch": 1.2580645161290323,
+      "grad_norm": 0.08340126276016235,
+      "learning_rate": 4.948351554413879e-05,
+      "loss": 0.3489,
+      "num_input_tokens_seen": 921912,
+      "step": 35
+    },
+    {
+      "epoch": 1.294930875576037,
+      "grad_norm": 0.11327336728572845,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.2921,
+      "num_input_tokens_seen": 945672,
+      "step": 36
+    },
+    {
+      "epoch": 1.3317972350230414,
+      "grad_norm": 0.10909265279769897,
+      "learning_rate": 4.942303682600178e-05,
+      "loss": 0.3421,
+      "num_input_tokens_seen": 970824,
+      "step": 37
+    },
+    {
+      "epoch": 1.368663594470046,
+      "grad_norm": 0.11239592730998993,
+      "learning_rate": 4.939155700460536e-05,
+      "loss": 0.3848,
+      "num_input_tokens_seen": 995840,
+      "step": 38
+    },
+    {
+      "epoch": 1.4055299539170507,
+      "grad_norm": 0.1034158244729042,
+      "learning_rate": 4.9359251619630886e-05,
+      "loss": 0.3434,
+      "num_input_tokens_seen": 1035880,
+      "step": 39
+    },
+    {
+      "epoch": 1.4423963133640554,
+      "grad_norm": 0.12086673080921173,
+      "learning_rate": 4.9326121764495596e-05,
+      "loss": 0.3588,
+      "num_input_tokens_seen": 1056488,
+      "step": 40
+    },
+    {
+      "epoch": 1.4792626728110598,
+      "grad_norm": 0.11408794671297073,
+      "learning_rate": 4.9292168560522014e-05,
+      "loss": 0.4021,
+      "num_input_tokens_seen": 1080080,
+      "step": 41
+    },
+    {
+      "epoch": 1.5161290322580645,
+      "grad_norm": 0.1417722851037979,
+      "learning_rate": 4.925739315689991e-05,
+      "loss": 0.3343,
+      "num_input_tokens_seen": 1095632,
+      "step": 42
+    },
+    {
+      "epoch": 1.5529953917050692,
+      "grad_norm": 0.2315663844347,
+      "learning_rate": 4.9221796730647516e-05,
+      "loss": 0.4025,
+      "num_input_tokens_seen": 1126040,
+      "step": 43
+    },
+    {
+      "epoch": 1.5898617511520738,
+      "grad_norm": 0.07856712490320206,
+      "learning_rate": 4.9185380486571595e-05,
+      "loss": 0.3337,
+      "num_input_tokens_seen": 1156656,
+      "step": 44
+    },
+    {
+      "epoch": 1.6267281105990783,
+      "grad_norm": 0.10064279288053513,
+      "learning_rate": 4.914814565722671e-05,
+      "loss": 0.3758,
+      "num_input_tokens_seen": 1183400,
+      "step": 45
+    },
+    {
+      "epoch": 1.663594470046083,
+      "grad_norm": 0.106510229408741,
+      "learning_rate": 4.9110093502873476e-05,
+      "loss": 0.3363,
+      "num_input_tokens_seen": 1209680,
+      "step": 46
+    },
+    {
+      "epoch": 1.7004608294930876,
+      "grad_norm": 0.10446175187826157,
+      "learning_rate": 4.907122531143594e-05,
+      "loss": 0.3853,
+      "num_input_tokens_seen": 1237864,
+      "step": 47
+    },
+    {
+      "epoch": 1.737327188940092,
+      "grad_norm": 0.2570022940635681,
+      "learning_rate": 4.9031542398457974e-05,
+      "loss": 0.3194,
+      "num_input_tokens_seen": 1273080,
+      "step": 48
+    },
+    {
+      "epoch": 1.7741935483870968,
+      "grad_norm": 0.0968039259314537,
+      "learning_rate": 4.8991046107058735e-05,
+      "loss": 0.2955,
+      "num_input_tokens_seen": 1300256,
+      "step": 49
+    },
+    {
+      "epoch": 1.8110599078341014,
+      "grad_norm": 0.11750692129135132,
+      "learning_rate": 4.894973780788722e-05,
+      "loss": 0.3314,
+      "num_input_tokens_seen": 1322616,
+      "step": 50
+    },
+    {
+      "epoch": 1.8479262672811059,
+      "grad_norm": 0.09568070620298386,
+      "learning_rate": 4.890761889907589e-05,
+      "loss": 0.3341,
+      "num_input_tokens_seen": 1353552,
+      "step": 51
+    },
+    {
+      "epoch": 1.8847926267281108,
+      "grad_norm": 0.11816362291574478,
+      "learning_rate": 4.88646908061933e-05,
+      "loss": 0.3202,
+      "num_input_tokens_seen": 1376456,
+      "step": 52
+    },
+    {
+      "epoch": 1.9216589861751152,
+      "grad_norm": 0.133891761302948,
+      "learning_rate": 4.8820954982195905e-05,
+      "loss": 0.2852,
+      "num_input_tokens_seen": 1398344,
+      "step": 53
+    },
+    {
+      "epoch": 1.9585253456221197,
+      "grad_norm": 0.11985364556312561,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.3413,
+      "num_input_tokens_seen": 1424264,
+      "step": 54
+    },
+    {
+      "epoch": 1.9953917050691246,
+      "grad_norm": 0.09828098118305206,
+      "learning_rate": 4.873106608932585e-05,
+      "loss": 0.2726,
+      "num_input_tokens_seen": 1460688,
+      "step": 55
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.31861889362335205,
+      "learning_rate": 4.868491606285823e-05,
+      "loss": 0.3512,
+      "num_input_tokens_seen": 1463952,
+      "step": 56
+    },
+    {
+      "epoch": 2.0368663594470044,
+      "grad_norm": 0.11089354753494263,
+      "learning_rate": 4.8637964389982926e-05,
+      "loss": 0.2953,
+      "num_input_tokens_seen": 1492600,
+      "step": 57
+    },
+    {
+      "epoch": 2.0737327188940093,
+      "grad_norm": 0.12034712731838226,
+      "learning_rate": 4.859021265983959e-05,
+      "loss": 0.2875,
+      "num_input_tokens_seen": 1514496,
+      "step": 58
+    },
+    {
+      "epoch": 2.110599078341014,
+      "grad_norm": 0.1372830867767334,
+      "learning_rate": 4.854166248864689e-05,
+      "loss": 0.3535,
+      "num_input_tokens_seen": 1541656,
+      "step": 59
+    },
+    {
+      "epoch": 2.1474654377880182,
+      "grad_norm": 0.10605650395154953,
+      "learning_rate": 4.849231551964771e-05,
+      "loss": 0.3013,
+      "num_input_tokens_seen": 1569672,
+      "step": 60
+    },
+    {
+      "epoch": 2.184331797235023,
+      "grad_norm": 0.12524856626987457,
+      "learning_rate": 4.844217342305363e-05,
+      "loss": 0.2558,
+      "num_input_tokens_seen": 1599304,
+      "step": 61
+    },
+    {
+      "epoch": 2.2211981566820276,
+      "grad_norm": 0.1151009052991867,
+      "learning_rate": 4.839123789598829e-05,
+      "loss": 0.2666,
+      "num_input_tokens_seen": 1628288,
+      "step": 62
+    },
+    {
+      "epoch": 2.258064516129032,
+      "grad_norm": 0.10297228395938873,
+      "learning_rate": 4.8339510662430046e-05,
+      "loss": 0.2982,
+      "num_input_tokens_seen": 1656384,
+      "step": 63
+    },
+    {
+      "epoch": 2.294930875576037,
+      "grad_norm": 0.11971966177225113,
+      "learning_rate": 4.828699347315356e-05,
+      "loss": 0.3746,
+      "num_input_tokens_seen": 1679344,
+      "step": 64
+    },
+    {
+      "epoch": 2.3317972350230414,
+      "grad_norm": 0.12384694814682007,
+      "learning_rate": 4.823368810567056e-05,
+      "loss": 0.2605,
+      "num_input_tokens_seen": 1703904,
+      "step": 65
+    },
+    {
+      "epoch": 2.3686635944700463,
+      "grad_norm": 0.14720793068408966,
+      "learning_rate": 4.817959636416969e-05,
+      "loss": 0.2816,
+      "num_input_tokens_seen": 1722360,
+      "step": 66
+    },
+    {
+      "epoch": 2.4055299539170507,
+      "grad_norm": 0.10553745925426483,
+      "learning_rate": 4.81247200794554e-05,
+      "loss": 0.2502,
+      "num_input_tokens_seen": 1752464,
+      "step": 67
+    },
+    {
+      "epoch": 2.442396313364055,
+      "grad_norm": 0.11162563413381577,
+      "learning_rate": 4.806906110888606e-05,
+      "loss": 0.2286,
+      "num_input_tokens_seen": 1782744,
+      "step": 68
+    },
+    {
+      "epoch": 2.47926267281106,
+      "grad_norm": 0.12341686338186264,
+      "learning_rate": 4.8012621336311016e-05,
+      "loss": 0.299,
+      "num_input_tokens_seen": 1815120,
+      "step": 69
+    },
+    {
+      "epoch": 2.5161290322580645,
+      "grad_norm": 0.12033814936876297,
+      "learning_rate": 4.7955402672006854e-05,
+      "loss": 0.2609,
+      "num_input_tokens_seen": 1840408,
+      "step": 70
+    },
+    {
+      "epoch": 2.5529953917050694,
+      "grad_norm": 0.11412363499403,
+      "learning_rate": 4.789740705261278e-05,
+      "loss": 0.3316,
+      "num_input_tokens_seen": 1873000,
+      "step": 71
+    },
+    {
+      "epoch": 2.589861751152074,
+      "grad_norm": 0.18193566799163818,
+      "learning_rate": 4.783863644106502e-05,
+      "loss": 0.2773,
+      "num_input_tokens_seen": 1900920,
+      "step": 72
+    },
+    {
+      "epoch": 2.6267281105990783,
+      "grad_norm": 0.14569184184074402,
+      "learning_rate": 4.777909282653042e-05,
+      "loss": 0.2644,
+      "num_input_tokens_seen": 1919272,
+      "step": 73
+    },
+    {
+      "epoch": 2.6635944700460827,
+      "grad_norm": 0.12456360459327698,
+      "learning_rate": 4.771877822433911e-05,
+      "loss": 0.2974,
+      "num_input_tokens_seen": 1959152,
+      "step": 74
+    },
+    {
+      "epoch": 2.7004608294930876,
+      "grad_norm": 0.1362839788198471,
+      "learning_rate": 4.765769467591625e-05,
+      "loss": 0.2743,
+      "num_input_tokens_seen": 1982960,
+      "step": 75
+    },
+    {
+      "epoch": 2.737327188940092,
+      "grad_norm": 0.15120629966259003,
+      "learning_rate": 4.759584424871302e-05,
+      "loss": 0.2425,
+      "num_input_tokens_seen": 2006736,
+      "step": 76
+    },
+    {
+      "epoch": 2.774193548387097,
+      "grad_norm": 0.15145516395568848,
+      "learning_rate": 4.7533229036136553e-05,
+      "loss": 0.2723,
+      "num_input_tokens_seen": 2031536,
+      "step": 77
+    },
+    {
+      "epoch": 2.8110599078341014,
+      "grad_norm": 0.1607382595539093,
+      "learning_rate": 4.7469851157479177e-05,
+      "loss": 0.2525,
+      "num_input_tokens_seen": 2053168,
+      "step": 78
+    },
+    {
+      "epoch": 2.847926267281106,
+      "grad_norm": 0.14976347982883453,
+      "learning_rate": 4.740571275784659e-05,
+      "loss": 0.2792,
+      "num_input_tokens_seen": 2078208,
+      "step": 79
+    },
+    {
+      "epoch": 2.8847926267281108,
+      "grad_norm": 0.12997782230377197,
+      "learning_rate": 4.734081600808531e-05,
+      "loss": 0.291,
+      "num_input_tokens_seen": 2101560,
+      "step": 80
+    },
+    {
+      "epoch": 2.921658986175115,
+      "grad_norm": 0.1369074434041977,
+      "learning_rate": 4.72751631047092e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2127824,
+      "step": 81
+    },
+    {
+      "epoch": 2.9585253456221197,
+      "grad_norm": 0.11734048277139664,
+      "learning_rate": 4.7208756269825104e-05,
+      "loss": 0.2442,
+      "num_input_tokens_seen": 2171536,
+      "step": 82
+    },
+    {
+      "epoch": 2.9953917050691246,
+      "grad_norm": 0.14909496903419495,
+      "learning_rate": 4.714159775105765e-05,
+      "loss": 0.2344,
+      "num_input_tokens_seen": 2193016,
+      "step": 83
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.40910854935646057,
+      "learning_rate": 4.707368982147318e-05,
+      "loss": 0.2645,
+      "num_input_tokens_seen": 2195928,
+      "step": 84
+    },
+    {
+      "epoch": 3.0368663594470044,
+      "grad_norm": 0.1433151513338089,
+      "learning_rate": 4.700503477950278e-05,
+      "loss": 0.2185,
+      "num_input_tokens_seen": 2219472,
+      "step": 85
+    },
+    {
+      "epoch": 3.0737327188940093,
+      "grad_norm": 0.13229826092720032,
+      "learning_rate": 4.693563494886455e-05,
+      "loss": 0.2364,
+      "num_input_tokens_seen": 2246504,
+      "step": 86
+    },
+    {
+      "epoch": 3.110599078341014,
+      "grad_norm": 0.1702108383178711,
+      "learning_rate": 4.6865492678484895e-05,
+      "loss": 0.2703,
+      "num_input_tokens_seen": 2270304,
+      "step": 87
+    },
+    {
+      "epoch": 3.1474654377880182,
+      "grad_norm": 0.1411813497543335,
+      "learning_rate": 4.679461034241906e-05,
+      "loss": 0.2807,
+      "num_input_tokens_seen": 2299968,
+      "step": 88
+    },
+    {
+      "epoch": 3.184331797235023,
+      "grad_norm": 0.12941044569015503,
+      "learning_rate": 4.672299033977076e-05,
+      "loss": 0.2182,
+      "num_input_tokens_seen": 2333432,
+      "step": 89
+    },
+    {
+      "epoch": 3.2211981566820276,
+      "grad_norm": 0.1649884432554245,
+      "learning_rate": 4.665063509461097e-05,
+      "loss": 0.2572,
+      "num_input_tokens_seen": 2355320,
+      "step": 90
+    },
+    {
+      "epoch": 3.258064516129032,
+      "grad_norm": 0.14386451244354248,
+      "learning_rate": 4.657754705589591e-05,
+      "loss": 0.2412,
+      "num_input_tokens_seen": 2378416,
+      "step": 91
+    },
+    {
+      "epoch": 3.294930875576037,
+      "grad_norm": 0.15350477397441864,
+      "learning_rate": 4.650372869738414e-05,
+      "loss": 0.242,
+      "num_input_tokens_seen": 2405432,
+      "step": 92
+    },
+    {
+      "epoch": 3.3317972350230414,
+      "grad_norm": 0.16243231296539307,
+      "learning_rate": 4.642918251755281e-05,
+      "loss": 0.2436,
+      "num_input_tokens_seen": 2429992,
+      "step": 93
+    },
+    {
+      "epoch": 3.3686635944700463,
+      "grad_norm": 0.18401454389095306,
+      "learning_rate": 4.6353911039513145e-05,
+      "loss": 0.2538,
+      "num_input_tokens_seen": 2456264,
+      "step": 94
+    },
+    {
+      "epoch": 3.4055299539170507,
+      "grad_norm": 0.13898858428001404,
+      "learning_rate": 4.627791681092499e-05,
+      "loss": 0.2065,
+      "num_input_tokens_seen": 2486032,
+      "step": 95
+    },
+    {
+      "epoch": 3.442396313364055,
+      "grad_norm": 0.16943225264549255,
+      "learning_rate": 4.620120240391065e-05,
+      "loss": 0.2086,
+      "num_input_tokens_seen": 2509272,
+      "step": 96
+    },
+    {
+      "epoch": 3.47926267281106,
+      "grad_norm": 0.15286582708358765,
+      "learning_rate": 4.612377041496776e-05,
+      "loss": 0.2517,
+      "num_input_tokens_seen": 2538392,
+      "step": 97
+    },
+    {
+      "epoch": 3.5161290322580645,
+      "grad_norm": 0.17737072706222534,
+      "learning_rate": 4.604562346488144e-05,
+      "loss": 0.2413,
+      "num_input_tokens_seen": 2562624,
+      "step": 98
+    },
+    {
+      "epoch": 3.5529953917050694,
+      "grad_norm": 0.13630996644496918,
+      "learning_rate": 4.5966764198635606e-05,
+      "loss": 0.2157,
+      "num_input_tokens_seen": 2595448,
+      "step": 99
+    },
+    {
+      "epoch": 3.589861751152074,
+      "grad_norm": 0.13918158411979675,
+      "learning_rate": 4.588719528532342e-05,
+      "loss": 0.2319,
+      "num_input_tokens_seen": 2633568,
+      "step": 100
+    },
+    {
+      "epoch": 3.6267281105990783,
+      "grad_norm": 0.16548505425453186,
+      "learning_rate": 4.580691941805695e-05,
+      "loss": 0.2473,
+      "num_input_tokens_seen": 2659408,
+      "step": 101
+    },
+    {
+      "epoch": 3.6635944700460827,
+      "grad_norm": 0.17397943139076233,
+      "learning_rate": 4.572593931387604e-05,
+      "loss": 0.2527,
+      "num_input_tokens_seen": 2687272,
+      "step": 102
+    },
+    {
+      "epoch": 3.7004608294930876,
+      "grad_norm": 0.15970823168754578,
+      "learning_rate": 4.5644257713656356e-05,
+      "loss": 0.22,
+      "num_input_tokens_seen": 2718232,
+      "step": 103
+    },
+    {
+      "epoch": 3.737327188940092,
+      "grad_norm": 0.16440241038799286,
+      "learning_rate": 4.556187738201656e-05,
+      "loss": 0.2374,
+      "num_input_tokens_seen": 2743336,
+      "step": 104
+    },
+    {
+      "epoch": 3.774193548387097,
+      "grad_norm": 0.15761925280094147,
+      "learning_rate": 4.54788011072248e-05,
+      "loss": 0.2023,
+      "num_input_tokens_seen": 2771384,
+      "step": 105
+    },
+    {
+      "epoch": 3.8110599078341014,
+      "grad_norm": 0.1528206616640091,
+      "learning_rate": 4.539503170110431e-05,
+      "loss": 0.2289,
+      "num_input_tokens_seen": 2797488,
+      "step": 106
+    },
+    {
+      "epoch": 3.847926267281106,
+      "grad_norm": 0.16380636394023895,
+      "learning_rate": 4.531057199893824e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2825408,
+      "step": 107
+    },
+    {
+      "epoch": 3.8847926267281108,
+      "grad_norm": 0.19076082110404968,
+      "learning_rate": 4.522542485937369e-05,
+      "loss": 0.2467,
+      "num_input_tokens_seen": 2848856,
+      "step": 108
+    },
+    {
+      "epoch": 3.921658986175115,
+      "grad_norm": 0.1884787231683731,
+      "learning_rate": 4.5139593164324986e-05,
+      "loss": 0.2069,
+      "num_input_tokens_seen": 2867536,
+      "step": 109
+    },
+    {
+      "epoch": 3.9585253456221197,
+      "grad_norm": 0.19939164817333221,
+      "learning_rate": 4.50530798188761e-05,
+      "loss": 0.1843,
+      "num_input_tokens_seen": 2892032,
+      "step": 110
+    },
+    {
+      "epoch": 3.9953917050691246,
+      "grad_norm": 0.17019645869731903,
+      "learning_rate": 4.496588775118232e-05,
+      "loss": 0.2576,
+      "num_input_tokens_seen": 2924800,
+      "step": 111
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.41787323355674744,
+      "learning_rate": 4.48780199123712e-05,
+      "loss": 0.2734,
+      "num_input_tokens_seen": 2927904,
+      "step": 112
+    },
+    {
+      "epoch": 4.0368663594470044,
+      "grad_norm": 0.18536736071109772,
+      "learning_rate": 4.478947927644258e-05,
+      "loss": 0.2363,
+      "num_input_tokens_seen": 2954728,
+      "step": 113
+    },
+    {
+      "epoch": 4.073732718894009,
+      "grad_norm": 0.16922569274902344,
+      "learning_rate": 4.4700268840168045e-05,
+      "loss": 0.1888,
+      "num_input_tokens_seen": 2983656,
+      "step": 114
+    },
+    {
+      "epoch": 4.110599078341014,
+      "grad_norm": 0.17613869905471802,
+      "learning_rate": 4.4610391622989396e-05,
+      "loss": 0.2077,
+      "num_input_tokens_seen": 3014208,
+      "step": 115
+    },
+    {
+      "epoch": 4.147465437788019,
+      "grad_norm": 0.16198807954788208,
+      "learning_rate": 4.4519850666916484e-05,
+      "loss": 0.1858,
+      "num_input_tokens_seen": 3040976,
+      "step": 116
+    },
+    {
+      "epoch": 4.184331797235023,
+      "grad_norm": 0.19020891189575195,
+      "learning_rate": 4.442864903642428e-05,
+      "loss": 0.2284,
+      "num_input_tokens_seen": 3066744,
+      "step": 117
+    },
+    {
+      "epoch": 4.221198156682028,
+      "grad_norm": 0.21541331708431244,
+      "learning_rate": 4.43367898183491e-05,
+      "loss": 0.1856,
+      "num_input_tokens_seen": 3090320,
+      "step": 118
+    },
+    {
+      "epoch": 4.258064516129032,
+      "grad_norm": 0.19018961489200592,
+      "learning_rate": 4.4244276121784195e-05,
+      "loss": 0.1788,
+      "num_input_tokens_seen": 3114576,
+      "step": 119
+    },
+    {
+      "epoch": 4.2949308755760365,
+      "grad_norm": 0.24048063158988953,
+      "learning_rate": 4.415111107797445e-05,
+      "loss": 0.2155,
+      "num_input_tokens_seen": 3135552,
+      "step": 120
+    },
+    {
+      "epoch": 4.331797235023042,
+      "grad_norm": 0.17166386544704437,
+      "learning_rate": 4.405729784021046e-05,
+      "loss": 0.1831,
+      "num_input_tokens_seen": 3165168,
+      "step": 121
+    },
+    {
+      "epoch": 4.368663594470046,
+      "grad_norm": 0.18577668070793152,
+      "learning_rate": 4.396283958372173e-05,
+      "loss": 0.2168,
+      "num_input_tokens_seen": 3200240,
+      "step": 122
+    },
+    {
+      "epoch": 4.405529953917051,
+      "grad_norm": 0.20443667471408844,
+      "learning_rate": 4.386773950556931e-05,
+      "loss": 0.1978,
+      "num_input_tokens_seen": 3225024,
+      "step": 123
+    },
+    {
+      "epoch": 4.442396313364055,
+      "grad_norm": 0.17636562883853912,
+      "learning_rate": 4.377200082453749e-05,
+      "loss": 0.1727,
+      "num_input_tokens_seen": 3251696,
+      "step": 124
+    },
+    {
+      "epoch": 4.47926267281106,
+      "grad_norm": 0.2019859254360199,
+      "learning_rate": 4.36756267810249e-05,
+      "loss": 0.198,
+      "num_input_tokens_seen": 3275672,
+      "step": 125
+    },
+    {
+      "epoch": 4.516129032258064,
+      "grad_norm": 0.2633707523345947,
+      "learning_rate": 4.357862063693486e-05,
+      "loss": 0.2254,
+      "num_input_tokens_seen": 3305592,
+      "step": 126
+    },
+    {
+      "epoch": 4.552995391705069,
+      "grad_norm": 0.17379336059093475,
+      "learning_rate": 4.34809856755649e-05,
+      "loss": 0.2467,
+      "num_input_tokens_seen": 3345072,
+      "step": 127
+    },
+    {
+      "epoch": 4.589861751152074,
+      "grad_norm": 0.16954895853996277,
+      "learning_rate": 4.3382725201495723e-05,
+      "loss": 0.1616,
+      "num_input_tokens_seen": 3371832,
+      "step": 128
+    },
+    {
+      "epoch": 4.626728110599078,
+      "grad_norm": 0.21207046508789062,
+      "learning_rate": 4.3283842540479264e-05,
+      "loss": 0.189,
+      "num_input_tokens_seen": 3395912,
+      "step": 129
+    },
+    {
+      "epoch": 4.663594470046083,
+      "grad_norm": 0.40675088763237,
+      "learning_rate": 4.318434103932622e-05,
+      "loss": 0.1739,
+      "num_input_tokens_seen": 3423352,
+      "step": 130
+    },
+    {
+      "epoch": 4.700460829493087,
+      "grad_norm": 0.21301084756851196,
+      "learning_rate": 4.30842240657927e-05,
+      "loss": 0.187,
+      "num_input_tokens_seen": 3447992,
+      "step": 131
+    },
+    {
+      "epoch": 4.7373271889400925,
+      "grad_norm": 0.20403216779232025,
+      "learning_rate": 4.2983495008466276e-05,
+      "loss": 0.1786,
+      "num_input_tokens_seen": 3479752,
+      "step": 132
+    },
+    {
+      "epoch": 4.774193548387097,
+      "grad_norm": 0.1969008892774582,
+      "learning_rate": 4.288215727665129e-05,
+      "loss": 0.214,
+      "num_input_tokens_seen": 3509072,
+      "step": 133
+    },
+    {
+      "epoch": 4.811059907834101,
+      "grad_norm": 0.20199979841709137,
+      "learning_rate": 4.278021430025343e-05,
+      "loss": 0.171,
+      "num_input_tokens_seen": 3531848,
+      "step": 134
+    },
+    {
+      "epoch": 4.847926267281106,
+      "grad_norm": 0.2074364721775055,
+      "learning_rate": 4.267766952966369e-05,
+      "loss": 0.1786,
+      "num_input_tokens_seen": 3557144,
+      "step": 135
+    },
+    {
+      "epoch": 4.88479262672811,
+      "grad_norm": 0.23071159422397614,
+      "learning_rate": 4.257452643564155e-05,
+      "loss": 0.1816,
+      "num_input_tokens_seen": 3578656,
+      "step": 136
+    },
+    {
+      "epoch": 4.921658986175116,
+      "grad_norm": 0.2532529830932617,
+      "learning_rate": 4.24707885091975e-05,
+      "loss": 0.1885,
+      "num_input_tokens_seen": 3608632,
+      "step": 137
+    },
+    {
+      "epoch": 4.95852534562212,
+      "grad_norm": 0.24982504546642303,
+      "learning_rate": 4.2366459261474933e-05,
+      "loss": 0.2125,
+      "num_input_tokens_seen": 3628672,
+      "step": 138
+    },
+    {
+      "epoch": 4.9953917050691246,
+      "grad_norm": 0.17841289937496185,
+      "learning_rate": 4.226154222363124e-05,
+      "loss": 0.2001,
+      "num_input_tokens_seen": 3657088,
+      "step": 139
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.6343802809715271,
+      "learning_rate": 4.215604094671835e-05,
+      "loss": 0.2376,
+      "num_input_tokens_seen": 3659880,
+      "step": 140
+    },
+    {
+      "epoch": 5.0368663594470044,
+      "grad_norm": 0.21179749071598053,
+      "learning_rate": 4.2049959001562464e-05,
+      "loss": 0.1942,
+      "num_input_tokens_seen": 3684760,
+      "step": 141
+    },
+    {
+      "epoch": 5.073732718894009,
+      "grad_norm": 0.22715945541858673,
+      "learning_rate": 4.194329997864331e-05,
+      "loss": 0.1849,
+      "num_input_tokens_seen": 3715176,
+      "step": 142
+    },
+    {
+      "epoch": 5.110599078341014,
+      "grad_norm": 0.20638404786586761,
+      "learning_rate": 4.183606748797251e-05,
+      "loss": 0.1492,
+      "num_input_tokens_seen": 3741480,
+      "step": 143
+    },
+    {
+      "epoch": 5.147465437788019,
+      "grad_norm": 0.25307852029800415,
+      "learning_rate": 4.172826515897146e-05,
+      "loss": 0.1699,
+      "num_input_tokens_seen": 3762976,
+      "step": 144
+    },
+    {
+      "epoch": 5.184331797235023,
+      "grad_norm": 0.21923506259918213,
+      "learning_rate": 4.1619896640348445e-05,
+      "loss": 0.1559,
+      "num_input_tokens_seen": 3794720,
+      "step": 145
+    },
+    {
+      "epoch": 5.221198156682028,
+      "grad_norm": 0.2600756883621216,
+      "learning_rate": 4.1510965599975196e-05,
+      "loss": 0.1843,
+      "num_input_tokens_seen": 3822744,
+      "step": 146
+    },
+    {
+      "epoch": 5.258064516129032,
+      "grad_norm": 0.2997046113014221,
+      "learning_rate": 4.140147572476268e-05,
+      "loss": 0.243,
+      "num_input_tokens_seen": 3845584,
+      "step": 147
+    },
+    {
+      "epoch": 5.2949308755760365,
+      "grad_norm": 0.2147158831357956,
+      "learning_rate": 4.129143072053638e-05,
+      "loss": 0.1476,
+      "num_input_tokens_seen": 3869184,
+      "step": 148
+    },
+    {
+      "epoch": 5.331797235023042,
+      "grad_norm": 0.26199281215667725,
+      "learning_rate": 4.118083431191081e-05,
+      "loss": 0.1619,
+      "num_input_tokens_seen": 3900384,
+      "step": 149
+    },
+    {
+      "epoch": 5.368663594470046,
+      "grad_norm": 0.4091247618198395,
+      "learning_rate": 4.1069690242163484e-05,
+      "loss": 0.1943,
+      "num_input_tokens_seen": 3923792,
+      "step": 150
+    },
+    {
+      "epoch": 5.405529953917051,
+      "grad_norm": 0.23406103253364563,
+      "learning_rate": 4.095800227310821e-05,
+      "loss": 0.1403,
+      "num_input_tokens_seen": 3942784,
+      "step": 151
+    },
+    {
+      "epoch": 5.442396313364055,
+      "grad_norm": 0.2293683886528015,
+      "learning_rate": 4.0845774184967754e-05,
+      "loss": 0.1756,
+      "num_input_tokens_seen": 3970880,
+      "step": 152
+    },
+    {
+      "epoch": 5.47926267281106,
+      "grad_norm": 0.1963953971862793,
+      "learning_rate": 4.073300977624594e-05,
+      "loss": 0.1785,
+      "num_input_tokens_seen": 3998976,
+      "step": 153
+    },
+    {
+      "epoch": 5.516129032258064,
+      "grad_norm": 0.23071235418319702,
+      "learning_rate": 4.0619712863599e-05,
+      "loss": 0.165,
+      "num_input_tokens_seen": 4020272,
+      "step": 154
+    },
+    {
+      "epoch": 5.552995391705069,
+      "grad_norm": 0.2570018172264099,
+      "learning_rate": 4.05058872817065e-05,
+      "loss": 0.1639,
+      "num_input_tokens_seen": 4043288,
+      "step": 155
+    },
+    {
+      "epoch": 5.589861751152074,
+      "grad_norm": 0.2581011950969696,
+      "learning_rate": 4.039153688314145e-05,
+      "loss": 0.1818,
+      "num_input_tokens_seen": 4067832,
+      "step": 156
+    },
+    {
+      "epoch": 5.626728110599078,
+      "grad_norm": 0.23090755939483643,
+      "learning_rate": 4.0276665538239996e-05,
+      "loss": 0.1372,
+      "num_input_tokens_seen": 4093672,
+      "step": 157
+    },
+    {
+      "epoch": 5.663594470046083,
+      "grad_norm": 0.217911958694458,
+      "learning_rate": 4.0161277134970345e-05,
+      "loss": 0.1648,
+      "num_input_tokens_seen": 4119400,
+      "step": 158
+    },
+    {
+      "epoch": 5.700460829493087,
+      "grad_norm": 0.3220454454421997,
+      "learning_rate": 4.0045375578801214e-05,
+      "loss": 0.1516,
+      "num_input_tokens_seen": 4148144,
+      "step": 159
+    },
+    {
+      "epoch": 5.7373271889400925,
+      "grad_norm": 0.2835656404495239,
+      "learning_rate": 3.9928964792569655e-05,
+      "loss": 0.1597,
+      "num_input_tokens_seen": 4169456,
+      "step": 160
+    },
+    {
+      "epoch": 5.774193548387097,
+      "grad_norm": 0.22007523477077484,
+      "learning_rate": 3.981204871634827e-05,
+      "loss": 0.1535,
+      "num_input_tokens_seen": 4207704,
+      "step": 161
+    },
+    {
+      "epoch": 5.811059907834101,
+      "grad_norm": 0.23791228234767914,
+      "learning_rate": 3.969463130731183e-05,
+      "loss": 0.1472,
+      "num_input_tokens_seen": 4230304,
+      "step": 162
+    },
+    {
+      "epoch": 5.847926267281106,
+      "grad_norm": 0.21866248548030853,
+      "learning_rate": 3.957671653960337e-05,
+      "loss": 0.1362,
+      "num_input_tokens_seen": 4255536,
+      "step": 163
+    },
+    {
+      "epoch": 5.88479262672811,
+      "grad_norm": 0.24037593603134155,
+      "learning_rate": 3.945830840419966e-05,
+      "loss": 0.1443,
+      "num_input_tokens_seen": 4285160,
+      "step": 164
+    },
+    {
+      "epoch": 5.921658986175116,
+      "grad_norm": 0.19541749358177185,
+      "learning_rate": 3.933941090877615e-05,
+      "loss": 0.1647,
+      "num_input_tokens_seen": 4323272,
+      "step": 165
+    },
+    {
+      "epoch": 5.95852534562212,
+      "grad_norm": 0.22034280002117157,
+      "learning_rate": 3.9220028077571295e-05,
+      "loss": 0.1921,
+      "num_input_tokens_seen": 4354912,
+      "step": 166
+    },
+    {
+      "epoch": 5.9953917050691246,
+      "grad_norm": 0.19790567457675934,
+      "learning_rate": 3.910016395125037e-05,
+      "loss": 0.1522,
+      "num_input_tokens_seen": 4387064,
+      "step": 167
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 0.6411383152008057,
+      "learning_rate": 3.897982258676867e-05,
+      "loss": 0.2109,
+      "num_input_tokens_seen": 4391856,
+      "step": 168
+    },
+    {
+      "epoch": 6.0368663594470044,
+      "grad_norm": 0.22729864716529846,
+      "learning_rate": 3.885900805723429e-05,
+      "loss": 0.1607,
+      "num_input_tokens_seen": 4419952,
+      "step": 169
+    },
+    {
+      "epoch": 6.073732718894009,
+      "grad_norm": 0.21639327704906464,
+      "learning_rate": 3.873772445177015e-05,
+      "loss": 0.1277,
+      "num_input_tokens_seen": 4442576,
+      "step": 170
+    },
+    {
+      "epoch": 6.110599078341014,
+      "grad_norm": 0.2176467776298523,
+      "learning_rate": 3.861597587537568e-05,
+      "loss": 0.121,
+      "num_input_tokens_seen": 4477568,
+      "step": 171
+    },
+    {
+      "epoch": 6.147465437788019,
+      "grad_norm": 0.21243643760681152,
+      "learning_rate": 3.8493766448787825e-05,
+      "loss": 0.1976,
+      "num_input_tokens_seen": 4509944,
+      "step": 172
+    },
+    {
+      "epoch": 6.184331797235023,
+      "grad_norm": 0.2905130386352539,
+      "learning_rate": 3.837110030834161e-05,
+      "loss": 0.141,
+      "num_input_tokens_seen": 4534704,
+      "step": 173
+    },
+    {
+      "epoch": 6.221198156682028,
+      "grad_norm": 0.2976662218570709,
+      "learning_rate": 3.824798160583012e-05,
+      "loss": 0.1389,
+      "num_input_tokens_seen": 4563912,
+      "step": 174
+    },
+    {
+      "epoch": 6.258064516129032,
+      "grad_norm": 0.2367183119058609,
+      "learning_rate": 3.8124414508364e-05,
+      "loss": 0.1241,
+      "num_input_tokens_seen": 4594712,
+      "step": 175
+    },
+    {
+      "epoch": 6.2949308755760365,
+      "grad_norm": 0.23760217428207397,
+      "learning_rate": 3.8000403198230387e-05,
+      "loss": 0.1415,
+      "num_input_tokens_seen": 4621448,
+      "step": 176
+    },
+    {
+      "epoch": 6.331797235023042,
+      "grad_norm": 0.23512595891952515,
+      "learning_rate": 3.787595187275136e-05,
+      "loss": 0.1277,
+      "num_input_tokens_seen": 4648744,
+      "step": 177
+    },
+    {
+      "epoch": 6.368663594470046,
+      "grad_norm": 0.24028612673282623,
+      "learning_rate": 3.775106474414188e-05,
+      "loss": 0.1636,
+      "num_input_tokens_seen": 4679480,
+      "step": 178
+    },
+    {
+      "epoch": 6.405529953917051,
+      "grad_norm": 0.2704714238643646,
+      "learning_rate": 3.762574603936725e-05,
+      "loss": 0.1341,
+      "num_input_tokens_seen": 4698440,
+      "step": 179
+    },
+    {
+      "epoch": 6.442396313364055,
+      "grad_norm": 0.27304500341415405,
+      "learning_rate": 3.7500000000000003e-05,
+      "loss": 0.1472,
+      "num_input_tokens_seen": 4721120,
+      "step": 180
+    },
+    {
+      "epoch": 6.47926267281106,
+      "grad_norm": 0.2529843747615814,
+      "learning_rate": 3.7373830882076354e-05,
+      "loss": 0.1344,
+      "num_input_tokens_seen": 4743536,
+      "step": 181
+    },
+    {
+      "epoch": 6.516129032258064,
+      "grad_norm": 0.23612789809703827,
+      "learning_rate": 3.7247242955952175e-05,
+      "loss": 0.1255,
+      "num_input_tokens_seen": 4773248,
+      "step": 182
+    },
+    {
+      "epoch": 6.552995391705069,
+      "grad_norm": 0.2793615162372589,
+      "learning_rate": 3.712024050615843e-05,
+      "loss": 0.141,
+      "num_input_tokens_seen": 4804600,
+      "step": 183
+    },
+    {
+      "epoch": 6.589861751152074,
+      "grad_norm": 0.29646047949790955,
+      "learning_rate": 3.699282783125616e-05,
+      "loss": 0.1226,
+      "num_input_tokens_seen": 4826744,
+      "step": 184
+    },
+    {
+      "epoch": 6.626728110599078,
+      "grad_norm": 0.25531846284866333,
+      "learning_rate": 3.686500924369101e-05,
+      "loss": 0.1074,
+      "num_input_tokens_seen": 4849584,
+      "step": 185
+    },
+    {
+      "epoch": 6.663594470046083,
+      "grad_norm": 0.24337461590766907,
+      "learning_rate": 3.673678906964727e-05,
+      "loss": 0.1084,
+      "num_input_tokens_seen": 4875336,
+      "step": 186
+    },
+    {
+      "epoch": 6.700460829493087,
+      "grad_norm": 0.30051150918006897,
+      "learning_rate": 3.660817164890143e-05,
+      "loss": 0.1273,
+      "num_input_tokens_seen": 4895896,
+      "step": 187
+    },
+    {
+      "epoch": 6.7373271889400925,
+      "grad_norm": 0.286495178937912,
+      "learning_rate": 3.6479161334675296e-05,
+      "loss": 0.1447,
+      "num_input_tokens_seen": 4924360,
+      "step": 188
+    },
+    {
+      "epoch": 6.774193548387097,
+      "grad_norm": 0.28501835465431213,
+      "learning_rate": 3.634976249348867e-05,
+      "loss": 0.1548,
+      "num_input_tokens_seen": 4948384,
+      "step": 189
+    },
+    {
+      "epoch": 6.811059907834101,
+      "grad_norm": 0.24231503903865814,
+      "learning_rate": 3.621997950501156e-05,
+      "loss": 0.1507,
+      "num_input_tokens_seen": 4980368,
+      "step": 190
+    },
+    {
+      "epoch": 6.847926267281106,
+      "grad_norm": 0.2752607464790344,
+      "learning_rate": 3.6089816761915906e-05,
+      "loss": 0.1282,
+      "num_input_tokens_seen": 5007744,
+      "step": 191
+    },
+    {
+      "epoch": 6.88479262672811,
+      "grad_norm": 0.27704674005508423,
+      "learning_rate": 3.5959278669726935e-05,
+      "loss": 0.1411,
+      "num_input_tokens_seen": 5030528,
+      "step": 192
+    },
+    {
+      "epoch": 6.921658986175116,
+      "grad_norm": 0.22601065039634705,
+      "learning_rate": 3.582836964667408e-05,
+      "loss": 0.1156,
+      "num_input_tokens_seen": 5065000,
+      "step": 193
+    },
+    {
+      "epoch": 6.95852534562212,
+      "grad_norm": 0.24917051196098328,
+      "learning_rate": 3.569709412354136e-05,
+      "loss": 0.1143,
+      "num_input_tokens_seen": 5091440,
+      "step": 194
+    },
+    {
+      "epoch": 6.9953917050691246,
+      "grad_norm": 0.31986546516418457,
+      "learning_rate": 3.556545654351749e-05,
+      "loss": 0.1466,
+      "num_input_tokens_seen": 5120064,
+      "step": 195
+    },
+    {
+      "epoch": 7.0,
+      "grad_norm": 0.6616023778915405,
+      "learning_rate": 3.543346136204545e-05,
+      "loss": 0.1637,
+      "num_input_tokens_seen": 5123832,
+      "step": 196
+    },
+    {
+      "epoch": 7.0368663594470044,
+      "grad_norm": 0.2567199766635895,
+      "learning_rate": 3.5301113046671714e-05,
+      "loss": 0.1111,
+      "num_input_tokens_seen": 5154136,
+      "step": 197
+    },
+    {
+      "epoch": 7.073732718894009,
+      "grad_norm": 0.24546091258525848,
+      "learning_rate": 3.516841607689501e-05,
+      "loss": 0.1015,
+      "num_input_tokens_seen": 5182792,
+      "step": 198
+    },
+    {
+      "epoch": 7.110599078341014,
+      "grad_norm": 0.25169137120246887,
+      "learning_rate": 3.503537494401473e-05,
+      "loss": 0.1091,
+      "num_input_tokens_seen": 5208280,
+      "step": 199
+    },
+    {
+      "epoch": 7.147465437788019,
+      "grad_norm": 0.27022022008895874,
+      "learning_rate": 3.490199415097892e-05,
+      "loss": 0.1165,
+      "num_input_tokens_seen": 5226024,
+      "step": 200
+    },
+    {
+      "epoch": 7.184331797235023,
+      "grad_norm": 0.2491355836391449,
+      "learning_rate": 3.476827821223184e-05,
+      "loss": 0.1047,
+      "num_input_tokens_seen": 5259096,
+      "step": 201
+    },
+    {
+      "epoch": 7.221198156682028,
+      "grad_norm": 0.28957781195640564,
+      "learning_rate": 3.463423165356121e-05,
+      "loss": 0.1066,
+      "num_input_tokens_seen": 5282232,
+      "step": 202
+    },
+    {
+      "epoch": 7.258064516129032,
+      "grad_norm": 0.29709964990615845,
+      "learning_rate": 3.449985901194498e-05,
+      "loss": 0.0902,
+      "num_input_tokens_seen": 5306272,
+      "step": 203
+    },
+    {
+      "epoch": 7.2949308755760365,
+      "grad_norm": 0.3285166919231415,
+      "learning_rate": 3.436516483539781e-05,
+      "loss": 0.1072,
+      "num_input_tokens_seen": 5330832,
+      "step": 204
+    },
+    {
+      "epoch": 7.331797235023042,
+      "grad_norm": 0.24284234642982483,
+      "learning_rate": 3.423015368281711e-05,
+      "loss": 0.1195,
+      "num_input_tokens_seen": 5359992,
+      "step": 205
+    },
+    {
+      "epoch": 7.368663594470046,
+      "grad_norm": 0.2531001567840576,
+      "learning_rate": 3.409483012382879e-05,
+      "loss": 0.1038,
+      "num_input_tokens_seen": 5392232,
+      "step": 206
+    },
+    {
+      "epoch": 7.405529953917051,
+      "grad_norm": 0.273573637008667,
+      "learning_rate": 3.39591987386325e-05,
+      "loss": 0.109,
+      "num_input_tokens_seen": 5416272,
+      "step": 207
+    },
+    {
+      "epoch": 7.442396313364055,
+      "grad_norm": 0.28594842553138733,
+      "learning_rate": 3.382326411784672e-05,
+      "loss": 0.1035,
+      "num_input_tokens_seen": 5442312,
+      "step": 208
+    },
+    {
+      "epoch": 7.47926267281106,
+      "grad_norm": 0.24845322966575623,
+      "learning_rate": 3.3687030862353286e-05,
+      "loss": 0.0997,
+      "num_input_tokens_seen": 5473408,
+      "step": 209
+    },
+    {
+      "epoch": 7.516129032258064,
+      "grad_norm": 0.2672761082649231,
+      "learning_rate": 3.355050358314172e-05,
+      "loss": 0.1034,
+      "num_input_tokens_seen": 5494256,
+      "step": 210
+    },
+    {
+      "epoch": 7.552995391705069,
+      "grad_norm": 0.29590561985969543,
+      "learning_rate": 3.3413686901153165e-05,
+      "loss": 0.0994,
+      "num_input_tokens_seen": 5519384,
+      "step": 211
+    },
+    {
+      "epoch": 7.589861751152074,
+      "grad_norm": 0.2702927887439728,
+      "learning_rate": 3.327658544712395e-05,
+      "loss": 0.1197,
+      "num_input_tokens_seen": 5544760,
+      "step": 212
+    },
+    {
+      "epoch": 7.626728110599078,
+      "grad_norm": 0.2904987633228302,
+      "learning_rate": 3.313920386142892e-05,
+      "loss": 0.1423,
+      "num_input_tokens_seen": 5574216,
+      "step": 213
+    },
+    {
+      "epoch": 7.663594470046083,
+      "grad_norm": 0.27985090017318726,
+      "learning_rate": 3.3001546793924285e-05,
+      "loss": 0.1171,
+      "num_input_tokens_seen": 5602072,
+      "step": 214
+    },
+    {
+      "epoch": 7.700460829493087,
+      "grad_norm": 0.25102686882019043,
+      "learning_rate": 3.2863618903790346e-05,
+      "loss": 0.1118,
+      "num_input_tokens_seen": 5632056,
+      "step": 215
+    },
+    {
+      "epoch": 7.7373271889400925,
+      "grad_norm": 0.3189446032047272,
+      "learning_rate": 3.272542485937369e-05,
+      "loss": 0.105,
+      "num_input_tokens_seen": 5658216,
+      "step": 216
+    },
+    {
+      "epoch": 7.774193548387097,
+      "grad_norm": 0.30590710043907166,
+      "learning_rate": 3.2586969338029274e-05,
+      "loss": 0.1083,
+      "num_input_tokens_seen": 5684712,
+      "step": 217
+    },
+    {
+      "epoch": 7.811059907834101,
+      "grad_norm": 0.3227143883705139,
+      "learning_rate": 3.244825702596205e-05,
+      "loss": 0.1477,
+      "num_input_tokens_seen": 5709472,
+      "step": 218
+    },
+    {
+      "epoch": 7.847926267281106,
+      "grad_norm": 0.25323161482810974,
+      "learning_rate": 3.230929261806842e-05,
+      "loss": 0.1057,
+      "num_input_tokens_seen": 5742728,
+      "step": 219
+    },
+    {
+      "epoch": 7.88479262672811,
+      "grad_norm": 0.27679508924484253,
+      "learning_rate": 3.217008081777726e-05,
+      "loss": 0.1094,
+      "num_input_tokens_seen": 5768848,
+      "step": 220
+    },
+    {
+      "epoch": 7.921658986175116,
+      "grad_norm": 0.3211810886859894,
+      "learning_rate": 3.203062633689077e-05,
+      "loss": 0.1092,
+      "num_input_tokens_seen": 5795008,
+      "step": 221
+    },
+    {
+      "epoch": 7.95852534562212,
+      "grad_norm": 0.3296988904476166,
+      "learning_rate": 3.1890933895424976e-05,
+      "loss": 0.1253,
+      "num_input_tokens_seen": 5818160,
+      "step": 222
+    },
+    {
+      "epoch": 7.9953917050691246,
+      "grad_norm": 0.2729380130767822,
+      "learning_rate": 3.1751008221450025e-05,
+      "loss": 0.1264,
+      "num_input_tokens_seen": 5850472,
+      "step": 223
+    },
+    {
+      "epoch": 8.0,
+      "grad_norm": 0.6995195150375366,
+      "learning_rate": 3.161085405093006e-05,
+      "loss": 0.1218,
+      "num_input_tokens_seen": 5855808,
+      "step": 224
+    },
+    {
+      "epoch": 8.036866359447005,
+      "grad_norm": 0.3297252357006073,
+      "learning_rate": 3.147047612756302e-05,
+      "loss": 0.0946,
+      "num_input_tokens_seen": 5884832,
+      "step": 225
+    },
+    {
+      "epoch": 8.073732718894009,
+      "grad_norm": 0.2687251567840576,
+      "learning_rate": 3.132987920262005e-05,
+      "loss": 0.0889,
+      "num_input_tokens_seen": 5903392,
+      "step": 226
+    },
+    {
+      "epoch": 8.110599078341014,
+      "grad_norm": 0.2826682925224304,
+      "learning_rate": 3.118906803478465e-05,
+      "loss": 0.1084,
+      "num_input_tokens_seen": 5928240,
+      "step": 227
+    },
+    {
+      "epoch": 8.147465437788018,
+      "grad_norm": 0.27045705914497375,
+      "learning_rate": 3.104804738999169e-05,
+      "loss": 0.1134,
+      "num_input_tokens_seen": 5958392,
+      "step": 228
+    },
+    {
+      "epoch": 8.184331797235023,
+      "grad_norm": 0.2565663754940033,
+      "learning_rate": 3.090682204126604e-05,
+      "loss": 0.0929,
+      "num_input_tokens_seen": 5984088,
+      "step": 229
+    },
+    {
+      "epoch": 8.221198156682028,
+      "grad_norm": 0.28110557794570923,
+      "learning_rate": 3.076539676856101e-05,
+      "loss": 0.0922,
+      "num_input_tokens_seen": 6007936,
+      "step": 230
+    },
+    {
+      "epoch": 8.258064516129032,
+      "grad_norm": 0.2717958092689514,
+      "learning_rate": 3.062377635859663e-05,
+      "loss": 0.072,
+      "num_input_tokens_seen": 6033336,
+      "step": 231
+    },
+    {
+      "epoch": 8.294930875576037,
+      "grad_norm": 0.3095947802066803,
+      "learning_rate": 3.048196560469758e-05,
+      "loss": 0.1176,
+      "num_input_tokens_seen": 6060160,
+      "step": 232
+    },
+    {
+      "epoch": 8.331797235023041,
+      "grad_norm": 0.3245174288749695,
+      "learning_rate": 3.0339969306631005e-05,
+      "loss": 0.0886,
+      "num_input_tokens_seen": 6088280,
+      "step": 233
+    },
+    {
+      "epoch": 8.368663594470046,
+      "grad_norm": 0.2761123776435852,
+      "learning_rate": 3.0197792270443982e-05,
+      "loss": 0.0831,
+      "num_input_tokens_seen": 6119752,
+      "step": 234
+    },
+    {
+      "epoch": 8.40552995391705,
+      "grad_norm": 0.33236628770828247,
+      "learning_rate": 3.0055439308300952e-05,
+      "loss": 0.1222,
+      "num_input_tokens_seen": 6154672,
+      "step": 235
+    },
+    {
+      "epoch": 8.442396313364055,
+      "grad_norm": 0.2539371848106384,
+      "learning_rate": 2.9912915238320754e-05,
+      "loss": 0.0856,
+      "num_input_tokens_seen": 6181152,
+      "step": 236
+    },
+    {
+      "epoch": 8.47926267281106,
+      "grad_norm": 0.25849923491477966,
+      "learning_rate": 2.9770224884413623e-05,
+      "loss": 0.0842,
+      "num_input_tokens_seen": 6208248,
+      "step": 237
+    },
+    {
+      "epoch": 8.516129032258064,
+      "grad_norm": 0.2990070879459381,
+      "learning_rate": 2.9627373076117863e-05,
+      "loss": 0.0816,
+      "num_input_tokens_seen": 6232704,
+      "step": 238
+    },
+    {
+      "epoch": 8.55299539170507,
+      "grad_norm": 0.2482941746711731,
+      "learning_rate": 2.9484364648436437e-05,
+      "loss": 0.0725,
+      "num_input_tokens_seen": 6264600,
+      "step": 239
+    },
+    {
+      "epoch": 8.589861751152073,
+      "grad_norm": 0.31631267070770264,
+      "learning_rate": 2.9341204441673266e-05,
+      "loss": 0.0917,
+      "num_input_tokens_seen": 6289416,
+      "step": 240
+    },
+    {
+      "epoch": 8.626728110599078,
+      "grad_norm": 0.32536354660987854,
+      "learning_rate": 2.9197897301269435e-05,
+      "loss": 0.1029,
+      "num_input_tokens_seen": 6313624,
+      "step": 241
+    },
+    {
+      "epoch": 8.663594470046084,
+      "grad_norm": 0.28533875942230225,
+      "learning_rate": 2.905444807763919e-05,
+      "loss": 0.0876,
+      "num_input_tokens_seen": 6337560,
+      "step": 242
+    },
+    {
+      "epoch": 8.700460829493087,
+      "grad_norm": 0.32342368364334106,
+      "learning_rate": 2.8910861626005776e-05,
+      "loss": 0.081,
+      "num_input_tokens_seen": 6362016,
+      "step": 243
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 540,
+  "num_input_tokens_seen": 6362016,
+  "num_train_epochs": 20,
+  "save_steps": 27,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.367103103382323e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-270/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-270/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-297/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-324/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-324/trainer_state.json

@@ -0,0 +1,2625 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 11.589861751152073,
+  "eval_steps": 500,
+  "global_step": 324,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03686635944700461,
+      "grad_norm": 0.06150835379958153,
+      "learning_rate": 4.999957692144361e-05,
+      "loss": 0.663,
+      "num_input_tokens_seen": 23408,
+      "step": 1
+    },
+    {
+      "epoch": 0.07373271889400922,
+      "grad_norm": 0.05766688659787178,
+      "learning_rate": 4.999830770009406e-05,
+      "loss": 0.5847,
+      "num_input_tokens_seen": 45520,
+      "step": 2
+    },
+    {
+      "epoch": 0.11059907834101383,
+      "grad_norm": 0.0780782401561737,
+      "learning_rate": 4.9996192378909786e-05,
+      "loss": 0.5545,
+      "num_input_tokens_seen": 68904,
+      "step": 3
+    },
+    {
+      "epoch": 0.14746543778801843,
+      "grad_norm": 0.07575194537639618,
+      "learning_rate": 4.9993231029486544e-05,
+      "loss": 0.7025,
+      "num_input_tokens_seen": 89696,
+      "step": 4
+    },
+    {
+      "epoch": 0.18433179723502305,
+      "grad_norm": 0.0708586797118187,
+      "learning_rate": 4.998942375205502e-05,
+      "loss": 0.6063,
+      "num_input_tokens_seen": 118440,
+      "step": 5
+    },
+    {
+      "epoch": 0.22119815668202766,
+      "grad_norm": 0.0794093981385231,
+      "learning_rate": 4.99847706754774e-05,
+      "loss": 0.593,
+      "num_input_tokens_seen": 145072,
+      "step": 6
+    },
+    {
+      "epoch": 0.25806451612903225,
+      "grad_norm": 0.04954744130373001,
+      "learning_rate": 4.997927195724303e-05,
+      "loss": 0.4857,
+      "num_input_tokens_seen": 182112,
+      "step": 7
+    },
+    {
+      "epoch": 0.29493087557603687,
+      "grad_norm": 0.06549246609210968,
+      "learning_rate": 4.997292778346312e-05,
+      "loss": 0.5393,
+      "num_input_tokens_seen": 212192,
+      "step": 8
+    },
+    {
+      "epoch": 0.3317972350230415,
+      "grad_norm": 0.07614441215991974,
+      "learning_rate": 4.996573836886435e-05,
+      "loss": 0.7044,
+      "num_input_tokens_seen": 233728,
+      "step": 9
+    },
+    {
+      "epoch": 0.3686635944700461,
+      "grad_norm": 0.07068340480327606,
+      "learning_rate": 4.995770395678171e-05,
+      "loss": 0.6086,
+      "num_input_tokens_seen": 259776,
+      "step": 10
+    },
+    {
+      "epoch": 0.4055299539170507,
+      "grad_norm": 0.07266585528850555,
+      "learning_rate": 4.9948824819150185e-05,
+      "loss": 0.6247,
+      "num_input_tokens_seen": 284816,
+      "step": 11
+    },
+    {
+      "epoch": 0.4423963133640553,
+      "grad_norm": 0.06145177036523819,
+      "learning_rate": 4.993910125649561e-05,
+      "loss": 0.4918,
+      "num_input_tokens_seen": 309776,
+      "step": 12
+    },
+    {
+      "epoch": 0.4792626728110599,
+      "grad_norm": 0.06720694154500961,
+      "learning_rate": 4.992853359792444e-05,
+      "loss": 0.5017,
+      "num_input_tokens_seen": 339064,
+      "step": 13
+    },
+    {
+      "epoch": 0.5161290322580645,
+      "grad_norm": 0.07309022545814514,
+      "learning_rate": 4.9917122201112656e-05,
+      "loss": 0.5447,
+      "num_input_tokens_seen": 369864,
+      "step": 14
+    },
+    {
+      "epoch": 0.5529953917050692,
+      "grad_norm": 0.0778132826089859,
+      "learning_rate": 4.990486745229364e-05,
+      "loss": 0.5034,
+      "num_input_tokens_seen": 397184,
+      "step": 15
+    },
+    {
+      "epoch": 0.5898617511520737,
+      "grad_norm": 0.07727856189012527,
+      "learning_rate": 4.989176976624511e-05,
+      "loss": 0.5277,
+      "num_input_tokens_seen": 426296,
+      "step": 16
+    },
+    {
+      "epoch": 0.6267281105990783,
+      "grad_norm": 0.0700518786907196,
+      "learning_rate": 4.987782958627508e-05,
+      "loss": 0.5356,
+      "num_input_tokens_seen": 459544,
+      "step": 17
+    },
+    {
+      "epoch": 0.663594470046083,
+      "grad_norm": 0.07351912558078766,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.5505,
+      "num_input_tokens_seen": 488616,
+      "step": 18
+    },
+    {
+      "epoch": 0.7004608294930875,
+      "grad_norm": 0.08268705755472183,
+      "learning_rate": 4.9847423660363e-05,
+      "loss": 0.5557,
+      "num_input_tokens_seen": 512176,
+      "step": 19
+    },
+    {
+      "epoch": 0.7373271889400922,
+      "grad_norm": 0.08081424236297607,
+      "learning_rate": 4.983095894354858e-05,
+      "loss": 0.4576,
+      "num_input_tokens_seen": 539456,
+      "step": 20
+    },
+    {
+      "epoch": 0.7741935483870968,
+      "grad_norm": 0.07748426496982574,
+      "learning_rate": 4.9813653791033057e-05,
+      "loss": 0.3918,
+      "num_input_tokens_seen": 561384,
+      "step": 21
+    },
+    {
+      "epoch": 0.8110599078341014,
+      "grad_norm": 0.10133849829435349,
+      "learning_rate": 4.979550878853154e-05,
+      "loss": 0.531,
+      "num_input_tokens_seen": 589792,
+      "step": 22
+    },
+    {
+      "epoch": 0.847926267281106,
+      "grad_norm": 0.09528470784425735,
+      "learning_rate": 4.9776524550184965e-05,
+      "loss": 0.4159,
+      "num_input_tokens_seen": 613944,
+      "step": 23
+    },
+    {
+      "epoch": 0.8847926267281107,
+      "grad_norm": 0.08829868584871292,
+      "learning_rate": 4.975670171853926e-05,
+      "loss": 0.403,
+      "num_input_tokens_seen": 641432,
+      "step": 24
+    },
+    {
+      "epoch": 0.9216589861751152,
+      "grad_norm": 0.10020875930786133,
+      "learning_rate": 4.973604096452361e-05,
+      "loss": 0.4238,
+      "num_input_tokens_seen": 665232,
+      "step": 25
+    },
+    {
+      "epoch": 0.9585253456221198,
+      "grad_norm": 0.1256554126739502,
+      "learning_rate": 4.971454298742779e-05,
+      "loss": 0.5033,
+      "num_input_tokens_seen": 687216,
+      "step": 26
+    },
+    {
+      "epoch": 0.9953917050691244,
+      "grad_norm": 0.17123407125473022,
+      "learning_rate": 4.9692208514878444e-05,
+      "loss": 0.3842,
+      "num_input_tokens_seen": 728880,
+      "step": 27
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.27383726835250854,
+      "learning_rate": 4.966903830281449e-05,
+      "loss": 0.4988,
+      "num_input_tokens_seen": 731976,
+      "step": 28
+    },
+    {
+      "epoch": 1.0368663594470047,
+      "grad_norm": 0.148647278547287,
+      "learning_rate": 4.9645033135461494e-05,
+      "loss": 0.4489,
+      "num_input_tokens_seen": 756184,
+      "step": 29
+    },
+    {
+      "epoch": 1.0737327188940091,
+      "grad_norm": 0.11988866329193115,
+      "learning_rate": 4.962019382530521e-05,
+      "loss": 0.4999,
+      "num_input_tokens_seen": 779072,
+      "step": 30
+    },
+    {
+      "epoch": 1.1105990783410138,
+      "grad_norm": 0.10590679943561554,
+      "learning_rate": 4.9594521213063974e-05,
+      "loss": 0.3711,
+      "num_input_tokens_seen": 805672,
+      "step": 31
+    },
+    {
+      "epoch": 1.1474654377880185,
+      "grad_norm": 0.11934591829776764,
+      "learning_rate": 4.9568016167660334e-05,
+      "loss": 0.3385,
+      "num_input_tokens_seen": 828840,
+      "step": 32
+    },
+    {
+      "epoch": 1.1843317972350231,
+      "grad_norm": 0.11900116503238678,
+      "learning_rate": 4.9540679586191605e-05,
+      "loss": 0.3812,
+      "num_input_tokens_seen": 850792,
+      "step": 33
+    },
+    {
+      "epoch": 1.2211981566820276,
+      "grad_norm": 0.11279546469449997,
+      "learning_rate": 4.951251239389948e-05,
+      "loss": 0.2924,
+      "num_input_tokens_seen": 885608,
+      "step": 34
+    },
+    {
+      "epoch": 1.2580645161290323,
+      "grad_norm": 0.08340126276016235,
+      "learning_rate": 4.948351554413879e-05,
+      "loss": 0.3489,
+      "num_input_tokens_seen": 921912,
+      "step": 35
+    },
+    {
+      "epoch": 1.294930875576037,
+      "grad_norm": 0.11327336728572845,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.2921,
+      "num_input_tokens_seen": 945672,
+      "step": 36
+    },
+    {
+      "epoch": 1.3317972350230414,
+      "grad_norm": 0.10909265279769897,
+      "learning_rate": 4.942303682600178e-05,
+      "loss": 0.3421,
+      "num_input_tokens_seen": 970824,
+      "step": 37
+    },
+    {
+      "epoch": 1.368663594470046,
+      "grad_norm": 0.11239592730998993,
+      "learning_rate": 4.939155700460536e-05,
+      "loss": 0.3848,
+      "num_input_tokens_seen": 995840,
+      "step": 38
+    },
+    {
+      "epoch": 1.4055299539170507,
+      "grad_norm": 0.1034158244729042,
+      "learning_rate": 4.9359251619630886e-05,
+      "loss": 0.3434,
+      "num_input_tokens_seen": 1035880,
+      "step": 39
+    },
+    {
+      "epoch": 1.4423963133640554,
+      "grad_norm": 0.12086673080921173,
+      "learning_rate": 4.9326121764495596e-05,
+      "loss": 0.3588,
+      "num_input_tokens_seen": 1056488,
+      "step": 40
+    },
+    {
+      "epoch": 1.4792626728110598,
+      "grad_norm": 0.11408794671297073,
+      "learning_rate": 4.9292168560522014e-05,
+      "loss": 0.4021,
+      "num_input_tokens_seen": 1080080,
+      "step": 41
+    },
+    {
+      "epoch": 1.5161290322580645,
+      "grad_norm": 0.1417722851037979,
+      "learning_rate": 4.925739315689991e-05,
+      "loss": 0.3343,
+      "num_input_tokens_seen": 1095632,
+      "step": 42
+    },
+    {
+      "epoch": 1.5529953917050692,
+      "grad_norm": 0.2315663844347,
+      "learning_rate": 4.9221796730647516e-05,
+      "loss": 0.4025,
+      "num_input_tokens_seen": 1126040,
+      "step": 43
+    },
+    {
+      "epoch": 1.5898617511520738,
+      "grad_norm": 0.07856712490320206,
+      "learning_rate": 4.9185380486571595e-05,
+      "loss": 0.3337,
+      "num_input_tokens_seen": 1156656,
+      "step": 44
+    },
+    {
+      "epoch": 1.6267281105990783,
+      "grad_norm": 0.10064279288053513,
+      "learning_rate": 4.914814565722671e-05,
+      "loss": 0.3758,
+      "num_input_tokens_seen": 1183400,
+      "step": 45
+    },
+    {
+      "epoch": 1.663594470046083,
+      "grad_norm": 0.106510229408741,
+      "learning_rate": 4.9110093502873476e-05,
+      "loss": 0.3363,
+      "num_input_tokens_seen": 1209680,
+      "step": 46
+    },
+    {
+      "epoch": 1.7004608294930876,
+      "grad_norm": 0.10446175187826157,
+      "learning_rate": 4.907122531143594e-05,
+      "loss": 0.3853,
+      "num_input_tokens_seen": 1237864,
+      "step": 47
+    },
+    {
+      "epoch": 1.737327188940092,
+      "grad_norm": 0.2570022940635681,
+      "learning_rate": 4.9031542398457974e-05,
+      "loss": 0.3194,
+      "num_input_tokens_seen": 1273080,
+      "step": 48
+    },
+    {
+      "epoch": 1.7741935483870968,
+      "grad_norm": 0.0968039259314537,
+      "learning_rate": 4.8991046107058735e-05,
+      "loss": 0.2955,
+      "num_input_tokens_seen": 1300256,
+      "step": 49
+    },
+    {
+      "epoch": 1.8110599078341014,
+      "grad_norm": 0.11750692129135132,
+      "learning_rate": 4.894973780788722e-05,
+      "loss": 0.3314,
+      "num_input_tokens_seen": 1322616,
+      "step": 50
+    },
+    {
+      "epoch": 1.8479262672811059,
+      "grad_norm": 0.09568070620298386,
+      "learning_rate": 4.890761889907589e-05,
+      "loss": 0.3341,
+      "num_input_tokens_seen": 1353552,
+      "step": 51
+    },
+    {
+      "epoch": 1.8847926267281108,
+      "grad_norm": 0.11816362291574478,
+      "learning_rate": 4.88646908061933e-05,
+      "loss": 0.3202,
+      "num_input_tokens_seen": 1376456,
+      "step": 52
+    },
+    {
+      "epoch": 1.9216589861751152,
+      "grad_norm": 0.133891761302948,
+      "learning_rate": 4.8820954982195905e-05,
+      "loss": 0.2852,
+      "num_input_tokens_seen": 1398344,
+      "step": 53
+    },
+    {
+      "epoch": 1.9585253456221197,
+      "grad_norm": 0.11985364556312561,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.3413,
+      "num_input_tokens_seen": 1424264,
+      "step": 54
+    },
+    {
+      "epoch": 1.9953917050691246,
+      "grad_norm": 0.09828098118305206,
+      "learning_rate": 4.873106608932585e-05,
+      "loss": 0.2726,
+      "num_input_tokens_seen": 1460688,
+      "step": 55
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.31861889362335205,
+      "learning_rate": 4.868491606285823e-05,
+      "loss": 0.3512,
+      "num_input_tokens_seen": 1463952,
+      "step": 56
+    },
+    {
+      "epoch": 2.0368663594470044,
+      "grad_norm": 0.11089354753494263,
+      "learning_rate": 4.8637964389982926e-05,
+      "loss": 0.2953,
+      "num_input_tokens_seen": 1492600,
+      "step": 57
+    },
+    {
+      "epoch": 2.0737327188940093,
+      "grad_norm": 0.12034712731838226,
+      "learning_rate": 4.859021265983959e-05,
+      "loss": 0.2875,
+      "num_input_tokens_seen": 1514496,
+      "step": 58
+    },
+    {
+      "epoch": 2.110599078341014,
+      "grad_norm": 0.1372830867767334,
+      "learning_rate": 4.854166248864689e-05,
+      "loss": 0.3535,
+      "num_input_tokens_seen": 1541656,
+      "step": 59
+    },
+    {
+      "epoch": 2.1474654377880182,
+      "grad_norm": 0.10605650395154953,
+      "learning_rate": 4.849231551964771e-05,
+      "loss": 0.3013,
+      "num_input_tokens_seen": 1569672,
+      "step": 60
+    },
+    {
+      "epoch": 2.184331797235023,
+      "grad_norm": 0.12524856626987457,
+      "learning_rate": 4.844217342305363e-05,
+      "loss": 0.2558,
+      "num_input_tokens_seen": 1599304,
+      "step": 61
+    },
+    {
+      "epoch": 2.2211981566820276,
+      "grad_norm": 0.1151009052991867,
+      "learning_rate": 4.839123789598829e-05,
+      "loss": 0.2666,
+      "num_input_tokens_seen": 1628288,
+      "step": 62
+    },
+    {
+      "epoch": 2.258064516129032,
+      "grad_norm": 0.10297228395938873,
+      "learning_rate": 4.8339510662430046e-05,
+      "loss": 0.2982,
+      "num_input_tokens_seen": 1656384,
+      "step": 63
+    },
+    {
+      "epoch": 2.294930875576037,
+      "grad_norm": 0.11971966177225113,
+      "learning_rate": 4.828699347315356e-05,
+      "loss": 0.3746,
+      "num_input_tokens_seen": 1679344,
+      "step": 64
+    },
+    {
+      "epoch": 2.3317972350230414,
+      "grad_norm": 0.12384694814682007,
+      "learning_rate": 4.823368810567056e-05,
+      "loss": 0.2605,
+      "num_input_tokens_seen": 1703904,
+      "step": 65
+    },
+    {
+      "epoch": 2.3686635944700463,
+      "grad_norm": 0.14720793068408966,
+      "learning_rate": 4.817959636416969e-05,
+      "loss": 0.2816,
+      "num_input_tokens_seen": 1722360,
+      "step": 66
+    },
+    {
+      "epoch": 2.4055299539170507,
+      "grad_norm": 0.10553745925426483,
+      "learning_rate": 4.81247200794554e-05,
+      "loss": 0.2502,
+      "num_input_tokens_seen": 1752464,
+      "step": 67
+    },
+    {
+      "epoch": 2.442396313364055,
+      "grad_norm": 0.11162563413381577,
+      "learning_rate": 4.806906110888606e-05,
+      "loss": 0.2286,
+      "num_input_tokens_seen": 1782744,
+      "step": 68
+    },
+    {
+      "epoch": 2.47926267281106,
+      "grad_norm": 0.12341686338186264,
+      "learning_rate": 4.8012621336311016e-05,
+      "loss": 0.299,
+      "num_input_tokens_seen": 1815120,
+      "step": 69
+    },
+    {
+      "epoch": 2.5161290322580645,
+      "grad_norm": 0.12033814936876297,
+      "learning_rate": 4.7955402672006854e-05,
+      "loss": 0.2609,
+      "num_input_tokens_seen": 1840408,
+      "step": 70
+    },
+    {
+      "epoch": 2.5529953917050694,
+      "grad_norm": 0.11412363499403,
+      "learning_rate": 4.789740705261278e-05,
+      "loss": 0.3316,
+      "num_input_tokens_seen": 1873000,
+      "step": 71
+    },
+    {
+      "epoch": 2.589861751152074,
+      "grad_norm": 0.18193566799163818,
+      "learning_rate": 4.783863644106502e-05,
+      "loss": 0.2773,
+      "num_input_tokens_seen": 1900920,
+      "step": 72
+    },
+    {
+      "epoch": 2.6267281105990783,
+      "grad_norm": 0.14569184184074402,
+      "learning_rate": 4.777909282653042e-05,
+      "loss": 0.2644,
+      "num_input_tokens_seen": 1919272,
+      "step": 73
+    },
+    {
+      "epoch": 2.6635944700460827,
+      "grad_norm": 0.12456360459327698,
+      "learning_rate": 4.771877822433911e-05,
+      "loss": 0.2974,
+      "num_input_tokens_seen": 1959152,
+      "step": 74
+    },
+    {
+      "epoch": 2.7004608294930876,
+      "grad_norm": 0.1362839788198471,
+      "learning_rate": 4.765769467591625e-05,
+      "loss": 0.2743,
+      "num_input_tokens_seen": 1982960,
+      "step": 75
+    },
+    {
+      "epoch": 2.737327188940092,
+      "grad_norm": 0.15120629966259003,
+      "learning_rate": 4.759584424871302e-05,
+      "loss": 0.2425,
+      "num_input_tokens_seen": 2006736,
+      "step": 76
+    },
+    {
+      "epoch": 2.774193548387097,
+      "grad_norm": 0.15145516395568848,
+      "learning_rate": 4.7533229036136553e-05,
+      "loss": 0.2723,
+      "num_input_tokens_seen": 2031536,
+      "step": 77
+    },
+    {
+      "epoch": 2.8110599078341014,
+      "grad_norm": 0.1607382595539093,
+      "learning_rate": 4.7469851157479177e-05,
+      "loss": 0.2525,
+      "num_input_tokens_seen": 2053168,
+      "step": 78
+    },
+    {
+      "epoch": 2.847926267281106,
+      "grad_norm": 0.14976347982883453,
+      "learning_rate": 4.740571275784659e-05,
+      "loss": 0.2792,
+      "num_input_tokens_seen": 2078208,
+      "step": 79
+    },
+    {
+      "epoch": 2.8847926267281108,
+      "grad_norm": 0.12997782230377197,
+      "learning_rate": 4.734081600808531e-05,
+      "loss": 0.291,
+      "num_input_tokens_seen": 2101560,
+      "step": 80
+    },
+    {
+      "epoch": 2.921658986175115,
+      "grad_norm": 0.1369074434041977,
+      "learning_rate": 4.72751631047092e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2127824,
+      "step": 81
+    },
+    {
+      "epoch": 2.9585253456221197,
+      "grad_norm": 0.11734048277139664,
+      "learning_rate": 4.7208756269825104e-05,
+      "loss": 0.2442,
+      "num_input_tokens_seen": 2171536,
+      "step": 82
+    },
+    {
+      "epoch": 2.9953917050691246,
+      "grad_norm": 0.14909496903419495,
+      "learning_rate": 4.714159775105765e-05,
+      "loss": 0.2344,
+      "num_input_tokens_seen": 2193016,
+      "step": 83
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.40910854935646057,
+      "learning_rate": 4.707368982147318e-05,
+      "loss": 0.2645,
+      "num_input_tokens_seen": 2195928,
+      "step": 84
+    },
+    {
+      "epoch": 3.0368663594470044,
+      "grad_norm": 0.1433151513338089,
+      "learning_rate": 4.700503477950278e-05,
+      "loss": 0.2185,
+      "num_input_tokens_seen": 2219472,
+      "step": 85
+    },
+    {
+      "epoch": 3.0737327188940093,
+      "grad_norm": 0.13229826092720032,
+      "learning_rate": 4.693563494886455e-05,
+      "loss": 0.2364,
+      "num_input_tokens_seen": 2246504,
+      "step": 86
+    },
+    {
+      "epoch": 3.110599078341014,
+      "grad_norm": 0.1702108383178711,
+      "learning_rate": 4.6865492678484895e-05,
+      "loss": 0.2703,
+      "num_input_tokens_seen": 2270304,
+      "step": 87
+    },
+    {
+      "epoch": 3.1474654377880182,
+      "grad_norm": 0.1411813497543335,
+      "learning_rate": 4.679461034241906e-05,
+      "loss": 0.2807,
+      "num_input_tokens_seen": 2299968,
+      "step": 88
+    },
+    {
+      "epoch": 3.184331797235023,
+      "grad_norm": 0.12941044569015503,
+      "learning_rate": 4.672299033977076e-05,
+      "loss": 0.2182,
+      "num_input_tokens_seen": 2333432,
+      "step": 89
+    },
+    {
+      "epoch": 3.2211981566820276,
+      "grad_norm": 0.1649884432554245,
+      "learning_rate": 4.665063509461097e-05,
+      "loss": 0.2572,
+      "num_input_tokens_seen": 2355320,
+      "step": 90
+    },
+    {
+      "epoch": 3.258064516129032,
+      "grad_norm": 0.14386451244354248,
+      "learning_rate": 4.657754705589591e-05,
+      "loss": 0.2412,
+      "num_input_tokens_seen": 2378416,
+      "step": 91
+    },
+    {
+      "epoch": 3.294930875576037,
+      "grad_norm": 0.15350477397441864,
+      "learning_rate": 4.650372869738414e-05,
+      "loss": 0.242,
+      "num_input_tokens_seen": 2405432,
+      "step": 92
+    },
+    {
+      "epoch": 3.3317972350230414,
+      "grad_norm": 0.16243231296539307,
+      "learning_rate": 4.642918251755281e-05,
+      "loss": 0.2436,
+      "num_input_tokens_seen": 2429992,
+      "step": 93
+    },
+    {
+      "epoch": 3.3686635944700463,
+      "grad_norm": 0.18401454389095306,
+      "learning_rate": 4.6353911039513145e-05,
+      "loss": 0.2538,
+      "num_input_tokens_seen": 2456264,
+      "step": 94
+    },
+    {
+      "epoch": 3.4055299539170507,
+      "grad_norm": 0.13898858428001404,
+      "learning_rate": 4.627791681092499e-05,
+      "loss": 0.2065,
+      "num_input_tokens_seen": 2486032,
+      "step": 95
+    },
+    {
+      "epoch": 3.442396313364055,
+      "grad_norm": 0.16943225264549255,
+      "learning_rate": 4.620120240391065e-05,
+      "loss": 0.2086,
+      "num_input_tokens_seen": 2509272,
+      "step": 96
+    },
+    {
+      "epoch": 3.47926267281106,
+      "grad_norm": 0.15286582708358765,
+      "learning_rate": 4.612377041496776e-05,
+      "loss": 0.2517,
+      "num_input_tokens_seen": 2538392,
+      "step": 97
+    },
+    {
+      "epoch": 3.5161290322580645,
+      "grad_norm": 0.17737072706222534,
+      "learning_rate": 4.604562346488144e-05,
+      "loss": 0.2413,
+      "num_input_tokens_seen": 2562624,
+      "step": 98
+    },
+    {
+      "epoch": 3.5529953917050694,
+      "grad_norm": 0.13630996644496918,
+      "learning_rate": 4.5966764198635606e-05,
+      "loss": 0.2157,
+      "num_input_tokens_seen": 2595448,
+      "step": 99
+    },
+    {
+      "epoch": 3.589861751152074,
+      "grad_norm": 0.13918158411979675,
+      "learning_rate": 4.588719528532342e-05,
+      "loss": 0.2319,
+      "num_input_tokens_seen": 2633568,
+      "step": 100
+    },
+    {
+      "epoch": 3.6267281105990783,
+      "grad_norm": 0.16548505425453186,
+      "learning_rate": 4.580691941805695e-05,
+      "loss": 0.2473,
+      "num_input_tokens_seen": 2659408,
+      "step": 101
+    },
+    {
+      "epoch": 3.6635944700460827,
+      "grad_norm": 0.17397943139076233,
+      "learning_rate": 4.572593931387604e-05,
+      "loss": 0.2527,
+      "num_input_tokens_seen": 2687272,
+      "step": 102
+    },
+    {
+      "epoch": 3.7004608294930876,
+      "grad_norm": 0.15970823168754578,
+      "learning_rate": 4.5644257713656356e-05,
+      "loss": 0.22,
+      "num_input_tokens_seen": 2718232,
+      "step": 103
+    },
+    {
+      "epoch": 3.737327188940092,
+      "grad_norm": 0.16440241038799286,
+      "learning_rate": 4.556187738201656e-05,
+      "loss": 0.2374,
+      "num_input_tokens_seen": 2743336,
+      "step": 104
+    },
+    {
+      "epoch": 3.774193548387097,
+      "grad_norm": 0.15761925280094147,
+      "learning_rate": 4.54788011072248e-05,
+      "loss": 0.2023,
+      "num_input_tokens_seen": 2771384,
+      "step": 105
+    },
+    {
+      "epoch": 3.8110599078341014,
+      "grad_norm": 0.1528206616640091,
+      "learning_rate": 4.539503170110431e-05,
+      "loss": 0.2289,
+      "num_input_tokens_seen": 2797488,
+      "step": 106
+    },
+    {
+      "epoch": 3.847926267281106,
+      "grad_norm": 0.16380636394023895,
+      "learning_rate": 4.531057199893824e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2825408,
+      "step": 107
+    },
+    {
+      "epoch": 3.8847926267281108,
+      "grad_norm": 0.19076082110404968,
+      "learning_rate": 4.522542485937369e-05,
+      "loss": 0.2467,
+      "num_input_tokens_seen": 2848856,
+      "step": 108
+    },
+    {
+      "epoch": 3.921658986175115,
+      "grad_norm": 0.1884787231683731,
+      "learning_rate": 4.5139593164324986e-05,
+      "loss": 0.2069,
+      "num_input_tokens_seen": 2867536,
+      "step": 109
+    },
+    {
+      "epoch": 3.9585253456221197,
+      "grad_norm": 0.19939164817333221,
+      "learning_rate": 4.50530798188761e-05,
+      "loss": 0.1843,
+      "num_input_tokens_seen": 2892032,
+      "step": 110
+    },
+    {
+      "epoch": 3.9953917050691246,
+      "grad_norm": 0.17019645869731903,
+      "learning_rate": 4.496588775118232e-05,
+      "loss": 0.2576,
+      "num_input_tokens_seen": 2924800,
+      "step": 111
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.41787323355674744,
+      "learning_rate": 4.48780199123712e-05,
+      "loss": 0.2734,
+      "num_input_tokens_seen": 2927904,
+      "step": 112
+    },
+    {
+      "epoch": 4.0368663594470044,
+      "grad_norm": 0.18536736071109772,
+      "learning_rate": 4.478947927644258e-05,
+      "loss": 0.2363,
+      "num_input_tokens_seen": 2954728,
+      "step": 113
+    },
+    {
+      "epoch": 4.073732718894009,
+      "grad_norm": 0.16922569274902344,
+      "learning_rate": 4.4700268840168045e-05,
+      "loss": 0.1888,
+      "num_input_tokens_seen": 2983656,
+      "step": 114
+    },
+    {
+      "epoch": 4.110599078341014,
+      "grad_norm": 0.17613869905471802,
+      "learning_rate": 4.4610391622989396e-05,
+      "loss": 0.2077,
+      "num_input_tokens_seen": 3014208,
+      "step": 115
+    },
+    {
+      "epoch": 4.147465437788019,
+      "grad_norm": 0.16198807954788208,
+      "learning_rate": 4.4519850666916484e-05,
+      "loss": 0.1858,
+      "num_input_tokens_seen": 3040976,
+      "step": 116
+    },
+    {
+      "epoch": 4.184331797235023,
+      "grad_norm": 0.19020891189575195,
+      "learning_rate": 4.442864903642428e-05,
+      "loss": 0.2284,
+      "num_input_tokens_seen": 3066744,
+      "step": 117
+    },
+    {
+      "epoch": 4.221198156682028,
+      "grad_norm": 0.21541331708431244,
+      "learning_rate": 4.43367898183491e-05,
+      "loss": 0.1856,
+      "num_input_tokens_seen": 3090320,
+      "step": 118
+    },
+    {
+      "epoch": 4.258064516129032,
+      "grad_norm": 0.19018961489200592,
+      "learning_rate": 4.4244276121784195e-05,
+      "loss": 0.1788,
+      "num_input_tokens_seen": 3114576,
+      "step": 119
+    },
+    {
+      "epoch": 4.2949308755760365,
+      "grad_norm": 0.24048063158988953,
+      "learning_rate": 4.415111107797445e-05,
+      "loss": 0.2155,
+      "num_input_tokens_seen": 3135552,
+      "step": 120
+    },
+    {
+      "epoch": 4.331797235023042,
+      "grad_norm": 0.17166386544704437,
+      "learning_rate": 4.405729784021046e-05,
+      "loss": 0.1831,
+      "num_input_tokens_seen": 3165168,
+      "step": 121
+    },
+    {
+      "epoch": 4.368663594470046,
+      "grad_norm": 0.18577668070793152,
+      "learning_rate": 4.396283958372173e-05,
+      "loss": 0.2168,
+      "num_input_tokens_seen": 3200240,
+      "step": 122
+    },
+    {
+      "epoch": 4.405529953917051,
+      "grad_norm": 0.20443667471408844,
+      "learning_rate": 4.386773950556931e-05,
+      "loss": 0.1978,
+      "num_input_tokens_seen": 3225024,
+      "step": 123
+    },
+    {
+      "epoch": 4.442396313364055,
+      "grad_norm": 0.17636562883853912,
+      "learning_rate": 4.377200082453749e-05,
+      "loss": 0.1727,
+      "num_input_tokens_seen": 3251696,
+      "step": 124
+    },
+    {
+      "epoch": 4.47926267281106,
+      "grad_norm": 0.2019859254360199,
+      "learning_rate": 4.36756267810249e-05,
+      "loss": 0.198,
+      "num_input_tokens_seen": 3275672,
+      "step": 125
+    },
+    {
+      "epoch": 4.516129032258064,
+      "grad_norm": 0.2633707523345947,
+      "learning_rate": 4.357862063693486e-05,
+      "loss": 0.2254,
+      "num_input_tokens_seen": 3305592,
+      "step": 126
+    },
+    {
+      "epoch": 4.552995391705069,
+      "grad_norm": 0.17379336059093475,
+      "learning_rate": 4.34809856755649e-05,
+      "loss": 0.2467,
+      "num_input_tokens_seen": 3345072,
+      "step": 127
+    },
+    {
+      "epoch": 4.589861751152074,
+      "grad_norm": 0.16954895853996277,
+      "learning_rate": 4.3382725201495723e-05,
+      "loss": 0.1616,
+      "num_input_tokens_seen": 3371832,
+      "step": 128
+    },
+    {
+      "epoch": 4.626728110599078,
+      "grad_norm": 0.21207046508789062,
+      "learning_rate": 4.3283842540479264e-05,
+      "loss": 0.189,
+      "num_input_tokens_seen": 3395912,
+      "step": 129
+    },
+    {
+      "epoch": 4.663594470046083,
+      "grad_norm": 0.40675088763237,
+      "learning_rate": 4.318434103932622e-05,
+      "loss": 0.1739,
+      "num_input_tokens_seen": 3423352,
+      "step": 130
+    },
+    {
+      "epoch": 4.700460829493087,
+      "grad_norm": 0.21301084756851196,
+      "learning_rate": 4.30842240657927e-05,
+      "loss": 0.187,
+      "num_input_tokens_seen": 3447992,
+      "step": 131
+    },
+    {
+      "epoch": 4.7373271889400925,
+      "grad_norm": 0.20403216779232025,
+      "learning_rate": 4.2983495008466276e-05,
+      "loss": 0.1786,
+      "num_input_tokens_seen": 3479752,
+      "step": 132
+    },
+    {
+      "epoch": 4.774193548387097,
+      "grad_norm": 0.1969008892774582,
+      "learning_rate": 4.288215727665129e-05,
+      "loss": 0.214,
+      "num_input_tokens_seen": 3509072,
+      "step": 133
+    },
+    {
+      "epoch": 4.811059907834101,
+      "grad_norm": 0.20199979841709137,
+      "learning_rate": 4.278021430025343e-05,
+      "loss": 0.171,
+      "num_input_tokens_seen": 3531848,
+      "step": 134
+    },
+    {
+      "epoch": 4.847926267281106,
+      "grad_norm": 0.2074364721775055,
+      "learning_rate": 4.267766952966369e-05,
+      "loss": 0.1786,
+      "num_input_tokens_seen": 3557144,
+      "step": 135
+    },
+    {
+      "epoch": 4.88479262672811,
+      "grad_norm": 0.23071159422397614,
+      "learning_rate": 4.257452643564155e-05,
+      "loss": 0.1816,
+      "num_input_tokens_seen": 3578656,
+      "step": 136
+    },
+    {
+      "epoch": 4.921658986175116,
+      "grad_norm": 0.2532529830932617,
+      "learning_rate": 4.24707885091975e-05,
+      "loss": 0.1885,
+      "num_input_tokens_seen": 3608632,
+      "step": 137
+    },
+    {
+      "epoch": 4.95852534562212,
+      "grad_norm": 0.24982504546642303,
+      "learning_rate": 4.2366459261474933e-05,
+      "loss": 0.2125,
+      "num_input_tokens_seen": 3628672,
+      "step": 138
+    },
+    {
+      "epoch": 4.9953917050691246,
+      "grad_norm": 0.17841289937496185,
+      "learning_rate": 4.226154222363124e-05,
+      "loss": 0.2001,
+      "num_input_tokens_seen": 3657088,
+      "step": 139
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.6343802809715271,
+      "learning_rate": 4.215604094671835e-05,
+      "loss": 0.2376,
+      "num_input_tokens_seen": 3659880,
+      "step": 140
+    },
+    {
+      "epoch": 5.0368663594470044,
+      "grad_norm": 0.21179749071598053,
+      "learning_rate": 4.2049959001562464e-05,
+      "loss": 0.1942,
+      "num_input_tokens_seen": 3684760,
+      "step": 141
+    },
+    {
+      "epoch": 5.073732718894009,
+      "grad_norm": 0.22715945541858673,
+      "learning_rate": 4.194329997864331e-05,
+      "loss": 0.1849,
+      "num_input_tokens_seen": 3715176,
+      "step": 142
+    },
+    {
+      "epoch": 5.110599078341014,
+      "grad_norm": 0.20638404786586761,
+      "learning_rate": 4.183606748797251e-05,
+      "loss": 0.1492,
+      "num_input_tokens_seen": 3741480,
+      "step": 143
+    },
+    {
+      "epoch": 5.147465437788019,
+      "grad_norm": 0.25307852029800415,
+      "learning_rate": 4.172826515897146e-05,
+      "loss": 0.1699,
+      "num_input_tokens_seen": 3762976,
+      "step": 144
+    },
+    {
+      "epoch": 5.184331797235023,
+      "grad_norm": 0.21923506259918213,
+      "learning_rate": 4.1619896640348445e-05,
+      "loss": 0.1559,
+      "num_input_tokens_seen": 3794720,
+      "step": 145
+    },
+    {
+      "epoch": 5.221198156682028,
+      "grad_norm": 0.2600756883621216,
+      "learning_rate": 4.1510965599975196e-05,
+      "loss": 0.1843,
+      "num_input_tokens_seen": 3822744,
+      "step": 146
+    },
+    {
+      "epoch": 5.258064516129032,
+      "grad_norm": 0.2997046113014221,
+      "learning_rate": 4.140147572476268e-05,
+      "loss": 0.243,
+      "num_input_tokens_seen": 3845584,
+      "step": 147
+    },
+    {
+      "epoch": 5.2949308755760365,
+      "grad_norm": 0.2147158831357956,
+      "learning_rate": 4.129143072053638e-05,
+      "loss": 0.1476,
+      "num_input_tokens_seen": 3869184,
+      "step": 148
+    },
+    {
+      "epoch": 5.331797235023042,
+      "grad_norm": 0.26199281215667725,
+      "learning_rate": 4.118083431191081e-05,
+      "loss": 0.1619,
+      "num_input_tokens_seen": 3900384,
+      "step": 149
+    },
+    {
+      "epoch": 5.368663594470046,
+      "grad_norm": 0.4091247618198395,
+      "learning_rate": 4.1069690242163484e-05,
+      "loss": 0.1943,
+      "num_input_tokens_seen": 3923792,
+      "step": 150
+    },
+    {
+      "epoch": 5.405529953917051,
+      "grad_norm": 0.23406103253364563,
+      "learning_rate": 4.095800227310821e-05,
+      "loss": 0.1403,
+      "num_input_tokens_seen": 3942784,
+      "step": 151
+    },
+    {
+      "epoch": 5.442396313364055,
+      "grad_norm": 0.2293683886528015,
+      "learning_rate": 4.0845774184967754e-05,
+      "loss": 0.1756,
+      "num_input_tokens_seen": 3970880,
+      "step": 152
+    },
+    {
+      "epoch": 5.47926267281106,
+      "grad_norm": 0.1963953971862793,
+      "learning_rate": 4.073300977624594e-05,
+      "loss": 0.1785,
+      "num_input_tokens_seen": 3998976,
+      "step": 153
+    },
+    {
+      "epoch": 5.516129032258064,
+      "grad_norm": 0.23071235418319702,
+      "learning_rate": 4.0619712863599e-05,
+      "loss": 0.165,
+      "num_input_tokens_seen": 4020272,
+      "step": 154
+    },
+    {
+      "epoch": 5.552995391705069,
+      "grad_norm": 0.2570018172264099,
+      "learning_rate": 4.05058872817065e-05,
+      "loss": 0.1639,
+      "num_input_tokens_seen": 4043288,
+      "step": 155
+    },
+    {
+      "epoch": 5.589861751152074,
+      "grad_norm": 0.2581011950969696,
+      "learning_rate": 4.039153688314145e-05,
+      "loss": 0.1818,
+      "num_input_tokens_seen": 4067832,
+      "step": 156
+    },
+    {
+      "epoch": 5.626728110599078,
+      "grad_norm": 0.23090755939483643,
+      "learning_rate": 4.0276665538239996e-05,
+      "loss": 0.1372,
+      "num_input_tokens_seen": 4093672,
+      "step": 157
+    },
+    {
+      "epoch": 5.663594470046083,
+      "grad_norm": 0.217911958694458,
+      "learning_rate": 4.0161277134970345e-05,
+      "loss": 0.1648,
+      "num_input_tokens_seen": 4119400,
+      "step": 158
+    },
+    {
+      "epoch": 5.700460829493087,
+      "grad_norm": 0.3220454454421997,
+      "learning_rate": 4.0045375578801214e-05,
+      "loss": 0.1516,
+      "num_input_tokens_seen": 4148144,
+      "step": 159
+    },
+    {
+      "epoch": 5.7373271889400925,
+      "grad_norm": 0.2835656404495239,
+      "learning_rate": 3.9928964792569655e-05,
+      "loss": 0.1597,
+      "num_input_tokens_seen": 4169456,
+      "step": 160
+    },
+    {
+      "epoch": 5.774193548387097,
+      "grad_norm": 0.22007523477077484,
+      "learning_rate": 3.981204871634827e-05,
+      "loss": 0.1535,
+      "num_input_tokens_seen": 4207704,
+      "step": 161
+    },
+    {
+      "epoch": 5.811059907834101,
+      "grad_norm": 0.23791228234767914,
+      "learning_rate": 3.969463130731183e-05,
+      "loss": 0.1472,
+      "num_input_tokens_seen": 4230304,
+      "step": 162
+    },
+    {
+      "epoch": 5.847926267281106,
+      "grad_norm": 0.21866248548030853,
+      "learning_rate": 3.957671653960337e-05,
+      "loss": 0.1362,
+      "num_input_tokens_seen": 4255536,
+      "step": 163
+    },
+    {
+      "epoch": 5.88479262672811,
+      "grad_norm": 0.24037593603134155,
+      "learning_rate": 3.945830840419966e-05,
+      "loss": 0.1443,
+      "num_input_tokens_seen": 4285160,
+      "step": 164
+    },
+    {
+      "epoch": 5.921658986175116,
+      "grad_norm": 0.19541749358177185,
+      "learning_rate": 3.933941090877615e-05,
+      "loss": 0.1647,
+      "num_input_tokens_seen": 4323272,
+      "step": 165
+    },
+    {
+      "epoch": 5.95852534562212,
+      "grad_norm": 0.22034280002117157,
+      "learning_rate": 3.9220028077571295e-05,
+      "loss": 0.1921,
+      "num_input_tokens_seen": 4354912,
+      "step": 166
+    },
+    {
+      "epoch": 5.9953917050691246,
+      "grad_norm": 0.19790567457675934,
+      "learning_rate": 3.910016395125037e-05,
+      "loss": 0.1522,
+      "num_input_tokens_seen": 4387064,
+      "step": 167
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 0.6411383152008057,
+      "learning_rate": 3.897982258676867e-05,
+      "loss": 0.2109,
+      "num_input_tokens_seen": 4391856,
+      "step": 168
+    },
+    {
+      "epoch": 6.0368663594470044,
+      "grad_norm": 0.22729864716529846,
+      "learning_rate": 3.885900805723429e-05,
+      "loss": 0.1607,
+      "num_input_tokens_seen": 4419952,
+      "step": 169
+    },
+    {
+      "epoch": 6.073732718894009,
+      "grad_norm": 0.21639327704906464,
+      "learning_rate": 3.873772445177015e-05,
+      "loss": 0.1277,
+      "num_input_tokens_seen": 4442576,
+      "step": 170
+    },
+    {
+      "epoch": 6.110599078341014,
+      "grad_norm": 0.2176467776298523,
+      "learning_rate": 3.861597587537568e-05,
+      "loss": 0.121,
+      "num_input_tokens_seen": 4477568,
+      "step": 171
+    },
+    {
+      "epoch": 6.147465437788019,
+      "grad_norm": 0.21243643760681152,
+      "learning_rate": 3.8493766448787825e-05,
+      "loss": 0.1976,
+      "num_input_tokens_seen": 4509944,
+      "step": 172
+    },
+    {
+      "epoch": 6.184331797235023,
+      "grad_norm": 0.2905130386352539,
+      "learning_rate": 3.837110030834161e-05,
+      "loss": 0.141,
+      "num_input_tokens_seen": 4534704,
+      "step": 173
+    },
+    {
+      "epoch": 6.221198156682028,
+      "grad_norm": 0.2976662218570709,
+      "learning_rate": 3.824798160583012e-05,
+      "loss": 0.1389,
+      "num_input_tokens_seen": 4563912,
+      "step": 174
+    },
+    {
+      "epoch": 6.258064516129032,
+      "grad_norm": 0.2367183119058609,
+      "learning_rate": 3.8124414508364e-05,
+      "loss": 0.1241,
+      "num_input_tokens_seen": 4594712,
+      "step": 175
+    },
+    {
+      "epoch": 6.2949308755760365,
+      "grad_norm": 0.23760217428207397,
+      "learning_rate": 3.8000403198230387e-05,
+      "loss": 0.1415,
+      "num_input_tokens_seen": 4621448,
+      "step": 176
+    },
+    {
+      "epoch": 6.331797235023042,
+      "grad_norm": 0.23512595891952515,
+      "learning_rate": 3.787595187275136e-05,
+      "loss": 0.1277,
+      "num_input_tokens_seen": 4648744,
+      "step": 177
+    },
+    {
+      "epoch": 6.368663594470046,
+      "grad_norm": 0.24028612673282623,
+      "learning_rate": 3.775106474414188e-05,
+      "loss": 0.1636,
+      "num_input_tokens_seen": 4679480,
+      "step": 178
+    },
+    {
+      "epoch": 6.405529953917051,
+      "grad_norm": 0.2704714238643646,
+      "learning_rate": 3.762574603936725e-05,
+      "loss": 0.1341,
+      "num_input_tokens_seen": 4698440,
+      "step": 179
+    },
+    {
+      "epoch": 6.442396313364055,
+      "grad_norm": 0.27304500341415405,
+      "learning_rate": 3.7500000000000003e-05,
+      "loss": 0.1472,
+      "num_input_tokens_seen": 4721120,
+      "step": 180
+    },
+    {
+      "epoch": 6.47926267281106,
+      "grad_norm": 0.2529843747615814,
+      "learning_rate": 3.7373830882076354e-05,
+      "loss": 0.1344,
+      "num_input_tokens_seen": 4743536,
+      "step": 181
+    },
+    {
+      "epoch": 6.516129032258064,
+      "grad_norm": 0.23612789809703827,
+      "learning_rate": 3.7247242955952175e-05,
+      "loss": 0.1255,
+      "num_input_tokens_seen": 4773248,
+      "step": 182
+    },
+    {
+      "epoch": 6.552995391705069,
+      "grad_norm": 0.2793615162372589,
+      "learning_rate": 3.712024050615843e-05,
+      "loss": 0.141,
+      "num_input_tokens_seen": 4804600,
+      "step": 183
+    },
+    {
+      "epoch": 6.589861751152074,
+      "grad_norm": 0.29646047949790955,
+      "learning_rate": 3.699282783125616e-05,
+      "loss": 0.1226,
+      "num_input_tokens_seen": 4826744,
+      "step": 184
+    },
+    {
+      "epoch": 6.626728110599078,
+      "grad_norm": 0.25531846284866333,
+      "learning_rate": 3.686500924369101e-05,
+      "loss": 0.1074,
+      "num_input_tokens_seen": 4849584,
+      "step": 185
+    },
+    {
+      "epoch": 6.663594470046083,
+      "grad_norm": 0.24337461590766907,
+      "learning_rate": 3.673678906964727e-05,
+      "loss": 0.1084,
+      "num_input_tokens_seen": 4875336,
+      "step": 186
+    },
+    {
+      "epoch": 6.700460829493087,
+      "grad_norm": 0.30051150918006897,
+      "learning_rate": 3.660817164890143e-05,
+      "loss": 0.1273,
+      "num_input_tokens_seen": 4895896,
+      "step": 187
+    },
+    {
+      "epoch": 6.7373271889400925,
+      "grad_norm": 0.286495178937912,
+      "learning_rate": 3.6479161334675296e-05,
+      "loss": 0.1447,
+      "num_input_tokens_seen": 4924360,
+      "step": 188
+    },
+    {
+      "epoch": 6.774193548387097,
+      "grad_norm": 0.28501835465431213,
+      "learning_rate": 3.634976249348867e-05,
+      "loss": 0.1548,
+      "num_input_tokens_seen": 4948384,
+      "step": 189
+    },
+    {
+      "epoch": 6.811059907834101,
+      "grad_norm": 0.24231503903865814,
+      "learning_rate": 3.621997950501156e-05,
+      "loss": 0.1507,
+      "num_input_tokens_seen": 4980368,
+      "step": 190
+    },
+    {
+      "epoch": 6.847926267281106,
+      "grad_norm": 0.2752607464790344,
+      "learning_rate": 3.6089816761915906e-05,
+      "loss": 0.1282,
+      "num_input_tokens_seen": 5007744,
+      "step": 191
+    },
+    {
+      "epoch": 6.88479262672811,
+      "grad_norm": 0.27704674005508423,
+      "learning_rate": 3.5959278669726935e-05,
+      "loss": 0.1411,
+      "num_input_tokens_seen": 5030528,
+      "step": 192
+    },
+    {
+      "epoch": 6.921658986175116,
+      "grad_norm": 0.22601065039634705,
+      "learning_rate": 3.582836964667408e-05,
+      "loss": 0.1156,
+      "num_input_tokens_seen": 5065000,
+      "step": 193
+    },
+    {
+      "epoch": 6.95852534562212,
+      "grad_norm": 0.24917051196098328,
+      "learning_rate": 3.569709412354136e-05,
+      "loss": 0.1143,
+      "num_input_tokens_seen": 5091440,
+      "step": 194
+    },
+    {
+      "epoch": 6.9953917050691246,
+      "grad_norm": 0.31986546516418457,
+      "learning_rate": 3.556545654351749e-05,
+      "loss": 0.1466,
+      "num_input_tokens_seen": 5120064,
+      "step": 195
+    },
+    {
+      "epoch": 7.0,
+      "grad_norm": 0.6616023778915405,
+      "learning_rate": 3.543346136204545e-05,
+      "loss": 0.1637,
+      "num_input_tokens_seen": 5123832,
+      "step": 196
+    },
+    {
+      "epoch": 7.0368663594470044,
+      "grad_norm": 0.2567199766635895,
+      "learning_rate": 3.5301113046671714e-05,
+      "loss": 0.1111,
+      "num_input_tokens_seen": 5154136,
+      "step": 197
+    },
+    {
+      "epoch": 7.073732718894009,
+      "grad_norm": 0.24546091258525848,
+      "learning_rate": 3.516841607689501e-05,
+      "loss": 0.1015,
+      "num_input_tokens_seen": 5182792,
+      "step": 198
+    },
+    {
+      "epoch": 7.110599078341014,
+      "grad_norm": 0.25169137120246887,
+      "learning_rate": 3.503537494401473e-05,
+      "loss": 0.1091,
+      "num_input_tokens_seen": 5208280,
+      "step": 199
+    },
+    {
+      "epoch": 7.147465437788019,
+      "grad_norm": 0.27022022008895874,
+      "learning_rate": 3.490199415097892e-05,
+      "loss": 0.1165,
+      "num_input_tokens_seen": 5226024,
+      "step": 200
+    },
+    {
+      "epoch": 7.184331797235023,
+      "grad_norm": 0.2491355836391449,
+      "learning_rate": 3.476827821223184e-05,
+      "loss": 0.1047,
+      "num_input_tokens_seen": 5259096,
+      "step": 201
+    },
+    {
+      "epoch": 7.221198156682028,
+      "grad_norm": 0.28957781195640564,
+      "learning_rate": 3.463423165356121e-05,
+      "loss": 0.1066,
+      "num_input_tokens_seen": 5282232,
+      "step": 202
+    },
+    {
+      "epoch": 7.258064516129032,
+      "grad_norm": 0.29709964990615845,
+      "learning_rate": 3.449985901194498e-05,
+      "loss": 0.0902,
+      "num_input_tokens_seen": 5306272,
+      "step": 203
+    },
+    {
+      "epoch": 7.2949308755760365,
+      "grad_norm": 0.3285166919231415,
+      "learning_rate": 3.436516483539781e-05,
+      "loss": 0.1072,
+      "num_input_tokens_seen": 5330832,
+      "step": 204
+    },
+    {
+      "epoch": 7.331797235023042,
+      "grad_norm": 0.24284234642982483,
+      "learning_rate": 3.423015368281711e-05,
+      "loss": 0.1195,
+      "num_input_tokens_seen": 5359992,
+      "step": 205
+    },
+    {
+      "epoch": 7.368663594470046,
+      "grad_norm": 0.2531001567840576,
+      "learning_rate": 3.409483012382879e-05,
+      "loss": 0.1038,
+      "num_input_tokens_seen": 5392232,
+      "step": 206
+    },
+    {
+      "epoch": 7.405529953917051,
+      "grad_norm": 0.273573637008667,
+      "learning_rate": 3.39591987386325e-05,
+      "loss": 0.109,
+      "num_input_tokens_seen": 5416272,
+      "step": 207
+    },
+    {
+      "epoch": 7.442396313364055,
+      "grad_norm": 0.28594842553138733,
+      "learning_rate": 3.382326411784672e-05,
+      "loss": 0.1035,
+      "num_input_tokens_seen": 5442312,
+      "step": 208
+    },
+    {
+      "epoch": 7.47926267281106,
+      "grad_norm": 0.24845322966575623,
+      "learning_rate": 3.3687030862353286e-05,
+      "loss": 0.0997,
+      "num_input_tokens_seen": 5473408,
+      "step": 209
+    },
+    {
+      "epoch": 7.516129032258064,
+      "grad_norm": 0.2672761082649231,
+      "learning_rate": 3.355050358314172e-05,
+      "loss": 0.1034,
+      "num_input_tokens_seen": 5494256,
+      "step": 210
+    },
+    {
+      "epoch": 7.552995391705069,
+      "grad_norm": 0.29590561985969543,
+      "learning_rate": 3.3413686901153165e-05,
+      "loss": 0.0994,
+      "num_input_tokens_seen": 5519384,
+      "step": 211
+    },
+    {
+      "epoch": 7.589861751152074,
+      "grad_norm": 0.2702927887439728,
+      "learning_rate": 3.327658544712395e-05,
+      "loss": 0.1197,
+      "num_input_tokens_seen": 5544760,
+      "step": 212
+    },
+    {
+      "epoch": 7.626728110599078,
+      "grad_norm": 0.2904987633228302,
+      "learning_rate": 3.313920386142892e-05,
+      "loss": 0.1423,
+      "num_input_tokens_seen": 5574216,
+      "step": 213
+    },
+    {
+      "epoch": 7.663594470046083,
+      "grad_norm": 0.27985090017318726,
+      "learning_rate": 3.3001546793924285e-05,
+      "loss": 0.1171,
+      "num_input_tokens_seen": 5602072,
+      "step": 214
+    },
+    {
+      "epoch": 7.700460829493087,
+      "grad_norm": 0.25102686882019043,
+      "learning_rate": 3.2863618903790346e-05,
+      "loss": 0.1118,
+      "num_input_tokens_seen": 5632056,
+      "step": 215
+    },
+    {
+      "epoch": 7.7373271889400925,
+      "grad_norm": 0.3189446032047272,
+      "learning_rate": 3.272542485937369e-05,
+      "loss": 0.105,
+      "num_input_tokens_seen": 5658216,
+      "step": 216
+    },
+    {
+      "epoch": 7.774193548387097,
+      "grad_norm": 0.30590710043907166,
+      "learning_rate": 3.2586969338029274e-05,
+      "loss": 0.1083,
+      "num_input_tokens_seen": 5684712,
+      "step": 217
+    },
+    {
+      "epoch": 7.811059907834101,
+      "grad_norm": 0.3227143883705139,
+      "learning_rate": 3.244825702596205e-05,
+      "loss": 0.1477,
+      "num_input_tokens_seen": 5709472,
+      "step": 218
+    },
+    {
+      "epoch": 7.847926267281106,
+      "grad_norm": 0.25323161482810974,
+      "learning_rate": 3.230929261806842e-05,
+      "loss": 0.1057,
+      "num_input_tokens_seen": 5742728,
+      "step": 219
+    },
+    {
+      "epoch": 7.88479262672811,
+      "grad_norm": 0.27679508924484253,
+      "learning_rate": 3.217008081777726e-05,
+      "loss": 0.1094,
+      "num_input_tokens_seen": 5768848,
+      "step": 220
+    },
+    {
+      "epoch": 7.921658986175116,
+      "grad_norm": 0.3211810886859894,
+      "learning_rate": 3.203062633689077e-05,
+      "loss": 0.1092,
+      "num_input_tokens_seen": 5795008,
+      "step": 221
+    },
+    {
+      "epoch": 7.95852534562212,
+      "grad_norm": 0.3296988904476166,
+      "learning_rate": 3.1890933895424976e-05,
+      "loss": 0.1253,
+      "num_input_tokens_seen": 5818160,
+      "step": 222
+    },
+    {
+      "epoch": 7.9953917050691246,
+      "grad_norm": 0.2729380130767822,
+      "learning_rate": 3.1751008221450025e-05,
+      "loss": 0.1264,
+      "num_input_tokens_seen": 5850472,
+      "step": 223
+    },
+    {
+      "epoch": 8.0,
+      "grad_norm": 0.6995195150375366,
+      "learning_rate": 3.161085405093006e-05,
+      "loss": 0.1218,
+      "num_input_tokens_seen": 5855808,
+      "step": 224
+    },
+    {
+      "epoch": 8.036866359447005,
+      "grad_norm": 0.3297252357006073,
+      "learning_rate": 3.147047612756302e-05,
+      "loss": 0.0946,
+      "num_input_tokens_seen": 5884832,
+      "step": 225
+    },
+    {
+      "epoch": 8.073732718894009,
+      "grad_norm": 0.2687251567840576,
+      "learning_rate": 3.132987920262005e-05,
+      "loss": 0.0889,
+      "num_input_tokens_seen": 5903392,
+      "step": 226
+    },
+    {
+      "epoch": 8.110599078341014,
+      "grad_norm": 0.2826682925224304,
+      "learning_rate": 3.118906803478465e-05,
+      "loss": 0.1084,
+      "num_input_tokens_seen": 5928240,
+      "step": 227
+    },
+    {
+      "epoch": 8.147465437788018,
+      "grad_norm": 0.27045705914497375,
+      "learning_rate": 3.104804738999169e-05,
+      "loss": 0.1134,
+      "num_input_tokens_seen": 5958392,
+      "step": 228
+    },
+    {
+      "epoch": 8.184331797235023,
+      "grad_norm": 0.2565663754940033,
+      "learning_rate": 3.090682204126604e-05,
+      "loss": 0.0929,
+      "num_input_tokens_seen": 5984088,
+      "step": 229
+    },
+    {
+      "epoch": 8.221198156682028,
+      "grad_norm": 0.28110557794570923,
+      "learning_rate": 3.076539676856101e-05,
+      "loss": 0.0922,
+      "num_input_tokens_seen": 6007936,
+      "step": 230
+    },
+    {
+      "epoch": 8.258064516129032,
+      "grad_norm": 0.2717958092689514,
+      "learning_rate": 3.062377635859663e-05,
+      "loss": 0.072,
+      "num_input_tokens_seen": 6033336,
+      "step": 231
+    },
+    {
+      "epoch": 8.294930875576037,
+      "grad_norm": 0.3095947802066803,
+      "learning_rate": 3.048196560469758e-05,
+      "loss": 0.1176,
+      "num_input_tokens_seen": 6060160,
+      "step": 232
+    },
+    {
+      "epoch": 8.331797235023041,
+      "grad_norm": 0.3245174288749695,
+      "learning_rate": 3.0339969306631005e-05,
+      "loss": 0.0886,
+      "num_input_tokens_seen": 6088280,
+      "step": 233
+    },
+    {
+      "epoch": 8.368663594470046,
+      "grad_norm": 0.2761123776435852,
+      "learning_rate": 3.0197792270443982e-05,
+      "loss": 0.0831,
+      "num_input_tokens_seen": 6119752,
+      "step": 234
+    },
+    {
+      "epoch": 8.40552995391705,
+      "grad_norm": 0.33236628770828247,
+      "learning_rate": 3.0055439308300952e-05,
+      "loss": 0.1222,
+      "num_input_tokens_seen": 6154672,
+      "step": 235
+    },
+    {
+      "epoch": 8.442396313364055,
+      "grad_norm": 0.2539371848106384,
+      "learning_rate": 2.9912915238320754e-05,
+      "loss": 0.0856,
+      "num_input_tokens_seen": 6181152,
+      "step": 236
+    },
+    {
+      "epoch": 8.47926267281106,
+      "grad_norm": 0.25849923491477966,
+      "learning_rate": 2.9770224884413623e-05,
+      "loss": 0.0842,
+      "num_input_tokens_seen": 6208248,
+      "step": 237
+    },
+    {
+      "epoch": 8.516129032258064,
+      "grad_norm": 0.2990070879459381,
+      "learning_rate": 2.9627373076117863e-05,
+      "loss": 0.0816,
+      "num_input_tokens_seen": 6232704,
+      "step": 238
+    },
+    {
+      "epoch": 8.55299539170507,
+      "grad_norm": 0.2482941746711731,
+      "learning_rate": 2.9484364648436437e-05,
+      "loss": 0.0725,
+      "num_input_tokens_seen": 6264600,
+      "step": 239
+    },
+    {
+      "epoch": 8.589861751152073,
+      "grad_norm": 0.31631267070770264,
+      "learning_rate": 2.9341204441673266e-05,
+      "loss": 0.0917,
+      "num_input_tokens_seen": 6289416,
+      "step": 240
+    },
+    {
+      "epoch": 8.626728110599078,
+      "grad_norm": 0.32536354660987854,
+      "learning_rate": 2.9197897301269435e-05,
+      "loss": 0.1029,
+      "num_input_tokens_seen": 6313624,
+      "step": 241
+    },
+    {
+      "epoch": 8.663594470046084,
+      "grad_norm": 0.28533875942230225,
+      "learning_rate": 2.905444807763919e-05,
+      "loss": 0.0876,
+      "num_input_tokens_seen": 6337560,
+      "step": 242
+    },
+    {
+      "epoch": 8.700460829493087,
+      "grad_norm": 0.32342368364334106,
+      "learning_rate": 2.8910861626005776e-05,
+      "loss": 0.081,
+      "num_input_tokens_seen": 6362016,
+      "step": 243
+    },
+    {
+      "epoch": 8.737327188940093,
+      "grad_norm": 0.2809208333492279,
+      "learning_rate": 2.876714280623708e-05,
+      "loss": 0.0846,
+      "num_input_tokens_seen": 6399368,
+      "step": 244
+    },
+    {
+      "epoch": 8.774193548387096,
+      "grad_norm": 0.2525993883609772,
+      "learning_rate": 2.8623296482681166e-05,
+      "loss": 0.0865,
+      "num_input_tokens_seen": 6428808,
+      "step": 245
+    },
+    {
+      "epoch": 8.811059907834101,
+      "grad_norm": 0.331150084733963,
+      "learning_rate": 2.8479327524001636e-05,
+      "loss": 0.1033,
+      "num_input_tokens_seen": 6450424,
+      "step": 246
+    },
+    {
+      "epoch": 8.847926267281107,
+      "grad_norm": 0.3234056532382965,
+      "learning_rate": 2.833524080301282e-05,
+      "loss": 0.0758,
+      "num_input_tokens_seen": 6477040,
+      "step": 247
+    },
+    {
+      "epoch": 8.88479262672811,
+      "grad_norm": 0.3200438320636749,
+      "learning_rate": 2.8191041196514873e-05,
+      "loss": 0.102,
+      "num_input_tokens_seen": 6504424,
+      "step": 248
+    },
+    {
+      "epoch": 8.921658986175116,
+      "grad_norm": 0.3212469220161438,
+      "learning_rate": 2.8046733585128687e-05,
+      "loss": 0.0812,
+      "num_input_tokens_seen": 6528144,
+      "step": 249
+    },
+    {
+      "epoch": 8.95852534562212,
+      "grad_norm": 0.37497377395629883,
+      "learning_rate": 2.7902322853130757e-05,
+      "loss": 0.0767,
+      "num_input_tokens_seen": 6549528,
+      "step": 250
+    },
+    {
+      "epoch": 8.995391705069125,
+      "grad_norm": 0.27313902974128723,
+      "learning_rate": 2.7757813888287798e-05,
+      "loss": 0.0734,
+      "num_input_tokens_seen": 6577392,
+      "step": 251
+    },
+    {
+      "epoch": 9.0,
+      "grad_norm": 0.4119221270084381,
+      "learning_rate": 2.761321158169134e-05,
+      "loss": 0.0718,
+      "num_input_tokens_seen": 6587784,
+      "step": 252
+    },
+    {
+      "epoch": 9.036866359447005,
+      "grad_norm": 0.23572459816932678,
+      "learning_rate": 2.7468520827592197e-05,
+      "loss": 0.0589,
+      "num_input_tokens_seen": 6614752,
+      "step": 253
+    },
+    {
+      "epoch": 9.073732718894009,
+      "grad_norm": 0.22828935086727142,
+      "learning_rate": 2.732374652323481e-05,
+      "loss": 0.0682,
+      "num_input_tokens_seen": 6648864,
+      "step": 254
+    },
+    {
+      "epoch": 9.110599078341014,
+      "grad_norm": 0.28416910767555237,
+      "learning_rate": 2.717889356869146e-05,
+      "loss": 0.0768,
+      "num_input_tokens_seen": 6678136,
+      "step": 255
+    },
+    {
+      "epoch": 9.147465437788018,
+      "grad_norm": 0.3248147666454315,
+      "learning_rate": 2.7033966866696457e-05,
+      "loss": 0.0739,
+      "num_input_tokens_seen": 6704880,
+      "step": 256
+    },
+    {
+      "epoch": 9.184331797235023,
+      "grad_norm": 0.2697289288043976,
+      "learning_rate": 2.6888971322480205e-05,
+      "loss": 0.0884,
+      "num_input_tokens_seen": 6742080,
+      "step": 257
+    },
+    {
+      "epoch": 9.221198156682028,
+      "grad_norm": 0.3310103714466095,
+      "learning_rate": 2.674391184360313e-05,
+      "loss": 0.0836,
+      "num_input_tokens_seen": 6771256,
+      "step": 258
+    },
+    {
+      "epoch": 9.258064516129032,
+      "grad_norm": 0.3416021168231964,
+      "learning_rate": 2.659879333978964e-05,
+      "loss": 0.0728,
+      "num_input_tokens_seen": 6798392,
+      "step": 259
+    },
+    {
+      "epoch": 9.294930875576037,
+      "grad_norm": 0.25257182121276855,
+      "learning_rate": 2.6453620722761896e-05,
+      "loss": 0.0682,
+      "num_input_tokens_seen": 6830648,
+      "step": 260
+    },
+    {
+      "epoch": 9.331797235023041,
+      "grad_norm": 0.31290990114212036,
+      "learning_rate": 2.63083989060736e-05,
+      "loss": 0.0701,
+      "num_input_tokens_seen": 6853800,
+      "step": 261
+    },
+    {
+      "epoch": 9.368663594470046,
+      "grad_norm": 0.28703242540359497,
+      "learning_rate": 2.6163132804943675e-05,
+      "loss": 0.0641,
+      "num_input_tokens_seen": 6880096,
+      "step": 262
+    },
+    {
+      "epoch": 9.40552995391705,
+      "grad_norm": 0.33095675706863403,
+      "learning_rate": 2.60178273360899e-05,
+      "loss": 0.0695,
+      "num_input_tokens_seen": 6905400,
+      "step": 263
+    },
+    {
+      "epoch": 9.442396313364055,
+      "grad_norm": 0.3316652774810791,
+      "learning_rate": 2.587248741756253e-05,
+      "loss": 0.0993,
+      "num_input_tokens_seen": 6933352,
+      "step": 264
+    },
+    {
+      "epoch": 9.47926267281106,
+      "grad_norm": 0.3296186029911041,
+      "learning_rate": 2.5727117968577784e-05,
+      "loss": 0.0702,
+      "num_input_tokens_seen": 6955160,
+      "step": 265
+    },
+    {
+      "epoch": 9.516129032258064,
+      "grad_norm": 0.288353830575943,
+      "learning_rate": 2.5581723909351406e-05,
+      "loss": 0.0632,
+      "num_input_tokens_seen": 6976256,
+      "step": 266
+    },
+    {
+      "epoch": 9.55299539170507,
+      "grad_norm": 0.34519389271736145,
+      "learning_rate": 2.5436310160932092e-05,
+      "loss": 0.0873,
+      "num_input_tokens_seen": 7002472,
+      "step": 267
+    },
+    {
+      "epoch": 9.589861751152073,
+      "grad_norm": 0.2644766569137573,
+      "learning_rate": 2.5290881645034932e-05,
+      "loss": 0.0515,
+      "num_input_tokens_seen": 7034352,
+      "step": 268
+    },
+    {
+      "epoch": 9.626728110599078,
+      "grad_norm": 0.34003907442092896,
+      "learning_rate": 2.5145443283874848e-05,
+      "loss": 0.0679,
+      "num_input_tokens_seen": 7062184,
+      "step": 269
+    },
+    {
+      "epoch": 9.663594470046084,
+      "grad_norm": 0.24440152943134308,
+      "learning_rate": 2.5e-05,
+      "loss": 0.0649,
+      "num_input_tokens_seen": 7090600,
+      "step": 270
+    },
+    {
+      "epoch": 9.700460829493087,
+      "grad_norm": 0.38781893253326416,
+      "learning_rate": 2.485455671612515e-05,
+      "loss": 0.0843,
+      "num_input_tokens_seen": 7112656,
+      "step": 271
+    },
+    {
+      "epoch": 9.737327188940093,
+      "grad_norm": 0.35135138034820557,
+      "learning_rate": 2.470911835496508e-05,
+      "loss": 0.0709,
+      "num_input_tokens_seen": 7134448,
+      "step": 272
+    },
+    {
+      "epoch": 9.774193548387096,
+      "grad_norm": 0.3243424594402313,
+      "learning_rate": 2.4563689839067913e-05,
+      "loss": 0.0566,
+      "num_input_tokens_seen": 7159000,
+      "step": 273
+    },
+    {
+      "epoch": 9.811059907834101,
+      "grad_norm": 0.2978050708770752,
+      "learning_rate": 2.4418276090648596e-05,
+      "loss": 0.0783,
+      "num_input_tokens_seen": 7193008,
+      "step": 274
+    },
+    {
+      "epoch": 9.847926267281107,
+      "grad_norm": 0.3357181251049042,
+      "learning_rate": 2.4272882031422215e-05,
+      "loss": 0.0755,
+      "num_input_tokens_seen": 7214720,
+      "step": 275
+    },
+    {
+      "epoch": 9.88479262672811,
+      "grad_norm": 0.3311111032962799,
+      "learning_rate": 2.4127512582437485e-05,
+      "loss": 0.0756,
+      "num_input_tokens_seen": 7236408,
+      "step": 276
+    },
+    {
+      "epoch": 9.921658986175116,
+      "grad_norm": 0.34926944971084595,
+      "learning_rate": 2.3982172663910108e-05,
+      "loss": 0.0774,
+      "num_input_tokens_seen": 7254272,
+      "step": 277
+    },
+    {
+      "epoch": 9.95852534562212,
+      "grad_norm": 0.27301025390625,
+      "learning_rate": 2.3836867195056335e-05,
+      "loss": 0.0665,
+      "num_input_tokens_seen": 7285272,
+      "step": 278
+    },
+    {
+      "epoch": 9.995391705069125,
+      "grad_norm": 0.31272560358047485,
+      "learning_rate": 2.3691601093926404e-05,
+      "loss": 0.075,
+      "num_input_tokens_seen": 7318112,
+      "step": 279
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 1.0591803789138794,
+      "learning_rate": 2.3546379277238107e-05,
+      "loss": 0.0886,
+      "num_input_tokens_seen": 7319760,
+      "step": 280
+    },
+    {
+      "epoch": 10.036866359447005,
+      "grad_norm": 0.2831885814666748,
+      "learning_rate": 2.3401206660210363e-05,
+      "loss": 0.0562,
+      "num_input_tokens_seen": 7343240,
+      "step": 281
+    },
+    {
+      "epoch": 10.073732718894009,
+      "grad_norm": 0.2866736650466919,
+      "learning_rate": 2.3256088156396868e-05,
+      "loss": 0.0519,
+      "num_input_tokens_seen": 7366928,
+      "step": 282
+    },
+    {
+      "epoch": 10.110599078341014,
+      "grad_norm": 0.28110256791114807,
+      "learning_rate": 2.3111028677519804e-05,
+      "loss": 0.0493,
+      "num_input_tokens_seen": 7391576,
+      "step": 283
+    },
+    {
+      "epoch": 10.147465437788018,
+      "grad_norm": 0.2599363327026367,
+      "learning_rate": 2.2966033133303545e-05,
+      "loss": 0.064,
+      "num_input_tokens_seen": 7422848,
+      "step": 284
+    },
+    {
+      "epoch": 10.184331797235023,
+      "grad_norm": 0.3725353181362152,
+      "learning_rate": 2.2821106431308544e-05,
+      "loss": 0.0876,
+      "num_input_tokens_seen": 7445112,
+      "step": 285
+    },
+    {
+      "epoch": 10.221198156682028,
+      "grad_norm": 0.31412675976753235,
+      "learning_rate": 2.2676253476765196e-05,
+      "loss": 0.0631,
+      "num_input_tokens_seen": 7469384,
+      "step": 286
+    },
+    {
+      "epoch": 10.258064516129032,
+      "grad_norm": 0.3256174921989441,
+      "learning_rate": 2.2531479172407805e-05,
+      "loss": 0.059,
+      "num_input_tokens_seen": 7493816,
+      "step": 287
+    },
+    {
+      "epoch": 10.294930875576037,
+      "grad_norm": 0.2877194285392761,
+      "learning_rate": 2.238678841830867e-05,
+      "loss": 0.0534,
+      "num_input_tokens_seen": 7523536,
+      "step": 288
+    },
+    {
+      "epoch": 10.331797235023041,
+      "grad_norm": 0.2373494654893875,
+      "learning_rate": 2.2242186111712208e-05,
+      "loss": 0.0479,
+      "num_input_tokens_seen": 7552320,
+      "step": 289
+    },
+    {
+      "epoch": 10.368663594470046,
+      "grad_norm": 0.36851245164871216,
+      "learning_rate": 2.2097677146869242e-05,
+      "loss": 0.0693,
+      "num_input_tokens_seen": 7574816,
+      "step": 290
+    },
+    {
+      "epoch": 10.40552995391705,
+      "grad_norm": 0.32222089171409607,
+      "learning_rate": 2.195326641487132e-05,
+      "loss": 0.0632,
+      "num_input_tokens_seen": 7598688,
+      "step": 291
+    },
+    {
+      "epoch": 10.442396313364055,
+      "grad_norm": 0.33085688948631287,
+      "learning_rate": 2.1808958803485136e-05,
+      "loss": 0.0589,
+      "num_input_tokens_seen": 7620856,
+      "step": 292
+    },
+    {
+      "epoch": 10.47926267281106,
+      "grad_norm": 0.28755688667297363,
+      "learning_rate": 2.1664759196987182e-05,
+      "loss": 0.0666,
+      "num_input_tokens_seen": 7647512,
+      "step": 293
+    },
+    {
+      "epoch": 10.516129032258064,
+      "grad_norm": 0.3906136751174927,
+      "learning_rate": 2.1520672475998373e-05,
+      "loss": 0.0885,
+      "num_input_tokens_seen": 7681488,
+      "step": 294
+    },
+    {
+      "epoch": 10.55299539170507,
+      "grad_norm": 0.2831442058086395,
+      "learning_rate": 2.1376703517318837e-05,
+      "loss": 0.0581,
+      "num_input_tokens_seen": 7707480,
+      "step": 295
+    },
+    {
+      "epoch": 10.589861751152073,
+      "grad_norm": 0.2866826355457306,
+      "learning_rate": 2.1232857193762924e-05,
+      "loss": 0.049,
+      "num_input_tokens_seen": 7733640,
+      "step": 296
+    },
+    {
+      "epoch": 10.626728110599078,
+      "grad_norm": 0.25126680731773376,
+      "learning_rate": 2.1089138373994223e-05,
+      "loss": 0.0476,
+      "num_input_tokens_seen": 7766744,
+      "step": 297
+    },
+    {
+      "epoch": 10.663594470046084,
+      "grad_norm": 0.28204652667045593,
+      "learning_rate": 2.0945551922360818e-05,
+      "loss": 0.0625,
+      "num_input_tokens_seen": 7796144,
+      "step": 298
+    },
+    {
+      "epoch": 10.700460829493087,
+      "grad_norm": 0.32275664806365967,
+      "learning_rate": 2.0802102698730574e-05,
+      "loss": 0.0523,
+      "num_input_tokens_seen": 7814368,
+      "step": 299
+    },
+    {
+      "epoch": 10.737327188940093,
+      "grad_norm": 0.3534482717514038,
+      "learning_rate": 2.0658795558326743e-05,
+      "loss": 0.0484,
+      "num_input_tokens_seen": 7844720,
+      "step": 300
+    },
+    {
+      "epoch": 10.774193548387096,
+      "grad_norm": 0.27784067392349243,
+      "learning_rate": 2.0515635351563565e-05,
+      "loss": 0.0549,
+      "num_input_tokens_seen": 7876192,
+      "step": 301
+    },
+    {
+      "epoch": 10.811059907834101,
+      "grad_norm": 0.3224843442440033,
+      "learning_rate": 2.037262692388214e-05,
+      "loss": 0.0545,
+      "num_input_tokens_seen": 7903336,
+      "step": 302
+    },
+    {
+      "epoch": 10.847926267281107,
+      "grad_norm": 0.4785996973514557,
+      "learning_rate": 2.022977511558638e-05,
+      "loss": 0.0662,
+      "num_input_tokens_seen": 7936544,
+      "step": 303
+    },
+    {
+      "epoch": 10.88479262672811,
+      "grad_norm": 0.2665662169456482,
+      "learning_rate": 2.0087084761679245e-05,
+      "loss": 0.0456,
+      "num_input_tokens_seen": 7966064,
+      "step": 304
+    },
+    {
+      "epoch": 10.921658986175116,
+      "grad_norm": 0.322708398103714,
+      "learning_rate": 1.9944560691699057e-05,
+      "loss": 0.0535,
+      "num_input_tokens_seen": 7989848,
+      "step": 305
+    },
+    {
+      "epoch": 10.95852534562212,
+      "grad_norm": 0.32365888357162476,
+      "learning_rate": 1.980220772955602e-05,
+      "loss": 0.0614,
+      "num_input_tokens_seen": 8020416,
+      "step": 306
+    },
+    {
+      "epoch": 10.995391705069125,
+      "grad_norm": 0.32327035069465637,
+      "learning_rate": 1.9660030693369004e-05,
+      "loss": 0.0456,
+      "num_input_tokens_seen": 8049840,
+      "step": 307
+    },
+    {
+      "epoch": 11.0,
+      "grad_norm": 0.9106179475784302,
+      "learning_rate": 1.9518034395302414e-05,
+      "loss": 0.0734,
+      "num_input_tokens_seen": 8051736,
+      "step": 308
+    },
+    {
+      "epoch": 11.036866359447005,
+      "grad_norm": 0.2582056224346161,
+      "learning_rate": 1.937622364140338e-05,
+      "loss": 0.0362,
+      "num_input_tokens_seen": 8082888,
+      "step": 309
+    },
+    {
+      "epoch": 11.073732718894009,
+      "grad_norm": 0.3024876117706299,
+      "learning_rate": 1.9234603231438995e-05,
+      "loss": 0.0362,
+      "num_input_tokens_seen": 8107768,
+      "step": 310
+    },
+    {
+      "epoch": 11.110599078341014,
+      "grad_norm": 0.2687247097492218,
+      "learning_rate": 1.9093177958733966e-05,
+      "loss": 0.0504,
+      "num_input_tokens_seen": 8149664,
+      "step": 311
+    },
+    {
+      "epoch": 11.147465437788018,
+      "grad_norm": 0.37591755390167236,
+      "learning_rate": 1.895195261000831e-05,
+      "loss": 0.0647,
+      "num_input_tokens_seen": 8172672,
+      "step": 312
+    },
+    {
+      "epoch": 11.184331797235023,
+      "grad_norm": 0.26167264580726624,
+      "learning_rate": 1.8810931965215356e-05,
+      "loss": 0.0391,
+      "num_input_tokens_seen": 8199368,
+      "step": 313
+    },
+    {
+      "epoch": 11.221198156682028,
+      "grad_norm": 0.2515069246292114,
+      "learning_rate": 1.8670120797379958e-05,
+      "loss": 0.0445,
+      "num_input_tokens_seen": 8225816,
+      "step": 314
+    },
+    {
+      "epoch": 11.258064516129032,
+      "grad_norm": 0.32947489619255066,
+      "learning_rate": 1.852952387243698e-05,
+      "loss": 0.0456,
+      "num_input_tokens_seen": 8250688,
+      "step": 315
+    },
+    {
+      "epoch": 11.294930875576037,
+      "grad_norm": 0.3417774438858032,
+      "learning_rate": 1.838914594906995e-05,
+      "loss": 0.0485,
+      "num_input_tokens_seen": 8273448,
+      "step": 316
+    },
+    {
+      "epoch": 11.331797235023041,
+      "grad_norm": 0.3106794059276581,
+      "learning_rate": 1.8248991778549984e-05,
+      "loss": 0.0417,
+      "num_input_tokens_seen": 8295960,
+      "step": 317
+    },
+    {
+      "epoch": 11.368663594470046,
+      "grad_norm": 0.3570440709590912,
+      "learning_rate": 1.8109066104575023e-05,
+      "loss": 0.0459,
+      "num_input_tokens_seen": 8319704,
+      "step": 318
+    },
+    {
+      "epoch": 11.40552995391705,
+      "grad_norm": 0.3612540364265442,
+      "learning_rate": 1.7969373663109234e-05,
+      "loss": 0.0434,
+      "num_input_tokens_seen": 8346544,
+      "step": 319
+    },
+    {
+      "epoch": 11.442396313364055,
+      "grad_norm": 0.3443238437175751,
+      "learning_rate": 1.7829919182222752e-05,
+      "loss": 0.0549,
+      "num_input_tokens_seen": 8373352,
+      "step": 320
+    },
+    {
+      "epoch": 11.47926267281106,
+      "grad_norm": 0.29259899258613586,
+      "learning_rate": 1.7690707381931583e-05,
+      "loss": 0.0326,
+      "num_input_tokens_seen": 8400136,
+      "step": 321
+    },
+    {
+      "epoch": 11.516129032258064,
+      "grad_norm": 0.255287230014801,
+      "learning_rate": 1.755174297403795e-05,
+      "loss": 0.0433,
+      "num_input_tokens_seen": 8435752,
+      "step": 322
+    },
+    {
+      "epoch": 11.55299539170507,
+      "grad_norm": 0.33487167954444885,
+      "learning_rate": 1.7413030661970742e-05,
+      "loss": 0.0451,
+      "num_input_tokens_seen": 8457320,
+      "step": 323
+    },
+    {
+      "epoch": 11.589861751152073,
+      "grad_norm": 0.27473774552345276,
+      "learning_rate": 1.7274575140626318e-05,
+      "loss": 0.0378,
+      "num_input_tokens_seen": 8484320,
+      "step": 324
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 540,
+  "num_input_tokens_seen": 8484320,
+  "num_train_epochs": 20,
+  "save_steps": 27,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.157514257448305e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-351/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-351/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-351/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-351/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-351/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-351/trainer_state.json

@@ -0,0 +1,2841 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 12.55299539170507,
+  "eval_steps": 500,
+  "global_step": 351,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03686635944700461,
+      "grad_norm": 0.06150835379958153,
+      "learning_rate": 4.999957692144361e-05,
+      "loss": 0.663,
+      "num_input_tokens_seen": 23408,
+      "step": 1
+    },
+    {
+      "epoch": 0.07373271889400922,
+      "grad_norm": 0.05766688659787178,
+      "learning_rate": 4.999830770009406e-05,
+      "loss": 0.5847,
+      "num_input_tokens_seen": 45520,
+      "step": 2
+    },
+    {
+      "epoch": 0.11059907834101383,
+      "grad_norm": 0.0780782401561737,
+      "learning_rate": 4.9996192378909786e-05,
+      "loss": 0.5545,
+      "num_input_tokens_seen": 68904,
+      "step": 3
+    },
+    {
+      "epoch": 0.14746543778801843,
+      "grad_norm": 0.07575194537639618,
+      "learning_rate": 4.9993231029486544e-05,
+      "loss": 0.7025,
+      "num_input_tokens_seen": 89696,
+      "step": 4
+    },
+    {
+      "epoch": 0.18433179723502305,
+      "grad_norm": 0.0708586797118187,
+      "learning_rate": 4.998942375205502e-05,
+      "loss": 0.6063,
+      "num_input_tokens_seen": 118440,
+      "step": 5
+    },
+    {
+      "epoch": 0.22119815668202766,
+      "grad_norm": 0.0794093981385231,
+      "learning_rate": 4.99847706754774e-05,
+      "loss": 0.593,
+      "num_input_tokens_seen": 145072,
+      "step": 6
+    },
+    {
+      "epoch": 0.25806451612903225,
+      "grad_norm": 0.04954744130373001,
+      "learning_rate": 4.997927195724303e-05,
+      "loss": 0.4857,
+      "num_input_tokens_seen": 182112,
+      "step": 7
+    },
+    {
+      "epoch": 0.29493087557603687,
+      "grad_norm": 0.06549246609210968,
+      "learning_rate": 4.997292778346312e-05,
+      "loss": 0.5393,
+      "num_input_tokens_seen": 212192,
+      "step": 8
+    },
+    {
+      "epoch": 0.3317972350230415,
+      "grad_norm": 0.07614441215991974,
+      "learning_rate": 4.996573836886435e-05,
+      "loss": 0.7044,
+      "num_input_tokens_seen": 233728,
+      "step": 9
+    },
+    {
+      "epoch": 0.3686635944700461,
+      "grad_norm": 0.07068340480327606,
+      "learning_rate": 4.995770395678171e-05,
+      "loss": 0.6086,
+      "num_input_tokens_seen": 259776,
+      "step": 10
+    },
+    {
+      "epoch": 0.4055299539170507,
+      "grad_norm": 0.07266585528850555,
+      "learning_rate": 4.9948824819150185e-05,
+      "loss": 0.6247,
+      "num_input_tokens_seen": 284816,
+      "step": 11
+    },
+    {
+      "epoch": 0.4423963133640553,
+      "grad_norm": 0.06145177036523819,
+      "learning_rate": 4.993910125649561e-05,
+      "loss": 0.4918,
+      "num_input_tokens_seen": 309776,
+      "step": 12
+    },
+    {
+      "epoch": 0.4792626728110599,
+      "grad_norm": 0.06720694154500961,
+      "learning_rate": 4.992853359792444e-05,
+      "loss": 0.5017,
+      "num_input_tokens_seen": 339064,
+      "step": 13
+    },
+    {
+      "epoch": 0.5161290322580645,
+      "grad_norm": 0.07309022545814514,
+      "learning_rate": 4.9917122201112656e-05,
+      "loss": 0.5447,
+      "num_input_tokens_seen": 369864,
+      "step": 14
+    },
+    {
+      "epoch": 0.5529953917050692,
+      "grad_norm": 0.0778132826089859,
+      "learning_rate": 4.990486745229364e-05,
+      "loss": 0.5034,
+      "num_input_tokens_seen": 397184,
+      "step": 15
+    },
+    {
+      "epoch": 0.5898617511520737,
+      "grad_norm": 0.07727856189012527,
+      "learning_rate": 4.989176976624511e-05,
+      "loss": 0.5277,
+      "num_input_tokens_seen": 426296,
+      "step": 16
+    },
+    {
+      "epoch": 0.6267281105990783,
+      "grad_norm": 0.0700518786907196,
+      "learning_rate": 4.987782958627508e-05,
+      "loss": 0.5356,
+      "num_input_tokens_seen": 459544,
+      "step": 17
+    },
+    {
+      "epoch": 0.663594470046083,
+      "grad_norm": 0.07351912558078766,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.5505,
+      "num_input_tokens_seen": 488616,
+      "step": 18
+    },
+    {
+      "epoch": 0.7004608294930875,
+      "grad_norm": 0.08268705755472183,
+      "learning_rate": 4.9847423660363e-05,
+      "loss": 0.5557,
+      "num_input_tokens_seen": 512176,
+      "step": 19
+    },
+    {
+      "epoch": 0.7373271889400922,
+      "grad_norm": 0.08081424236297607,
+      "learning_rate": 4.983095894354858e-05,
+      "loss": 0.4576,
+      "num_input_tokens_seen": 539456,
+      "step": 20
+    },
+    {
+      "epoch": 0.7741935483870968,
+      "grad_norm": 0.07748426496982574,
+      "learning_rate": 4.9813653791033057e-05,
+      "loss": 0.3918,
+      "num_input_tokens_seen": 561384,
+      "step": 21
+    },
+    {
+      "epoch": 0.8110599078341014,
+      "grad_norm": 0.10133849829435349,
+      "learning_rate": 4.979550878853154e-05,
+      "loss": 0.531,
+      "num_input_tokens_seen": 589792,
+      "step": 22
+    },
+    {
+      "epoch": 0.847926267281106,
+      "grad_norm": 0.09528470784425735,
+      "learning_rate": 4.9776524550184965e-05,
+      "loss": 0.4159,
+      "num_input_tokens_seen": 613944,
+      "step": 23
+    },
+    {
+      "epoch": 0.8847926267281107,
+      "grad_norm": 0.08829868584871292,
+      "learning_rate": 4.975670171853926e-05,
+      "loss": 0.403,
+      "num_input_tokens_seen": 641432,
+      "step": 24
+    },
+    {
+      "epoch": 0.9216589861751152,
+      "grad_norm": 0.10020875930786133,
+      "learning_rate": 4.973604096452361e-05,
+      "loss": 0.4238,
+      "num_input_tokens_seen": 665232,
+      "step": 25
+    },
+    {
+      "epoch": 0.9585253456221198,
+      "grad_norm": 0.1256554126739502,
+      "learning_rate": 4.971454298742779e-05,
+      "loss": 0.5033,
+      "num_input_tokens_seen": 687216,
+      "step": 26
+    },
+    {
+      "epoch": 0.9953917050691244,
+      "grad_norm": 0.17123407125473022,
+      "learning_rate": 4.9692208514878444e-05,
+      "loss": 0.3842,
+      "num_input_tokens_seen": 728880,
+      "step": 27
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.27383726835250854,
+      "learning_rate": 4.966903830281449e-05,
+      "loss": 0.4988,
+      "num_input_tokens_seen": 731976,
+      "step": 28
+    },
+    {
+      "epoch": 1.0368663594470047,
+      "grad_norm": 0.148647278547287,
+      "learning_rate": 4.9645033135461494e-05,
+      "loss": 0.4489,
+      "num_input_tokens_seen": 756184,
+      "step": 29
+    },
+    {
+      "epoch": 1.0737327188940091,
+      "grad_norm": 0.11988866329193115,
+      "learning_rate": 4.962019382530521e-05,
+      "loss": 0.4999,
+      "num_input_tokens_seen": 779072,
+      "step": 30
+    },
+    {
+      "epoch": 1.1105990783410138,
+      "grad_norm": 0.10590679943561554,
+      "learning_rate": 4.9594521213063974e-05,
+      "loss": 0.3711,
+      "num_input_tokens_seen": 805672,
+      "step": 31
+    },
+    {
+      "epoch": 1.1474654377880185,
+      "grad_norm": 0.11934591829776764,
+      "learning_rate": 4.9568016167660334e-05,
+      "loss": 0.3385,
+      "num_input_tokens_seen": 828840,
+      "step": 32
+    },
+    {
+      "epoch": 1.1843317972350231,
+      "grad_norm": 0.11900116503238678,
+      "learning_rate": 4.9540679586191605e-05,
+      "loss": 0.3812,
+      "num_input_tokens_seen": 850792,
+      "step": 33
+    },
+    {
+      "epoch": 1.2211981566820276,
+      "grad_norm": 0.11279546469449997,
+      "learning_rate": 4.951251239389948e-05,
+      "loss": 0.2924,
+      "num_input_tokens_seen": 885608,
+      "step": 34
+    },
+    {
+      "epoch": 1.2580645161290323,
+      "grad_norm": 0.08340126276016235,
+      "learning_rate": 4.948351554413879e-05,
+      "loss": 0.3489,
+      "num_input_tokens_seen": 921912,
+      "step": 35
+    },
+    {
+      "epoch": 1.294930875576037,
+      "grad_norm": 0.11327336728572845,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.2921,
+      "num_input_tokens_seen": 945672,
+      "step": 36
+    },
+    {
+      "epoch": 1.3317972350230414,
+      "grad_norm": 0.10909265279769897,
+      "learning_rate": 4.942303682600178e-05,
+      "loss": 0.3421,
+      "num_input_tokens_seen": 970824,
+      "step": 37
+    },
+    {
+      "epoch": 1.368663594470046,
+      "grad_norm": 0.11239592730998993,
+      "learning_rate": 4.939155700460536e-05,
+      "loss": 0.3848,
+      "num_input_tokens_seen": 995840,
+      "step": 38
+    },
+    {
+      "epoch": 1.4055299539170507,
+      "grad_norm": 0.1034158244729042,
+      "learning_rate": 4.9359251619630886e-05,
+      "loss": 0.3434,
+      "num_input_tokens_seen": 1035880,
+      "step": 39
+    },
+    {
+      "epoch": 1.4423963133640554,
+      "grad_norm": 0.12086673080921173,
+      "learning_rate": 4.9326121764495596e-05,
+      "loss": 0.3588,
+      "num_input_tokens_seen": 1056488,
+      "step": 40
+    },
+    {
+      "epoch": 1.4792626728110598,
+      "grad_norm": 0.11408794671297073,
+      "learning_rate": 4.9292168560522014e-05,
+      "loss": 0.4021,
+      "num_input_tokens_seen": 1080080,
+      "step": 41
+    },
+    {
+      "epoch": 1.5161290322580645,
+      "grad_norm": 0.1417722851037979,
+      "learning_rate": 4.925739315689991e-05,
+      "loss": 0.3343,
+      "num_input_tokens_seen": 1095632,
+      "step": 42
+    },
+    {
+      "epoch": 1.5529953917050692,
+      "grad_norm": 0.2315663844347,
+      "learning_rate": 4.9221796730647516e-05,
+      "loss": 0.4025,
+      "num_input_tokens_seen": 1126040,
+      "step": 43
+    },
+    {
+      "epoch": 1.5898617511520738,
+      "grad_norm": 0.07856712490320206,
+      "learning_rate": 4.9185380486571595e-05,
+      "loss": 0.3337,
+      "num_input_tokens_seen": 1156656,
+      "step": 44
+    },
+    {
+      "epoch": 1.6267281105990783,
+      "grad_norm": 0.10064279288053513,
+      "learning_rate": 4.914814565722671e-05,
+      "loss": 0.3758,
+      "num_input_tokens_seen": 1183400,
+      "step": 45
+    },
+    {
+      "epoch": 1.663594470046083,
+      "grad_norm": 0.106510229408741,
+      "learning_rate": 4.9110093502873476e-05,
+      "loss": 0.3363,
+      "num_input_tokens_seen": 1209680,
+      "step": 46
+    },
+    {
+      "epoch": 1.7004608294930876,
+      "grad_norm": 0.10446175187826157,
+      "learning_rate": 4.907122531143594e-05,
+      "loss": 0.3853,
+      "num_input_tokens_seen": 1237864,
+      "step": 47
+    },
+    {
+      "epoch": 1.737327188940092,
+      "grad_norm": 0.2570022940635681,
+      "learning_rate": 4.9031542398457974e-05,
+      "loss": 0.3194,
+      "num_input_tokens_seen": 1273080,
+      "step": 48
+    },
+    {
+      "epoch": 1.7741935483870968,
+      "grad_norm": 0.0968039259314537,
+      "learning_rate": 4.8991046107058735e-05,
+      "loss": 0.2955,
+      "num_input_tokens_seen": 1300256,
+      "step": 49
+    },
+    {
+      "epoch": 1.8110599078341014,
+      "grad_norm": 0.11750692129135132,
+      "learning_rate": 4.894973780788722e-05,
+      "loss": 0.3314,
+      "num_input_tokens_seen": 1322616,
+      "step": 50
+    },
+    {
+      "epoch": 1.8479262672811059,
+      "grad_norm": 0.09568070620298386,
+      "learning_rate": 4.890761889907589e-05,
+      "loss": 0.3341,
+      "num_input_tokens_seen": 1353552,
+      "step": 51
+    },
+    {
+      "epoch": 1.8847926267281108,
+      "grad_norm": 0.11816362291574478,
+      "learning_rate": 4.88646908061933e-05,
+      "loss": 0.3202,
+      "num_input_tokens_seen": 1376456,
+      "step": 52
+    },
+    {
+      "epoch": 1.9216589861751152,
+      "grad_norm": 0.133891761302948,
+      "learning_rate": 4.8820954982195905e-05,
+      "loss": 0.2852,
+      "num_input_tokens_seen": 1398344,
+      "step": 53
+    },
+    {
+      "epoch": 1.9585253456221197,
+      "grad_norm": 0.11985364556312561,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.3413,
+      "num_input_tokens_seen": 1424264,
+      "step": 54
+    },
+    {
+      "epoch": 1.9953917050691246,
+      "grad_norm": 0.09828098118305206,
+      "learning_rate": 4.873106608932585e-05,
+      "loss": 0.2726,
+      "num_input_tokens_seen": 1460688,
+      "step": 55
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.31861889362335205,
+      "learning_rate": 4.868491606285823e-05,
+      "loss": 0.3512,
+      "num_input_tokens_seen": 1463952,
+      "step": 56
+    },
+    {
+      "epoch": 2.0368663594470044,
+      "grad_norm": 0.11089354753494263,
+      "learning_rate": 4.8637964389982926e-05,
+      "loss": 0.2953,
+      "num_input_tokens_seen": 1492600,
+      "step": 57
+    },
+    {
+      "epoch": 2.0737327188940093,
+      "grad_norm": 0.12034712731838226,
+      "learning_rate": 4.859021265983959e-05,
+      "loss": 0.2875,
+      "num_input_tokens_seen": 1514496,
+      "step": 58
+    },
+    {
+      "epoch": 2.110599078341014,
+      "grad_norm": 0.1372830867767334,
+      "learning_rate": 4.854166248864689e-05,
+      "loss": 0.3535,
+      "num_input_tokens_seen": 1541656,
+      "step": 59
+    },
+    {
+      "epoch": 2.1474654377880182,
+      "grad_norm": 0.10605650395154953,
+      "learning_rate": 4.849231551964771e-05,
+      "loss": 0.3013,
+      "num_input_tokens_seen": 1569672,
+      "step": 60
+    },
+    {
+      "epoch": 2.184331797235023,
+      "grad_norm": 0.12524856626987457,
+      "learning_rate": 4.844217342305363e-05,
+      "loss": 0.2558,
+      "num_input_tokens_seen": 1599304,
+      "step": 61
+    },
+    {
+      "epoch": 2.2211981566820276,
+      "grad_norm": 0.1151009052991867,
+      "learning_rate": 4.839123789598829e-05,
+      "loss": 0.2666,
+      "num_input_tokens_seen": 1628288,
+      "step": 62
+    },
+    {
+      "epoch": 2.258064516129032,
+      "grad_norm": 0.10297228395938873,
+      "learning_rate": 4.8339510662430046e-05,
+      "loss": 0.2982,
+      "num_input_tokens_seen": 1656384,
+      "step": 63
+    },
+    {
+      "epoch": 2.294930875576037,
+      "grad_norm": 0.11971966177225113,
+      "learning_rate": 4.828699347315356e-05,
+      "loss": 0.3746,
+      "num_input_tokens_seen": 1679344,
+      "step": 64
+    },
+    {
+      "epoch": 2.3317972350230414,
+      "grad_norm": 0.12384694814682007,
+      "learning_rate": 4.823368810567056e-05,
+      "loss": 0.2605,
+      "num_input_tokens_seen": 1703904,
+      "step": 65
+    },
+    {
+      "epoch": 2.3686635944700463,
+      "grad_norm": 0.14720793068408966,
+      "learning_rate": 4.817959636416969e-05,
+      "loss": 0.2816,
+      "num_input_tokens_seen": 1722360,
+      "step": 66
+    },
+    {
+      "epoch": 2.4055299539170507,
+      "grad_norm": 0.10553745925426483,
+      "learning_rate": 4.81247200794554e-05,
+      "loss": 0.2502,
+      "num_input_tokens_seen": 1752464,
+      "step": 67
+    },
+    {
+      "epoch": 2.442396313364055,
+      "grad_norm": 0.11162563413381577,
+      "learning_rate": 4.806906110888606e-05,
+      "loss": 0.2286,
+      "num_input_tokens_seen": 1782744,
+      "step": 68
+    },
+    {
+      "epoch": 2.47926267281106,
+      "grad_norm": 0.12341686338186264,
+      "learning_rate": 4.8012621336311016e-05,
+      "loss": 0.299,
+      "num_input_tokens_seen": 1815120,
+      "step": 69
+    },
+    {
+      "epoch": 2.5161290322580645,
+      "grad_norm": 0.12033814936876297,
+      "learning_rate": 4.7955402672006854e-05,
+      "loss": 0.2609,
+      "num_input_tokens_seen": 1840408,
+      "step": 70
+    },
+    {
+      "epoch": 2.5529953917050694,
+      "grad_norm": 0.11412363499403,
+      "learning_rate": 4.789740705261278e-05,
+      "loss": 0.3316,
+      "num_input_tokens_seen": 1873000,
+      "step": 71
+    },
+    {
+      "epoch": 2.589861751152074,
+      "grad_norm": 0.18193566799163818,
+      "learning_rate": 4.783863644106502e-05,
+      "loss": 0.2773,
+      "num_input_tokens_seen": 1900920,
+      "step": 72
+    },
+    {
+      "epoch": 2.6267281105990783,
+      "grad_norm": 0.14569184184074402,
+      "learning_rate": 4.777909282653042e-05,
+      "loss": 0.2644,
+      "num_input_tokens_seen": 1919272,
+      "step": 73
+    },
+    {
+      "epoch": 2.6635944700460827,
+      "grad_norm": 0.12456360459327698,
+      "learning_rate": 4.771877822433911e-05,
+      "loss": 0.2974,
+      "num_input_tokens_seen": 1959152,
+      "step": 74
+    },
+    {
+      "epoch": 2.7004608294930876,
+      "grad_norm": 0.1362839788198471,
+      "learning_rate": 4.765769467591625e-05,
+      "loss": 0.2743,
+      "num_input_tokens_seen": 1982960,
+      "step": 75
+    },
+    {
+      "epoch": 2.737327188940092,
+      "grad_norm": 0.15120629966259003,
+      "learning_rate": 4.759584424871302e-05,
+      "loss": 0.2425,
+      "num_input_tokens_seen": 2006736,
+      "step": 76
+    },
+    {
+      "epoch": 2.774193548387097,
+      "grad_norm": 0.15145516395568848,
+      "learning_rate": 4.7533229036136553e-05,
+      "loss": 0.2723,
+      "num_input_tokens_seen": 2031536,
+      "step": 77
+    },
+    {
+      "epoch": 2.8110599078341014,
+      "grad_norm": 0.1607382595539093,
+      "learning_rate": 4.7469851157479177e-05,
+      "loss": 0.2525,
+      "num_input_tokens_seen": 2053168,
+      "step": 78
+    },
+    {
+      "epoch": 2.847926267281106,
+      "grad_norm": 0.14976347982883453,
+      "learning_rate": 4.740571275784659e-05,
+      "loss": 0.2792,
+      "num_input_tokens_seen": 2078208,
+      "step": 79
+    },
+    {
+      "epoch": 2.8847926267281108,
+      "grad_norm": 0.12997782230377197,
+      "learning_rate": 4.734081600808531e-05,
+      "loss": 0.291,
+      "num_input_tokens_seen": 2101560,
+      "step": 80
+    },
+    {
+      "epoch": 2.921658986175115,
+      "grad_norm": 0.1369074434041977,
+      "learning_rate": 4.72751631047092e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2127824,
+      "step": 81
+    },
+    {
+      "epoch": 2.9585253456221197,
+      "grad_norm": 0.11734048277139664,
+      "learning_rate": 4.7208756269825104e-05,
+      "loss": 0.2442,
+      "num_input_tokens_seen": 2171536,
+      "step": 82
+    },
+    {
+      "epoch": 2.9953917050691246,
+      "grad_norm": 0.14909496903419495,
+      "learning_rate": 4.714159775105765e-05,
+      "loss": 0.2344,
+      "num_input_tokens_seen": 2193016,
+      "step": 83
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.40910854935646057,
+      "learning_rate": 4.707368982147318e-05,
+      "loss": 0.2645,
+      "num_input_tokens_seen": 2195928,
+      "step": 84
+    },
+    {
+      "epoch": 3.0368663594470044,
+      "grad_norm": 0.1433151513338089,
+      "learning_rate": 4.700503477950278e-05,
+      "loss": 0.2185,
+      "num_input_tokens_seen": 2219472,
+      "step": 85
+    },
+    {
+      "epoch": 3.0737327188940093,
+      "grad_norm": 0.13229826092720032,
+      "learning_rate": 4.693563494886455e-05,
+      "loss": 0.2364,
+      "num_input_tokens_seen": 2246504,
+      "step": 86
+    },
+    {
+      "epoch": 3.110599078341014,
+      "grad_norm": 0.1702108383178711,
+      "learning_rate": 4.6865492678484895e-05,
+      "loss": 0.2703,
+      "num_input_tokens_seen": 2270304,
+      "step": 87
+    },
+    {
+      "epoch": 3.1474654377880182,
+      "grad_norm": 0.1411813497543335,
+      "learning_rate": 4.679461034241906e-05,
+      "loss": 0.2807,
+      "num_input_tokens_seen": 2299968,
+      "step": 88
+    },
+    {
+      "epoch": 3.184331797235023,
+      "grad_norm": 0.12941044569015503,
+      "learning_rate": 4.672299033977076e-05,
+      "loss": 0.2182,
+      "num_input_tokens_seen": 2333432,
+      "step": 89
+    },
+    {
+      "epoch": 3.2211981566820276,
+      "grad_norm": 0.1649884432554245,
+      "learning_rate": 4.665063509461097e-05,
+      "loss": 0.2572,
+      "num_input_tokens_seen": 2355320,
+      "step": 90
+    },
+    {
+      "epoch": 3.258064516129032,
+      "grad_norm": 0.14386451244354248,
+      "learning_rate": 4.657754705589591e-05,
+      "loss": 0.2412,
+      "num_input_tokens_seen": 2378416,
+      "step": 91
+    },
+    {
+      "epoch": 3.294930875576037,
+      "grad_norm": 0.15350477397441864,
+      "learning_rate": 4.650372869738414e-05,
+      "loss": 0.242,
+      "num_input_tokens_seen": 2405432,
+      "step": 92
+    },
+    {
+      "epoch": 3.3317972350230414,
+      "grad_norm": 0.16243231296539307,
+      "learning_rate": 4.642918251755281e-05,
+      "loss": 0.2436,
+      "num_input_tokens_seen": 2429992,
+      "step": 93
+    },
+    {
+      "epoch": 3.3686635944700463,
+      "grad_norm": 0.18401454389095306,
+      "learning_rate": 4.6353911039513145e-05,
+      "loss": 0.2538,
+      "num_input_tokens_seen": 2456264,
+      "step": 94
+    },
+    {
+      "epoch": 3.4055299539170507,
+      "grad_norm": 0.13898858428001404,
+      "learning_rate": 4.627791681092499e-05,
+      "loss": 0.2065,
+      "num_input_tokens_seen": 2486032,
+      "step": 95
+    },
+    {
+      "epoch": 3.442396313364055,
+      "grad_norm": 0.16943225264549255,
+      "learning_rate": 4.620120240391065e-05,
+      "loss": 0.2086,
+      "num_input_tokens_seen": 2509272,
+      "step": 96
+    },
+    {
+      "epoch": 3.47926267281106,
+      "grad_norm": 0.15286582708358765,
+      "learning_rate": 4.612377041496776e-05,
+      "loss": 0.2517,
+      "num_input_tokens_seen": 2538392,
+      "step": 97
+    },
+    {
+      "epoch": 3.5161290322580645,
+      "grad_norm": 0.17737072706222534,
+      "learning_rate": 4.604562346488144e-05,
+      "loss": 0.2413,
+      "num_input_tokens_seen": 2562624,
+      "step": 98
+    },
+    {
+      "epoch": 3.5529953917050694,
+      "grad_norm": 0.13630996644496918,
+      "learning_rate": 4.5966764198635606e-05,
+      "loss": 0.2157,
+      "num_input_tokens_seen": 2595448,
+      "step": 99
+    },
+    {
+      "epoch": 3.589861751152074,
+      "grad_norm": 0.13918158411979675,
+      "learning_rate": 4.588719528532342e-05,
+      "loss": 0.2319,
+      "num_input_tokens_seen": 2633568,
+      "step": 100
+    },
+    {
+      "epoch": 3.6267281105990783,
+      "grad_norm": 0.16548505425453186,
+      "learning_rate": 4.580691941805695e-05,
+      "loss": 0.2473,
+      "num_input_tokens_seen": 2659408,
+      "step": 101
+    },
+    {
+      "epoch": 3.6635944700460827,
+      "grad_norm": 0.17397943139076233,
+      "learning_rate": 4.572593931387604e-05,
+      "loss": 0.2527,
+      "num_input_tokens_seen": 2687272,
+      "step": 102
+    },
+    {
+      "epoch": 3.7004608294930876,
+      "grad_norm": 0.15970823168754578,
+      "learning_rate": 4.5644257713656356e-05,
+      "loss": 0.22,
+      "num_input_tokens_seen": 2718232,
+      "step": 103
+    },
+    {
+      "epoch": 3.737327188940092,
+      "grad_norm": 0.16440241038799286,
+      "learning_rate": 4.556187738201656e-05,
+      "loss": 0.2374,
+      "num_input_tokens_seen": 2743336,
+      "step": 104
+    },
+    {
+      "epoch": 3.774193548387097,
+      "grad_norm": 0.15761925280094147,
+      "learning_rate": 4.54788011072248e-05,
+      "loss": 0.2023,
+      "num_input_tokens_seen": 2771384,
+      "step": 105
+    },
+    {
+      "epoch": 3.8110599078341014,
+      "grad_norm": 0.1528206616640091,
+      "learning_rate": 4.539503170110431e-05,
+      "loss": 0.2289,
+      "num_input_tokens_seen": 2797488,
+      "step": 106
+    },
+    {
+      "epoch": 3.847926267281106,
+      "grad_norm": 0.16380636394023895,
+      "learning_rate": 4.531057199893824e-05,
+      "loss": 0.2492,
+      "num_input_tokens_seen": 2825408,
+      "step": 107
+    },
+    {
+      "epoch": 3.8847926267281108,
+      "grad_norm": 0.19076082110404968,
+      "learning_rate": 4.522542485937369e-05,
+      "loss": 0.2467,
+      "num_input_tokens_seen": 2848856,
+      "step": 108
+    },
+    {
+      "epoch": 3.921658986175115,
+      "grad_norm": 0.1884787231683731,
+      "learning_rate": 4.5139593164324986e-05,
+      "loss": 0.2069,
+      "num_input_tokens_seen": 2867536,
+      "step": 109
+    },
+    {
+      "epoch": 3.9585253456221197,
+      "grad_norm": 0.19939164817333221,
+      "learning_rate": 4.50530798188761e-05,
+      "loss": 0.1843,
+      "num_input_tokens_seen": 2892032,
+      "step": 110
+    },
+    {
+      "epoch": 3.9953917050691246,
+      "grad_norm": 0.17019645869731903,
+      "learning_rate": 4.496588775118232e-05,
+      "loss": 0.2576,
+      "num_input_tokens_seen": 2924800,
+      "step": 111
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.41787323355674744,
+      "learning_rate": 4.48780199123712e-05,
+      "loss": 0.2734,
+      "num_input_tokens_seen": 2927904,
+      "step": 112
+    },
+    {
+      "epoch": 4.0368663594470044,
+      "grad_norm": 0.18536736071109772,
+      "learning_rate": 4.478947927644258e-05,
+      "loss": 0.2363,
+      "num_input_tokens_seen": 2954728,
+      "step": 113
+    },
+    {
+      "epoch": 4.073732718894009,
+      "grad_norm": 0.16922569274902344,
+      "learning_rate": 4.4700268840168045e-05,
+      "loss": 0.1888,
+      "num_input_tokens_seen": 2983656,
+      "step": 114
+    },
+    {
+      "epoch": 4.110599078341014,
+      "grad_norm": 0.17613869905471802,
+      "learning_rate": 4.4610391622989396e-05,
+      "loss": 0.2077,
+      "num_input_tokens_seen": 3014208,
+      "step": 115
+    },
+    {
+      "epoch": 4.147465437788019,
+      "grad_norm": 0.16198807954788208,
+      "learning_rate": 4.4519850666916484e-05,
+      "loss": 0.1858,
+      "num_input_tokens_seen": 3040976,
+      "step": 116
+    },
+    {
+      "epoch": 4.184331797235023,
+      "grad_norm": 0.19020891189575195,
+      "learning_rate": 4.442864903642428e-05,
+      "loss": 0.2284,
+      "num_input_tokens_seen": 3066744,
+      "step": 117
+    },
+    {
+      "epoch": 4.221198156682028,
+      "grad_norm": 0.21541331708431244,
+      "learning_rate": 4.43367898183491e-05,
+      "loss": 0.1856,
+      "num_input_tokens_seen": 3090320,
+      "step": 118
+    },
+    {
+      "epoch": 4.258064516129032,
+      "grad_norm": 0.19018961489200592,
+      "learning_rate": 4.4244276121784195e-05,
+      "loss": 0.1788,
+      "num_input_tokens_seen": 3114576,
+      "step": 119
+    },
+    {
+      "epoch": 4.2949308755760365,
+      "grad_norm": 0.24048063158988953,
+      "learning_rate": 4.415111107797445e-05,
+      "loss": 0.2155,
+      "num_input_tokens_seen": 3135552,
+      "step": 120
+    },
+    {
+      "epoch": 4.331797235023042,
+      "grad_norm": 0.17166386544704437,
+      "learning_rate": 4.405729784021046e-05,
+      "loss": 0.1831,
+      "num_input_tokens_seen": 3165168,
+      "step": 121
+    },
+    {
+      "epoch": 4.368663594470046,
+      "grad_norm": 0.18577668070793152,
+      "learning_rate": 4.396283958372173e-05,
+      "loss": 0.2168,
+      "num_input_tokens_seen": 3200240,
+      "step": 122
+    },
+    {
+      "epoch": 4.405529953917051,
+      "grad_norm": 0.20443667471408844,
+      "learning_rate": 4.386773950556931e-05,
+      "loss": 0.1978,
+      "num_input_tokens_seen": 3225024,
+      "step": 123
+    },
+    {
+      "epoch": 4.442396313364055,
+      "grad_norm": 0.17636562883853912,
+      "learning_rate": 4.377200082453749e-05,
+      "loss": 0.1727,
+      "num_input_tokens_seen": 3251696,
+      "step": 124
+    },
+    {
+      "epoch": 4.47926267281106,
+      "grad_norm": 0.2019859254360199,
+      "learning_rate": 4.36756267810249e-05,
+      "loss": 0.198,
+      "num_input_tokens_seen": 3275672,
+      "step": 125
+    },
+    {
+      "epoch": 4.516129032258064,
+      "grad_norm": 0.2633707523345947,
+      "learning_rate": 4.357862063693486e-05,
+      "loss": 0.2254,
+      "num_input_tokens_seen": 3305592,
+      "step": 126
+    },
+    {
+      "epoch": 4.552995391705069,
+      "grad_norm": 0.17379336059093475,
+      "learning_rate": 4.34809856755649e-05,
+      "loss": 0.2467,
+      "num_input_tokens_seen": 3345072,
+      "step": 127
+    },
+    {
+      "epoch": 4.589861751152074,
+      "grad_norm": 0.16954895853996277,
+      "learning_rate": 4.3382725201495723e-05,
+      "loss": 0.1616,
+      "num_input_tokens_seen": 3371832,
+      "step": 128
+    },
+    {
+      "epoch": 4.626728110599078,
+      "grad_norm": 0.21207046508789062,
+      "learning_rate": 4.3283842540479264e-05,
+      "loss": 0.189,
+      "num_input_tokens_seen": 3395912,
+      "step": 129
+    },
+    {
+      "epoch": 4.663594470046083,
+      "grad_norm": 0.40675088763237,
+      "learning_rate": 4.318434103932622e-05,
+      "loss": 0.1739,
+      "num_input_tokens_seen": 3423352,
+      "step": 130
+    },
+    {
+      "epoch": 4.700460829493087,
+      "grad_norm": 0.21301084756851196,
+      "learning_rate": 4.30842240657927e-05,
+      "loss": 0.187,
+      "num_input_tokens_seen": 3447992,
+      "step": 131
+    },
+    {
+      "epoch": 4.7373271889400925,
+      "grad_norm": 0.20403216779232025,
+      "learning_rate": 4.2983495008466276e-05,
+      "loss": 0.1786,
+      "num_input_tokens_seen": 3479752,
+      "step": 132
+    },
+    {
+      "epoch": 4.774193548387097,
+      "grad_norm": 0.1969008892774582,
+      "learning_rate": 4.288215727665129e-05,
+      "loss": 0.214,
+      "num_input_tokens_seen": 3509072,
+      "step": 133
+    },
+    {
+      "epoch": 4.811059907834101,
+      "grad_norm": 0.20199979841709137,
+      "learning_rate": 4.278021430025343e-05,
+      "loss": 0.171,
+      "num_input_tokens_seen": 3531848,
+      "step": 134
+    },
+    {
+      "epoch": 4.847926267281106,
+      "grad_norm": 0.2074364721775055,
+      "learning_rate": 4.267766952966369e-05,
+      "loss": 0.1786,
+      "num_input_tokens_seen": 3557144,
+      "step": 135
+    },
+    {
+      "epoch": 4.88479262672811,
+      "grad_norm": 0.23071159422397614,
+      "learning_rate": 4.257452643564155e-05,
+      "loss": 0.1816,
+      "num_input_tokens_seen": 3578656,
+      "step": 136
+    },
+    {
+      "epoch": 4.921658986175116,
+      "grad_norm": 0.2532529830932617,
+      "learning_rate": 4.24707885091975e-05,
+      "loss": 0.1885,
+      "num_input_tokens_seen": 3608632,
+      "step": 137
+    },
+    {
+      "epoch": 4.95852534562212,
+      "grad_norm": 0.24982504546642303,
+      "learning_rate": 4.2366459261474933e-05,
+      "loss": 0.2125,
+      "num_input_tokens_seen": 3628672,
+      "step": 138
+    },
+    {
+      "epoch": 4.9953917050691246,
+      "grad_norm": 0.17841289937496185,
+      "learning_rate": 4.226154222363124e-05,
+      "loss": 0.2001,
+      "num_input_tokens_seen": 3657088,
+      "step": 139
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.6343802809715271,
+      "learning_rate": 4.215604094671835e-05,
+      "loss": 0.2376,
+      "num_input_tokens_seen": 3659880,
+      "step": 140
+    },
+    {
+      "epoch": 5.0368663594470044,
+      "grad_norm": 0.21179749071598053,
+      "learning_rate": 4.2049959001562464e-05,
+      "loss": 0.1942,
+      "num_input_tokens_seen": 3684760,
+      "step": 141
+    },
+    {
+      "epoch": 5.073732718894009,
+      "grad_norm": 0.22715945541858673,
+      "learning_rate": 4.194329997864331e-05,
+      "loss": 0.1849,
+      "num_input_tokens_seen": 3715176,
+      "step": 142
+    },
+    {
+      "epoch": 5.110599078341014,
+      "grad_norm": 0.20638404786586761,
+      "learning_rate": 4.183606748797251e-05,
+      "loss": 0.1492,
+      "num_input_tokens_seen": 3741480,
+      "step": 143
+    },
+    {
+      "epoch": 5.147465437788019,
+      "grad_norm": 0.25307852029800415,
+      "learning_rate": 4.172826515897146e-05,
+      "loss": 0.1699,
+      "num_input_tokens_seen": 3762976,
+      "step": 144
+    },
+    {
+      "epoch": 5.184331797235023,
+      "grad_norm": 0.21923506259918213,
+      "learning_rate": 4.1619896640348445e-05,
+      "loss": 0.1559,
+      "num_input_tokens_seen": 3794720,
+      "step": 145
+    },
+    {
+      "epoch": 5.221198156682028,
+      "grad_norm": 0.2600756883621216,
+      "learning_rate": 4.1510965599975196e-05,
+      "loss": 0.1843,
+      "num_input_tokens_seen": 3822744,
+      "step": 146
+    },
+    {
+      "epoch": 5.258064516129032,
+      "grad_norm": 0.2997046113014221,
+      "learning_rate": 4.140147572476268e-05,
+      "loss": 0.243,
+      "num_input_tokens_seen": 3845584,
+      "step": 147
+    },
+    {
+      "epoch": 5.2949308755760365,
+      "grad_norm": 0.2147158831357956,
+      "learning_rate": 4.129143072053638e-05,
+      "loss": 0.1476,
+      "num_input_tokens_seen": 3869184,
+      "step": 148
+    },
+    {
+      "epoch": 5.331797235023042,
+      "grad_norm": 0.26199281215667725,
+      "learning_rate": 4.118083431191081e-05,
+      "loss": 0.1619,
+      "num_input_tokens_seen": 3900384,
+      "step": 149
+    },
+    {
+      "epoch": 5.368663594470046,
+      "grad_norm": 0.4091247618198395,
+      "learning_rate": 4.1069690242163484e-05,
+      "loss": 0.1943,
+      "num_input_tokens_seen": 3923792,
+      "step": 150
+    },
+    {
+      "epoch": 5.405529953917051,
+      "grad_norm": 0.23406103253364563,
+      "learning_rate": 4.095800227310821e-05,
+      "loss": 0.1403,
+      "num_input_tokens_seen": 3942784,
+      "step": 151
+    },
+    {
+      "epoch": 5.442396313364055,
+      "grad_norm": 0.2293683886528015,
+      "learning_rate": 4.0845774184967754e-05,
+      "loss": 0.1756,
+      "num_input_tokens_seen": 3970880,
+      "step": 152
+    },
+    {
+      "epoch": 5.47926267281106,
+      "grad_norm": 0.1963953971862793,
+      "learning_rate": 4.073300977624594e-05,
+      "loss": 0.1785,
+      "num_input_tokens_seen": 3998976,
+      "step": 153
+    },
+    {
+      "epoch": 5.516129032258064,
+      "grad_norm": 0.23071235418319702,
+      "learning_rate": 4.0619712863599e-05,
+      "loss": 0.165,
+      "num_input_tokens_seen": 4020272,
+      "step": 154
+    },
+    {
+      "epoch": 5.552995391705069,
+      "grad_norm": 0.2570018172264099,
+      "learning_rate": 4.05058872817065e-05,
+      "loss": 0.1639,
+      "num_input_tokens_seen": 4043288,
+      "step": 155
+    },
+    {
+      "epoch": 5.589861751152074,
+      "grad_norm": 0.2581011950969696,
+      "learning_rate": 4.039153688314145e-05,
+      "loss": 0.1818,
+      "num_input_tokens_seen": 4067832,
+      "step": 156
+    },
+    {
+      "epoch": 5.626728110599078,
+      "grad_norm": 0.23090755939483643,
+      "learning_rate": 4.0276665538239996e-05,
+      "loss": 0.1372,
+      "num_input_tokens_seen": 4093672,
+      "step": 157
+    },
+    {
+      "epoch": 5.663594470046083,
+      "grad_norm": 0.217911958694458,
+      "learning_rate": 4.0161277134970345e-05,
+      "loss": 0.1648,
+      "num_input_tokens_seen": 4119400,
+      "step": 158
+    },
+    {
+      "epoch": 5.700460829493087,
+      "grad_norm": 0.3220454454421997,
+      "learning_rate": 4.0045375578801214e-05,
+      "loss": 0.1516,
+      "num_input_tokens_seen": 4148144,
+      "step": 159
+    },
+    {
+      "epoch": 5.7373271889400925,
+      "grad_norm": 0.2835656404495239,
+      "learning_rate": 3.9928964792569655e-05,
+      "loss": 0.1597,
+      "num_input_tokens_seen": 4169456,
+      "step": 160
+    },
+    {
+      "epoch": 5.774193548387097,
+      "grad_norm": 0.22007523477077484,
+      "learning_rate": 3.981204871634827e-05,
+      "loss": 0.1535,
+      "num_input_tokens_seen": 4207704,
+      "step": 161
+    },
+    {
+      "epoch": 5.811059907834101,
+      "grad_norm": 0.23791228234767914,
+      "learning_rate": 3.969463130731183e-05,
+      "loss": 0.1472,
+      "num_input_tokens_seen": 4230304,
+      "step": 162
+    },
+    {
+      "epoch": 5.847926267281106,
+      "grad_norm": 0.21866248548030853,
+      "learning_rate": 3.957671653960337e-05,
+      "loss": 0.1362,
+      "num_input_tokens_seen": 4255536,
+      "step": 163
+    },
+    {
+      "epoch": 5.88479262672811,
+      "grad_norm": 0.24037593603134155,
+      "learning_rate": 3.945830840419966e-05,
+      "loss": 0.1443,
+      "num_input_tokens_seen": 4285160,
+      "step": 164
+    },
+    {
+      "epoch": 5.921658986175116,
+      "grad_norm": 0.19541749358177185,
+      "learning_rate": 3.933941090877615e-05,
+      "loss": 0.1647,
+      "num_input_tokens_seen": 4323272,
+      "step": 165
+    },
+    {
+      "epoch": 5.95852534562212,
+      "grad_norm": 0.22034280002117157,
+      "learning_rate": 3.9220028077571295e-05,
+      "loss": 0.1921,
+      "num_input_tokens_seen": 4354912,
+      "step": 166
+    },
+    {
+      "epoch": 5.9953917050691246,
+      "grad_norm": 0.19790567457675934,
+      "learning_rate": 3.910016395125037e-05,
+      "loss": 0.1522,
+      "num_input_tokens_seen": 4387064,
+      "step": 167
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 0.6411383152008057,
+      "learning_rate": 3.897982258676867e-05,
+      "loss": 0.2109,
+      "num_input_tokens_seen": 4391856,
+      "step": 168
+    },
+    {
+      "epoch": 6.0368663594470044,
+      "grad_norm": 0.22729864716529846,
+      "learning_rate": 3.885900805723429e-05,
+      "loss": 0.1607,
+      "num_input_tokens_seen": 4419952,
+      "step": 169
+    },
+    {
+      "epoch": 6.073732718894009,
+      "grad_norm": 0.21639327704906464,
+      "learning_rate": 3.873772445177015e-05,
+      "loss": 0.1277,
+      "num_input_tokens_seen": 4442576,
+      "step": 170
+    },
+    {
+      "epoch": 6.110599078341014,
+      "grad_norm": 0.2176467776298523,
+      "learning_rate": 3.861597587537568e-05,
+      "loss": 0.121,
+      "num_input_tokens_seen": 4477568,
+      "step": 171
+    },
+    {
+      "epoch": 6.147465437788019,
+      "grad_norm": 0.21243643760681152,
+      "learning_rate": 3.8493766448787825e-05,
+      "loss": 0.1976,
+      "num_input_tokens_seen": 4509944,
+      "step": 172
+    },
+    {
+      "epoch": 6.184331797235023,
+      "grad_norm": 0.2905130386352539,
+      "learning_rate": 3.837110030834161e-05,
+      "loss": 0.141,
+      "num_input_tokens_seen": 4534704,
+      "step": 173
+    },
+    {
+      "epoch": 6.221198156682028,
+      "grad_norm": 0.2976662218570709,
+      "learning_rate": 3.824798160583012e-05,
+      "loss": 0.1389,
+      "num_input_tokens_seen": 4563912,
+      "step": 174
+    },
+    {
+      "epoch": 6.258064516129032,
+      "grad_norm": 0.2367183119058609,
+      "learning_rate": 3.8124414508364e-05,
+      "loss": 0.1241,
+      "num_input_tokens_seen": 4594712,
+      "step": 175
+    },
+    {
+      "epoch": 6.2949308755760365,
+      "grad_norm": 0.23760217428207397,
+      "learning_rate": 3.8000403198230387e-05,
+      "loss": 0.1415,
+      "num_input_tokens_seen": 4621448,
+      "step": 176
+    },
+    {
+      "epoch": 6.331797235023042,
+      "grad_norm": 0.23512595891952515,
+      "learning_rate": 3.787595187275136e-05,
+      "loss": 0.1277,
+      "num_input_tokens_seen": 4648744,
+      "step": 177
+    },
+    {
+      "epoch": 6.368663594470046,
+      "grad_norm": 0.24028612673282623,
+      "learning_rate": 3.775106474414188e-05,
+      "loss": 0.1636,
+      "num_input_tokens_seen": 4679480,
+      "step": 178
+    },
+    {
+      "epoch": 6.405529953917051,
+      "grad_norm": 0.2704714238643646,
+      "learning_rate": 3.762574603936725e-05,
+      "loss": 0.1341,
+      "num_input_tokens_seen": 4698440,
+      "step": 179
+    },
+    {
+      "epoch": 6.442396313364055,
+      "grad_norm": 0.27304500341415405,
+      "learning_rate": 3.7500000000000003e-05,
+      "loss": 0.1472,
+      "num_input_tokens_seen": 4721120,
+      "step": 180
+    },
+    {
+      "epoch": 6.47926267281106,
+      "grad_norm": 0.2529843747615814,
+      "learning_rate": 3.7373830882076354e-05,
+      "loss": 0.1344,
+      "num_input_tokens_seen": 4743536,
+      "step": 181
+    },
+    {
+      "epoch": 6.516129032258064,
+      "grad_norm": 0.23612789809703827,
+      "learning_rate": 3.7247242955952175e-05,
+      "loss": 0.1255,
+      "num_input_tokens_seen": 4773248,
+      "step": 182
+    },
+    {
+      "epoch": 6.552995391705069,
+      "grad_norm": 0.2793615162372589,
+      "learning_rate": 3.712024050615843e-05,
+      "loss": 0.141,
+      "num_input_tokens_seen": 4804600,
+      "step": 183
+    },
+    {
+      "epoch": 6.589861751152074,
+      "grad_norm": 0.29646047949790955,
+      "learning_rate": 3.699282783125616e-05,
+      "loss": 0.1226,
+      "num_input_tokens_seen": 4826744,
+      "step": 184
+    },
+    {
+      "epoch": 6.626728110599078,
+      "grad_norm": 0.25531846284866333,
+      "learning_rate": 3.686500924369101e-05,
+      "loss": 0.1074,
+      "num_input_tokens_seen": 4849584,
+      "step": 185
+    },
+    {
+      "epoch": 6.663594470046083,
+      "grad_norm": 0.24337461590766907,
+      "learning_rate": 3.673678906964727e-05,
+      "loss": 0.1084,
+      "num_input_tokens_seen": 4875336,
+      "step": 186
+    },
+    {
+      "epoch": 6.700460829493087,
+      "grad_norm": 0.30051150918006897,
+      "learning_rate": 3.660817164890143e-05,
+      "loss": 0.1273,
+      "num_input_tokens_seen": 4895896,
+      "step": 187
+    },
+    {
+      "epoch": 6.7373271889400925,
+      "grad_norm": 0.286495178937912,
+      "learning_rate": 3.6479161334675296e-05,
+      "loss": 0.1447,
+      "num_input_tokens_seen": 4924360,
+      "step": 188
+    },
+    {
+      "epoch": 6.774193548387097,
+      "grad_norm": 0.28501835465431213,
+      "learning_rate": 3.634976249348867e-05,
+      "loss": 0.1548,
+      "num_input_tokens_seen": 4948384,
+      "step": 189
+    },
+    {
+      "epoch": 6.811059907834101,
+      "grad_norm": 0.24231503903865814,
+      "learning_rate": 3.621997950501156e-05,
+      "loss": 0.1507,
+      "num_input_tokens_seen": 4980368,
+      "step": 190
+    },
+    {
+      "epoch": 6.847926267281106,
+      "grad_norm": 0.2752607464790344,
+      "learning_rate": 3.6089816761915906e-05,
+      "loss": 0.1282,
+      "num_input_tokens_seen": 5007744,
+      "step": 191
+    },
+    {
+      "epoch": 6.88479262672811,
+      "grad_norm": 0.27704674005508423,
+      "learning_rate": 3.5959278669726935e-05,
+      "loss": 0.1411,
+      "num_input_tokens_seen": 5030528,
+      "step": 192
+    },
+    {
+      "epoch": 6.921658986175116,
+      "grad_norm": 0.22601065039634705,
+      "learning_rate": 3.582836964667408e-05,
+      "loss": 0.1156,
+      "num_input_tokens_seen": 5065000,
+      "step": 193
+    },
+    {
+      "epoch": 6.95852534562212,
+      "grad_norm": 0.24917051196098328,
+      "learning_rate": 3.569709412354136e-05,
+      "loss": 0.1143,
+      "num_input_tokens_seen": 5091440,
+      "step": 194
+    },
+    {
+      "epoch": 6.9953917050691246,
+      "grad_norm": 0.31986546516418457,
+      "learning_rate": 3.556545654351749e-05,
+      "loss": 0.1466,
+      "num_input_tokens_seen": 5120064,
+      "step": 195
+    },
+    {
+      "epoch": 7.0,
+      "grad_norm": 0.6616023778915405,
+      "learning_rate": 3.543346136204545e-05,
+      "loss": 0.1637,
+      "num_input_tokens_seen": 5123832,
+      "step": 196
+    },
+    {
+      "epoch": 7.0368663594470044,
+      "grad_norm": 0.2567199766635895,
+      "learning_rate": 3.5301113046671714e-05,
+      "loss": 0.1111,
+      "num_input_tokens_seen": 5154136,
+      "step": 197
+    },
+    {
+      "epoch": 7.073732718894009,
+      "grad_norm": 0.24546091258525848,
+      "learning_rate": 3.516841607689501e-05,
+      "loss": 0.1015,
+      "num_input_tokens_seen": 5182792,
+      "step": 198
+    },
+    {
+      "epoch": 7.110599078341014,
+      "grad_norm": 0.25169137120246887,
+      "learning_rate": 3.503537494401473e-05,
+      "loss": 0.1091,
+      "num_input_tokens_seen": 5208280,
+      "step": 199
+    },
+    {
+      "epoch": 7.147465437788019,
+      "grad_norm": 0.27022022008895874,
+      "learning_rate": 3.490199415097892e-05,
+      "loss": 0.1165,
+      "num_input_tokens_seen": 5226024,
+      "step": 200
+    },
+    {
+      "epoch": 7.184331797235023,
+      "grad_norm": 0.2491355836391449,
+      "learning_rate": 3.476827821223184e-05,
+      "loss": 0.1047,
+      "num_input_tokens_seen": 5259096,
+      "step": 201
+    },
+    {
+      "epoch": 7.221198156682028,
+      "grad_norm": 0.28957781195640564,
+      "learning_rate": 3.463423165356121e-05,
+      "loss": 0.1066,
+      "num_input_tokens_seen": 5282232,
+      "step": 202
+    },
+    {
+      "epoch": 7.258064516129032,
+      "grad_norm": 0.29709964990615845,
+      "learning_rate": 3.449985901194498e-05,
+      "loss": 0.0902,
+      "num_input_tokens_seen": 5306272,
+      "step": 203
+    },
+    {
+      "epoch": 7.2949308755760365,
+      "grad_norm": 0.3285166919231415,
+      "learning_rate": 3.436516483539781e-05,
+      "loss": 0.1072,
+      "num_input_tokens_seen": 5330832,
+      "step": 204
+    },
+    {
+      "epoch": 7.331797235023042,
+      "grad_norm": 0.24284234642982483,
+      "learning_rate": 3.423015368281711e-05,
+      "loss": 0.1195,
+      "num_input_tokens_seen": 5359992,
+      "step": 205
+    },
+    {
+      "epoch": 7.368663594470046,
+      "grad_norm": 0.2531001567840576,
+      "learning_rate": 3.409483012382879e-05,
+      "loss": 0.1038,
+      "num_input_tokens_seen": 5392232,
+      "step": 206
+    },
+    {
+      "epoch": 7.405529953917051,
+      "grad_norm": 0.273573637008667,
+      "learning_rate": 3.39591987386325e-05,
+      "loss": 0.109,
+      "num_input_tokens_seen": 5416272,
+      "step": 207
+    },
+    {
+      "epoch": 7.442396313364055,
+      "grad_norm": 0.28594842553138733,
+      "learning_rate": 3.382326411784672e-05,
+      "loss": 0.1035,
+      "num_input_tokens_seen": 5442312,
+      "step": 208
+    },
+    {
+      "epoch": 7.47926267281106,
+      "grad_norm": 0.24845322966575623,
+      "learning_rate": 3.3687030862353286e-05,
+      "loss": 0.0997,
+      "num_input_tokens_seen": 5473408,
+      "step": 209
+    },
+    {
+      "epoch": 7.516129032258064,
+      "grad_norm": 0.2672761082649231,
+      "learning_rate": 3.355050358314172e-05,
+      "loss": 0.1034,
+      "num_input_tokens_seen": 5494256,
+      "step": 210
+    },
+    {
+      "epoch": 7.552995391705069,
+      "grad_norm": 0.29590561985969543,
+      "learning_rate": 3.3413686901153165e-05,
+      "loss": 0.0994,
+      "num_input_tokens_seen": 5519384,
+      "step": 211
+    },
+    {
+      "epoch": 7.589861751152074,
+      "grad_norm": 0.2702927887439728,
+      "learning_rate": 3.327658544712395e-05,
+      "loss": 0.1197,
+      "num_input_tokens_seen": 5544760,
+      "step": 212
+    },
+    {
+      "epoch": 7.626728110599078,
+      "grad_norm": 0.2904987633228302,
+      "learning_rate": 3.313920386142892e-05,
+      "loss": 0.1423,
+      "num_input_tokens_seen": 5574216,
+      "step": 213
+    },
+    {
+      "epoch": 7.663594470046083,
+      "grad_norm": 0.27985090017318726,
+      "learning_rate": 3.3001546793924285e-05,
+      "loss": 0.1171,
+      "num_input_tokens_seen": 5602072,
+      "step": 214
+    },
+    {
+      "epoch": 7.700460829493087,
+      "grad_norm": 0.25102686882019043,
+      "learning_rate": 3.2863618903790346e-05,
+      "loss": 0.1118,
+      "num_input_tokens_seen": 5632056,
+      "step": 215
+    },
+    {
+      "epoch": 7.7373271889400925,
+      "grad_norm": 0.3189446032047272,
+      "learning_rate": 3.272542485937369e-05,
+      "loss": 0.105,
+      "num_input_tokens_seen": 5658216,
+      "step": 216
+    },
+    {
+      "epoch": 7.774193548387097,
+      "grad_norm": 0.30590710043907166,
+      "learning_rate": 3.2586969338029274e-05,
+      "loss": 0.1083,
+      "num_input_tokens_seen": 5684712,
+      "step": 217
+    },
+    {
+      "epoch": 7.811059907834101,
+      "grad_norm": 0.3227143883705139,
+      "learning_rate": 3.244825702596205e-05,
+      "loss": 0.1477,
+      "num_input_tokens_seen": 5709472,
+      "step": 218
+    },
+    {
+      "epoch": 7.847926267281106,
+      "grad_norm": 0.25323161482810974,
+      "learning_rate": 3.230929261806842e-05,
+      "loss": 0.1057,
+      "num_input_tokens_seen": 5742728,
+      "step": 219
+    },
+    {
+      "epoch": 7.88479262672811,
+      "grad_norm": 0.27679508924484253,
+      "learning_rate": 3.217008081777726e-05,
+      "loss": 0.1094,
+      "num_input_tokens_seen": 5768848,
+      "step": 220
+    },
+    {
+      "epoch": 7.921658986175116,
+      "grad_norm": 0.3211810886859894,
+      "learning_rate": 3.203062633689077e-05,
+      "loss": 0.1092,
+      "num_input_tokens_seen": 5795008,
+      "step": 221
+    },
+    {
+      "epoch": 7.95852534562212,
+      "grad_norm": 0.3296988904476166,
+      "learning_rate": 3.1890933895424976e-05,
+      "loss": 0.1253,
+      "num_input_tokens_seen": 5818160,
+      "step": 222
+    },
+    {
+      "epoch": 7.9953917050691246,
+      "grad_norm": 0.2729380130767822,
+      "learning_rate": 3.1751008221450025e-05,
+      "loss": 0.1264,
+      "num_input_tokens_seen": 5850472,
+      "step": 223
+    },
+    {
+      "epoch": 8.0,
+      "grad_norm": 0.6995195150375366,
+      "learning_rate": 3.161085405093006e-05,
+      "loss": 0.1218,
+      "num_input_tokens_seen": 5855808,
+      "step": 224
+    },
+    {
+      "epoch": 8.036866359447005,
+      "grad_norm": 0.3297252357006073,
+      "learning_rate": 3.147047612756302e-05,
+      "loss": 0.0946,
+      "num_input_tokens_seen": 5884832,
+      "step": 225
+    },
+    {
+      "epoch": 8.073732718894009,
+      "grad_norm": 0.2687251567840576,
+      "learning_rate": 3.132987920262005e-05,
+      "loss": 0.0889,
+      "num_input_tokens_seen": 5903392,
+      "step": 226
+    },
+    {
+      "epoch": 8.110599078341014,
+      "grad_norm": 0.2826682925224304,
+      "learning_rate": 3.118906803478465e-05,
+      "loss": 0.1084,
+      "num_input_tokens_seen": 5928240,
+      "step": 227
+    },
+    {
+      "epoch": 8.147465437788018,
+      "grad_norm": 0.27045705914497375,
+      "learning_rate": 3.104804738999169e-05,
+      "loss": 0.1134,
+      "num_input_tokens_seen": 5958392,
+      "step": 228
+    },
+    {
+      "epoch": 8.184331797235023,
+      "grad_norm": 0.2565663754940033,
+      "learning_rate": 3.090682204126604e-05,
+      "loss": 0.0929,
+      "num_input_tokens_seen": 5984088,
+      "step": 229
+    },
+    {
+      "epoch": 8.221198156682028,
+      "grad_norm": 0.28110557794570923,
+      "learning_rate": 3.076539676856101e-05,
+      "loss": 0.0922,
+      "num_input_tokens_seen": 6007936,
+      "step": 230
+    },
+    {
+      "epoch": 8.258064516129032,
+      "grad_norm": 0.2717958092689514,
+      "learning_rate": 3.062377635859663e-05,
+      "loss": 0.072,
+      "num_input_tokens_seen": 6033336,
+      "step": 231
+    },
+    {
+      "epoch": 8.294930875576037,
+      "grad_norm": 0.3095947802066803,
+      "learning_rate": 3.048196560469758e-05,
+      "loss": 0.1176,
+      "num_input_tokens_seen": 6060160,
+      "step": 232
+    },
+    {
+      "epoch": 8.331797235023041,
+      "grad_norm": 0.3245174288749695,
+      "learning_rate": 3.0339969306631005e-05,
+      "loss": 0.0886,
+      "num_input_tokens_seen": 6088280,
+      "step": 233
+    },
+    {
+      "epoch": 8.368663594470046,
+      "grad_norm": 0.2761123776435852,
+      "learning_rate": 3.0197792270443982e-05,
+      "loss": 0.0831,
+      "num_input_tokens_seen": 6119752,
+      "step": 234
+    },
+    {
+      "epoch": 8.40552995391705,
+      "grad_norm": 0.33236628770828247,
+      "learning_rate": 3.0055439308300952e-05,
+      "loss": 0.1222,
+      "num_input_tokens_seen": 6154672,
+      "step": 235
+    },
+    {
+      "epoch": 8.442396313364055,
+      "grad_norm": 0.2539371848106384,
+      "learning_rate": 2.9912915238320754e-05,
+      "loss": 0.0856,
+      "num_input_tokens_seen": 6181152,
+      "step": 236
+    },
+    {
+      "epoch": 8.47926267281106,
+      "grad_norm": 0.25849923491477966,
+      "learning_rate": 2.9770224884413623e-05,
+      "loss": 0.0842,
+      "num_input_tokens_seen": 6208248,
+      "step": 237
+    },
+    {
+      "epoch": 8.516129032258064,
+      "grad_norm": 0.2990070879459381,
+      "learning_rate": 2.9627373076117863e-05,
+      "loss": 0.0816,
+      "num_input_tokens_seen": 6232704,
+      "step": 238
+    },
+    {
+      "epoch": 8.55299539170507,
+      "grad_norm": 0.2482941746711731,
+      "learning_rate": 2.9484364648436437e-05,
+      "loss": 0.0725,
+      "num_input_tokens_seen": 6264600,
+      "step": 239
+    },
+    {
+      "epoch": 8.589861751152073,
+      "grad_norm": 0.31631267070770264,
+      "learning_rate": 2.9341204441673266e-05,
+      "loss": 0.0917,
+      "num_input_tokens_seen": 6289416,
+      "step": 240
+    },
+    {
+      "epoch": 8.626728110599078,
+      "grad_norm": 0.32536354660987854,
+      "learning_rate": 2.9197897301269435e-05,
+      "loss": 0.1029,
+      "num_input_tokens_seen": 6313624,
+      "step": 241
+    },
+    {
+      "epoch": 8.663594470046084,
+      "grad_norm": 0.28533875942230225,
+      "learning_rate": 2.905444807763919e-05,
+      "loss": 0.0876,
+      "num_input_tokens_seen": 6337560,
+      "step": 242
+    },
+    {
+      "epoch": 8.700460829493087,
+      "grad_norm": 0.32342368364334106,
+      "learning_rate": 2.8910861626005776e-05,
+      "loss": 0.081,
+      "num_input_tokens_seen": 6362016,
+      "step": 243
+    },
+    {
+      "epoch": 8.737327188940093,
+      "grad_norm": 0.2809208333492279,
+      "learning_rate": 2.876714280623708e-05,
+      "loss": 0.0846,
+      "num_input_tokens_seen": 6399368,
+      "step": 244
+    },
+    {
+      "epoch": 8.774193548387096,
+      "grad_norm": 0.2525993883609772,
+      "learning_rate": 2.8623296482681166e-05,
+      "loss": 0.0865,
+      "num_input_tokens_seen": 6428808,
+      "step": 245
+    },
+    {
+      "epoch": 8.811059907834101,
+      "grad_norm": 0.331150084733963,
+      "learning_rate": 2.8479327524001636e-05,
+      "loss": 0.1033,
+      "num_input_tokens_seen": 6450424,
+      "step": 246
+    },
+    {
+      "epoch": 8.847926267281107,
+      "grad_norm": 0.3234056532382965,
+      "learning_rate": 2.833524080301282e-05,
+      "loss": 0.0758,
+      "num_input_tokens_seen": 6477040,
+      "step": 247
+    },
+    {
+      "epoch": 8.88479262672811,
+      "grad_norm": 0.3200438320636749,
+      "learning_rate": 2.8191041196514873e-05,
+      "loss": 0.102,
+      "num_input_tokens_seen": 6504424,
+      "step": 248
+    },
+    {
+      "epoch": 8.921658986175116,
+      "grad_norm": 0.3212469220161438,
+      "learning_rate": 2.8046733585128687e-05,
+      "loss": 0.0812,
+      "num_input_tokens_seen": 6528144,
+      "step": 249
+    },
+    {
+      "epoch": 8.95852534562212,
+      "grad_norm": 0.37497377395629883,
+      "learning_rate": 2.7902322853130757e-05,
+      "loss": 0.0767,
+      "num_input_tokens_seen": 6549528,
+      "step": 250
+    },
+    {
+      "epoch": 8.995391705069125,
+      "grad_norm": 0.27313902974128723,
+      "learning_rate": 2.7757813888287798e-05,
+      "loss": 0.0734,
+      "num_input_tokens_seen": 6577392,
+      "step": 251
+    },
+    {
+      "epoch": 9.0,
+      "grad_norm": 0.4119221270084381,
+      "learning_rate": 2.761321158169134e-05,
+      "loss": 0.0718,
+      "num_input_tokens_seen": 6587784,
+      "step": 252
+    },
+    {
+      "epoch": 9.036866359447005,
+      "grad_norm": 0.23572459816932678,
+      "learning_rate": 2.7468520827592197e-05,
+      "loss": 0.0589,
+      "num_input_tokens_seen": 6614752,
+      "step": 253
+    },
+    {
+      "epoch": 9.073732718894009,
+      "grad_norm": 0.22828935086727142,
+      "learning_rate": 2.732374652323481e-05,
+      "loss": 0.0682,
+      "num_input_tokens_seen": 6648864,
+      "step": 254
+    },
+    {
+      "epoch": 9.110599078341014,
+      "grad_norm": 0.28416910767555237,
+      "learning_rate": 2.717889356869146e-05,
+      "loss": 0.0768,
+      "num_input_tokens_seen": 6678136,
+      "step": 255
+    },
+    {
+      "epoch": 9.147465437788018,
+      "grad_norm": 0.3248147666454315,
+      "learning_rate": 2.7033966866696457e-05,
+      "loss": 0.0739,
+      "num_input_tokens_seen": 6704880,
+      "step": 256
+    },
+    {
+      "epoch": 9.184331797235023,
+      "grad_norm": 0.2697289288043976,
+      "learning_rate": 2.6888971322480205e-05,
+      "loss": 0.0884,
+      "num_input_tokens_seen": 6742080,
+      "step": 257
+    },
+    {
+      "epoch": 9.221198156682028,
+      "grad_norm": 0.3310103714466095,
+      "learning_rate": 2.674391184360313e-05,
+      "loss": 0.0836,
+      "num_input_tokens_seen": 6771256,
+      "step": 258
+    },
+    {
+      "epoch": 9.258064516129032,
+      "grad_norm": 0.3416021168231964,
+      "learning_rate": 2.659879333978964e-05,
+      "loss": 0.0728,
+      "num_input_tokens_seen": 6798392,
+      "step": 259
+    },
+    {
+      "epoch": 9.294930875576037,
+      "grad_norm": 0.25257182121276855,
+      "learning_rate": 2.6453620722761896e-05,
+      "loss": 0.0682,
+      "num_input_tokens_seen": 6830648,
+      "step": 260
+    },
+    {
+      "epoch": 9.331797235023041,
+      "grad_norm": 0.31290990114212036,
+      "learning_rate": 2.63083989060736e-05,
+      "loss": 0.0701,
+      "num_input_tokens_seen": 6853800,
+      "step": 261
+    },
+    {
+      "epoch": 9.368663594470046,
+      "grad_norm": 0.28703242540359497,
+      "learning_rate": 2.6163132804943675e-05,
+      "loss": 0.0641,
+      "num_input_tokens_seen": 6880096,
+      "step": 262
+    },
+    {
+      "epoch": 9.40552995391705,
+      "grad_norm": 0.33095675706863403,
+      "learning_rate": 2.60178273360899e-05,
+      "loss": 0.0695,
+      "num_input_tokens_seen": 6905400,
+      "step": 263
+    },
+    {
+      "epoch": 9.442396313364055,
+      "grad_norm": 0.3316652774810791,
+      "learning_rate": 2.587248741756253e-05,
+      "loss": 0.0993,
+      "num_input_tokens_seen": 6933352,
+      "step": 264
+    },
+    {
+      "epoch": 9.47926267281106,
+      "grad_norm": 0.3296186029911041,
+      "learning_rate": 2.5727117968577784e-05,
+      "loss": 0.0702,
+      "num_input_tokens_seen": 6955160,
+      "step": 265
+    },
+    {
+      "epoch": 9.516129032258064,
+      "grad_norm": 0.288353830575943,
+      "learning_rate": 2.5581723909351406e-05,
+      "loss": 0.0632,
+      "num_input_tokens_seen": 6976256,
+      "step": 266
+    },
+    {
+      "epoch": 9.55299539170507,
+      "grad_norm": 0.34519389271736145,
+      "learning_rate": 2.5436310160932092e-05,
+      "loss": 0.0873,
+      "num_input_tokens_seen": 7002472,
+      "step": 267
+    },
+    {
+      "epoch": 9.589861751152073,
+      "grad_norm": 0.2644766569137573,
+      "learning_rate": 2.5290881645034932e-05,
+      "loss": 0.0515,
+      "num_input_tokens_seen": 7034352,
+      "step": 268
+    },
+    {
+      "epoch": 9.626728110599078,
+      "grad_norm": 0.34003907442092896,
+      "learning_rate": 2.5145443283874848e-05,
+      "loss": 0.0679,
+      "num_input_tokens_seen": 7062184,
+      "step": 269
+    },
+    {
+      "epoch": 9.663594470046084,
+      "grad_norm": 0.24440152943134308,
+      "learning_rate": 2.5e-05,
+      "loss": 0.0649,
+      "num_input_tokens_seen": 7090600,
+      "step": 270
+    },
+    {
+      "epoch": 9.700460829493087,
+      "grad_norm": 0.38781893253326416,
+      "learning_rate": 2.485455671612515e-05,
+      "loss": 0.0843,
+      "num_input_tokens_seen": 7112656,
+      "step": 271
+    },
+    {
+      "epoch": 9.737327188940093,
+      "grad_norm": 0.35135138034820557,
+      "learning_rate": 2.470911835496508e-05,
+      "loss": 0.0709,
+      "num_input_tokens_seen": 7134448,
+      "step": 272
+    },
+    {
+      "epoch": 9.774193548387096,
+      "grad_norm": 0.3243424594402313,
+      "learning_rate": 2.4563689839067913e-05,
+      "loss": 0.0566,
+      "num_input_tokens_seen": 7159000,
+      "step": 273
+    },
+    {
+      "epoch": 9.811059907834101,
+      "grad_norm": 0.2978050708770752,
+      "learning_rate": 2.4418276090648596e-05,
+      "loss": 0.0783,
+      "num_input_tokens_seen": 7193008,
+      "step": 274
+    },
+    {
+      "epoch": 9.847926267281107,
+      "grad_norm": 0.3357181251049042,
+      "learning_rate": 2.4272882031422215e-05,
+      "loss": 0.0755,
+      "num_input_tokens_seen": 7214720,
+      "step": 275
+    },
+    {
+      "epoch": 9.88479262672811,
+      "grad_norm": 0.3311111032962799,
+      "learning_rate": 2.4127512582437485e-05,
+      "loss": 0.0756,
+      "num_input_tokens_seen": 7236408,
+      "step": 276
+    },
+    {
+      "epoch": 9.921658986175116,
+      "grad_norm": 0.34926944971084595,
+      "learning_rate": 2.3982172663910108e-05,
+      "loss": 0.0774,
+      "num_input_tokens_seen": 7254272,
+      "step": 277
+    },
+    {
+      "epoch": 9.95852534562212,
+      "grad_norm": 0.27301025390625,
+      "learning_rate": 2.3836867195056335e-05,
+      "loss": 0.0665,
+      "num_input_tokens_seen": 7285272,
+      "step": 278
+    },
+    {
+      "epoch": 9.995391705069125,
+      "grad_norm": 0.31272560358047485,
+      "learning_rate": 2.3691601093926404e-05,
+      "loss": 0.075,
+      "num_input_tokens_seen": 7318112,
+      "step": 279
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 1.0591803789138794,
+      "learning_rate": 2.3546379277238107e-05,
+      "loss": 0.0886,
+      "num_input_tokens_seen": 7319760,
+      "step": 280
+    },
+    {
+      "epoch": 10.036866359447005,
+      "grad_norm": 0.2831885814666748,
+      "learning_rate": 2.3401206660210363e-05,
+      "loss": 0.0562,
+      "num_input_tokens_seen": 7343240,
+      "step": 281
+    },
+    {
+      "epoch": 10.073732718894009,
+      "grad_norm": 0.2866736650466919,
+      "learning_rate": 2.3256088156396868e-05,
+      "loss": 0.0519,
+      "num_input_tokens_seen": 7366928,
+      "step": 282
+    },
+    {
+      "epoch": 10.110599078341014,
+      "grad_norm": 0.28110256791114807,
+      "learning_rate": 2.3111028677519804e-05,
+      "loss": 0.0493,
+      "num_input_tokens_seen": 7391576,
+      "step": 283
+    },
+    {
+      "epoch": 10.147465437788018,
+      "grad_norm": 0.2599363327026367,
+      "learning_rate": 2.2966033133303545e-05,
+      "loss": 0.064,
+      "num_input_tokens_seen": 7422848,
+      "step": 284
+    },
+    {
+      "epoch": 10.184331797235023,
+      "grad_norm": 0.3725353181362152,
+      "learning_rate": 2.2821106431308544e-05,
+      "loss": 0.0876,
+      "num_input_tokens_seen": 7445112,
+      "step": 285
+    },
+    {
+      "epoch": 10.221198156682028,
+      "grad_norm": 0.31412675976753235,
+      "learning_rate": 2.2676253476765196e-05,
+      "loss": 0.0631,
+      "num_input_tokens_seen": 7469384,
+      "step": 286
+    },
+    {
+      "epoch": 10.258064516129032,
+      "grad_norm": 0.3256174921989441,
+      "learning_rate": 2.2531479172407805e-05,
+      "loss": 0.059,
+      "num_input_tokens_seen": 7493816,
+      "step": 287
+    },
+    {
+      "epoch": 10.294930875576037,
+      "grad_norm": 0.2877194285392761,
+      "learning_rate": 2.238678841830867e-05,
+      "loss": 0.0534,
+      "num_input_tokens_seen": 7523536,
+      "step": 288
+    },
+    {
+      "epoch": 10.331797235023041,
+      "grad_norm": 0.2373494654893875,
+      "learning_rate": 2.2242186111712208e-05,
+      "loss": 0.0479,
+      "num_input_tokens_seen": 7552320,
+      "step": 289
+    },
+    {
+      "epoch": 10.368663594470046,
+      "grad_norm": 0.36851245164871216,
+      "learning_rate": 2.2097677146869242e-05,
+      "loss": 0.0693,
+      "num_input_tokens_seen": 7574816,
+      "step": 290
+    },
+    {
+      "epoch": 10.40552995391705,
+      "grad_norm": 0.32222089171409607,
+      "learning_rate": 2.195326641487132e-05,
+      "loss": 0.0632,
+      "num_input_tokens_seen": 7598688,
+      "step": 291
+    },
+    {
+      "epoch": 10.442396313364055,
+      "grad_norm": 0.33085688948631287,
+      "learning_rate": 2.1808958803485136e-05,
+      "loss": 0.0589,
+      "num_input_tokens_seen": 7620856,
+      "step": 292
+    },
+    {
+      "epoch": 10.47926267281106,
+      "grad_norm": 0.28755688667297363,
+      "learning_rate": 2.1664759196987182e-05,
+      "loss": 0.0666,
+      "num_input_tokens_seen": 7647512,
+      "step": 293
+    },
+    {
+      "epoch": 10.516129032258064,
+      "grad_norm": 0.3906136751174927,
+      "learning_rate": 2.1520672475998373e-05,
+      "loss": 0.0885,
+      "num_input_tokens_seen": 7681488,
+      "step": 294
+    },
+    {
+      "epoch": 10.55299539170507,
+      "grad_norm": 0.2831442058086395,
+      "learning_rate": 2.1376703517318837e-05,
+      "loss": 0.0581,
+      "num_input_tokens_seen": 7707480,
+      "step": 295
+    },
+    {
+      "epoch": 10.589861751152073,
+      "grad_norm": 0.2866826355457306,
+      "learning_rate": 2.1232857193762924e-05,
+      "loss": 0.049,
+      "num_input_tokens_seen": 7733640,
+      "step": 296
+    },
+    {
+      "epoch": 10.626728110599078,
+      "grad_norm": 0.25126680731773376,
+      "learning_rate": 2.1089138373994223e-05,
+      "loss": 0.0476,
+      "num_input_tokens_seen": 7766744,
+      "step": 297
+    },
+    {
+      "epoch": 10.663594470046084,
+      "grad_norm": 0.28204652667045593,
+      "learning_rate": 2.0945551922360818e-05,
+      "loss": 0.0625,
+      "num_input_tokens_seen": 7796144,
+      "step": 298
+    },
+    {
+      "epoch": 10.700460829493087,
+      "grad_norm": 0.32275664806365967,
+      "learning_rate": 2.0802102698730574e-05,
+      "loss": 0.0523,
+      "num_input_tokens_seen": 7814368,
+      "step": 299
+    },
+    {
+      "epoch": 10.737327188940093,
+      "grad_norm": 0.3534482717514038,
+      "learning_rate": 2.0658795558326743e-05,
+      "loss": 0.0484,
+      "num_input_tokens_seen": 7844720,
+      "step": 300
+    },
+    {
+      "epoch": 10.774193548387096,
+      "grad_norm": 0.27784067392349243,
+      "learning_rate": 2.0515635351563565e-05,
+      "loss": 0.0549,
+      "num_input_tokens_seen": 7876192,
+      "step": 301
+    },
+    {
+      "epoch": 10.811059907834101,
+      "grad_norm": 0.3224843442440033,
+      "learning_rate": 2.037262692388214e-05,
+      "loss": 0.0545,
+      "num_input_tokens_seen": 7903336,
+      "step": 302
+    },
+    {
+      "epoch": 10.847926267281107,
+      "grad_norm": 0.4785996973514557,
+      "learning_rate": 2.022977511558638e-05,
+      "loss": 0.0662,
+      "num_input_tokens_seen": 7936544,
+      "step": 303
+    },
+    {
+      "epoch": 10.88479262672811,
+      "grad_norm": 0.2665662169456482,
+      "learning_rate": 2.0087084761679245e-05,
+      "loss": 0.0456,
+      "num_input_tokens_seen": 7966064,
+      "step": 304
+    },
+    {
+      "epoch": 10.921658986175116,
+      "grad_norm": 0.322708398103714,
+      "learning_rate": 1.9944560691699057e-05,
+      "loss": 0.0535,
+      "num_input_tokens_seen": 7989848,
+      "step": 305
+    },
+    {
+      "epoch": 10.95852534562212,
+      "grad_norm": 0.32365888357162476,
+      "learning_rate": 1.980220772955602e-05,
+      "loss": 0.0614,
+      "num_input_tokens_seen": 8020416,
+      "step": 306
+    },
+    {
+      "epoch": 10.995391705069125,
+      "grad_norm": 0.32327035069465637,
+      "learning_rate": 1.9660030693369004e-05,
+      "loss": 0.0456,
+      "num_input_tokens_seen": 8049840,
+      "step": 307
+    },
+    {
+      "epoch": 11.0,
+      "grad_norm": 0.9106179475784302,
+      "learning_rate": 1.9518034395302414e-05,
+      "loss": 0.0734,
+      "num_input_tokens_seen": 8051736,
+      "step": 308
+    },
+    {
+      "epoch": 11.036866359447005,
+      "grad_norm": 0.2582056224346161,
+      "learning_rate": 1.937622364140338e-05,
+      "loss": 0.0362,
+      "num_input_tokens_seen": 8082888,
+      "step": 309
+    },
+    {
+      "epoch": 11.073732718894009,
+      "grad_norm": 0.3024876117706299,
+      "learning_rate": 1.9234603231438995e-05,
+      "loss": 0.0362,
+      "num_input_tokens_seen": 8107768,
+      "step": 310
+    },
+    {
+      "epoch": 11.110599078341014,
+      "grad_norm": 0.2687247097492218,
+      "learning_rate": 1.9093177958733966e-05,
+      "loss": 0.0504,
+      "num_input_tokens_seen": 8149664,
+      "step": 311
+    },
+    {
+      "epoch": 11.147465437788018,
+      "grad_norm": 0.37591755390167236,
+      "learning_rate": 1.895195261000831e-05,
+      "loss": 0.0647,
+      "num_input_tokens_seen": 8172672,
+      "step": 312
+    },
+    {
+      "epoch": 11.184331797235023,
+      "grad_norm": 0.26167264580726624,
+      "learning_rate": 1.8810931965215356e-05,
+      "loss": 0.0391,
+      "num_input_tokens_seen": 8199368,
+      "step": 313
+    },
+    {
+      "epoch": 11.221198156682028,
+      "grad_norm": 0.2515069246292114,
+      "learning_rate": 1.8670120797379958e-05,
+      "loss": 0.0445,
+      "num_input_tokens_seen": 8225816,
+      "step": 314
+    },
+    {
+      "epoch": 11.258064516129032,
+      "grad_norm": 0.32947489619255066,
+      "learning_rate": 1.852952387243698e-05,
+      "loss": 0.0456,
+      "num_input_tokens_seen": 8250688,
+      "step": 315
+    },
+    {
+      "epoch": 11.294930875576037,
+      "grad_norm": 0.3417774438858032,
+      "learning_rate": 1.838914594906995e-05,
+      "loss": 0.0485,
+      "num_input_tokens_seen": 8273448,
+      "step": 316
+    },
+    {
+      "epoch": 11.331797235023041,
+      "grad_norm": 0.3106794059276581,
+      "learning_rate": 1.8248991778549984e-05,
+      "loss": 0.0417,
+      "num_input_tokens_seen": 8295960,
+      "step": 317
+    },
+    {
+      "epoch": 11.368663594470046,
+      "grad_norm": 0.3570440709590912,
+      "learning_rate": 1.8109066104575023e-05,
+      "loss": 0.0459,
+      "num_input_tokens_seen": 8319704,
+      "step": 318
+    },
+    {
+      "epoch": 11.40552995391705,
+      "grad_norm": 0.3612540364265442,
+      "learning_rate": 1.7969373663109234e-05,
+      "loss": 0.0434,
+      "num_input_tokens_seen": 8346544,
+      "step": 319
+    },
+    {
+      "epoch": 11.442396313364055,
+      "grad_norm": 0.3443238437175751,
+      "learning_rate": 1.7829919182222752e-05,
+      "loss": 0.0549,
+      "num_input_tokens_seen": 8373352,
+      "step": 320
+    },
+    {
+      "epoch": 11.47926267281106,
+      "grad_norm": 0.29259899258613586,
+      "learning_rate": 1.7690707381931583e-05,
+      "loss": 0.0326,
+      "num_input_tokens_seen": 8400136,
+      "step": 321
+    },
+    {
+      "epoch": 11.516129032258064,
+      "grad_norm": 0.255287230014801,
+      "learning_rate": 1.755174297403795e-05,
+      "loss": 0.0433,
+      "num_input_tokens_seen": 8435752,
+      "step": 322
+    },
+    {
+      "epoch": 11.55299539170507,
+      "grad_norm": 0.33487167954444885,
+      "learning_rate": 1.7413030661970742e-05,
+      "loss": 0.0451,
+      "num_input_tokens_seen": 8457320,
+      "step": 323
+    },
+    {
+      "epoch": 11.589861751152073,
+      "grad_norm": 0.27473774552345276,
+      "learning_rate": 1.7274575140626318e-05,
+      "loss": 0.0378,
+      "num_input_tokens_seen": 8484320,
+      "step": 324
+    },
+    {
+      "epoch": 11.626728110599078,
+      "grad_norm": 0.26888421177864075,
+      "learning_rate": 1.7136381096209664e-05,
+      "loss": 0.0408,
+      "num_input_tokens_seen": 8508448,
+      "step": 325
+    },
+    {
+      "epoch": 11.663594470046084,
+      "grad_norm": 0.25122973322868347,
+      "learning_rate": 1.699845320607571e-05,
+      "loss": 0.0402,
+      "num_input_tokens_seen": 8541472,
+      "step": 326
+    },
+    {
+      "epoch": 11.700460829493087,
+      "grad_norm": 0.34217768907546997,
+      "learning_rate": 1.686079613857109e-05,
+      "loss": 0.057,
+      "num_input_tokens_seen": 8566672,
+      "step": 327
+    },
+    {
+      "epoch": 11.737327188940093,
+      "grad_norm": 0.31365272402763367,
+      "learning_rate": 1.672341455287605e-05,
+      "loss": 0.054,
+      "num_input_tokens_seen": 8596632,
+      "step": 328
+    },
+    {
+      "epoch": 11.774193548387096,
+      "grad_norm": 0.2996351420879364,
+      "learning_rate": 1.658631309884684e-05,
+      "loss": 0.0512,
+      "num_input_tokens_seen": 8627480,
+      "step": 329
+    },
+    {
+      "epoch": 11.811059907834101,
+      "grad_norm": 0.37132295966148376,
+      "learning_rate": 1.6449496416858284e-05,
+      "loss": 0.0775,
+      "num_input_tokens_seen": 8656960,
+      "step": 330
+    },
+    {
+      "epoch": 11.847926267281107,
+      "grad_norm": 0.3101523816585541,
+      "learning_rate": 1.6312969137646716e-05,
+      "loss": 0.0547,
+      "num_input_tokens_seen": 8682816,
+      "step": 331
+    },
+    {
+      "epoch": 11.88479262672811,
+      "grad_norm": 0.3477034866809845,
+      "learning_rate": 1.617673588215328e-05,
+      "loss": 0.0519,
+      "num_input_tokens_seen": 8704760,
+      "step": 332
+    },
+    {
+      "epoch": 11.921658986175116,
+      "grad_norm": 0.26099440455436707,
+      "learning_rate": 1.6040801261367493e-05,
+      "loss": 0.0402,
+      "num_input_tokens_seen": 8731512,
+      "step": 333
+    },
+    {
+      "epoch": 11.95852534562212,
+      "grad_norm": 0.29072248935699463,
+      "learning_rate": 1.5905169876171223e-05,
+      "loss": 0.0422,
+      "num_input_tokens_seen": 8757016,
+      "step": 334
+    },
+    {
+      "epoch": 11.995391705069125,
+      "grad_norm": 0.294264018535614,
+      "learning_rate": 1.5769846317182893e-05,
+      "loss": 0.0595,
+      "num_input_tokens_seen": 8782184,
+      "step": 335
+    },
+    {
+      "epoch": 12.0,
+      "grad_norm": 1.3008415699005127,
+      "learning_rate": 1.56348351646022e-05,
+      "loss": 0.062,
+      "num_input_tokens_seen": 8783712,
+      "step": 336
+    },
+    {
+      "epoch": 12.036866359447005,
+      "grad_norm": 0.2572445869445801,
+      "learning_rate": 1.550014098805503e-05,
+      "loss": 0.0341,
+      "num_input_tokens_seen": 8810408,
+      "step": 337
+    },
+    {
+      "epoch": 12.073732718894009,
+      "grad_norm": 0.2236901968717575,
+      "learning_rate": 1.5365768346438797e-05,
+      "loss": 0.0343,
+      "num_input_tokens_seen": 8838328,
+      "step": 338
+    },
+    {
+      "epoch": 12.110599078341014,
+      "grad_norm": 0.26900872588157654,
+      "learning_rate": 1.523172178776816e-05,
+      "loss": 0.0366,
+      "num_input_tokens_seen": 8867416,
+      "step": 339
+    },
+    {
+      "epoch": 12.147465437788018,
+      "grad_norm": 0.26596930623054504,
+      "learning_rate": 1.509800584902108e-05,
+      "loss": 0.0363,
+      "num_input_tokens_seen": 8893096,
+      "step": 340
+    },
+    {
+      "epoch": 12.184331797235023,
+      "grad_norm": 0.27418944239616394,
+      "learning_rate": 1.4964625055985265e-05,
+      "loss": 0.0296,
+      "num_input_tokens_seen": 8915936,
+      "step": 341
+    },
+    {
+      "epoch": 12.221198156682028,
+      "grad_norm": 0.33651241660118103,
+      "learning_rate": 1.4831583923104999e-05,
+      "loss": 0.0377,
+      "num_input_tokens_seen": 8941560,
+      "step": 342
+    },
+    {
+      "epoch": 12.258064516129032,
+      "grad_norm": 0.24450698494911194,
+      "learning_rate": 1.4698886953328292e-05,
+      "loss": 0.0355,
+      "num_input_tokens_seen": 8971584,
+      "step": 343
+    },
+    {
+      "epoch": 12.294930875576037,
+      "grad_norm": 0.27189865708351135,
+      "learning_rate": 1.4566538637954554e-05,
+      "loss": 0.0333,
+      "num_input_tokens_seen": 9000944,
+      "step": 344
+    },
+    {
+      "epoch": 12.331797235023041,
+      "grad_norm": 0.32970282435417175,
+      "learning_rate": 1.443454345648252e-05,
+      "loss": 0.0443,
+      "num_input_tokens_seen": 9025720,
+      "step": 345
+    },
+    {
+      "epoch": 12.368663594470046,
+      "grad_norm": 0.39073866605758667,
+      "learning_rate": 1.430290587645865e-05,
+      "loss": 0.0505,
+      "num_input_tokens_seen": 9046184,
+      "step": 346
+    },
+    {
+      "epoch": 12.40552995391705,
+      "grad_norm": 0.351654052734375,
+      "learning_rate": 1.4171630353325932e-05,
+      "loss": 0.0655,
+      "num_input_tokens_seen": 9072816,
+      "step": 347
+    },
+    {
+      "epoch": 12.442396313364055,
+      "grad_norm": 0.2837273180484772,
+      "learning_rate": 1.4040721330273062e-05,
+      "loss": 0.0323,
+      "num_input_tokens_seen": 9105192,
+      "step": 348
+    },
+    {
+      "epoch": 12.47926267281106,
+      "grad_norm": 0.3341452479362488,
+      "learning_rate": 1.3910183238084112e-05,
+      "loss": 0.0458,
+      "num_input_tokens_seen": 9126232,
+      "step": 349
+    },
+    {
+      "epoch": 12.516129032258064,
+      "grad_norm": 0.29763755202293396,
+      "learning_rate": 1.3780020494988446e-05,
+      "loss": 0.038,
+      "num_input_tokens_seen": 9147168,
+      "step": 350
+    },
+    {
+      "epoch": 12.55299539170507,
+      "grad_norm": 0.2759022116661072,
+      "learning_rate": 1.3650237506511331e-05,
+      "loss": 0.0343,
+      "num_input_tokens_seen": 9174584,
+      "step": 351
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 540,
+  "num_input_tokens_seen": 9174584,
+  "num_train_epochs": 20,
+  "save_steps": 27,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.739832508665037e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-378/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-378/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-378/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-378/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-378/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}