Existance/CIS_qlora_summarization_model

README.md

@@ -1,7 +1,7 @@
 ---
 library_name: peft
-license: apache-2.0
-base_model: Qwen/Qwen2.5-0.5B
+license: llama3.2
+base_model: meta-llama/Llama-3.2-3B
 tags:
 - generated_from_trainer
 model-index:
@@ -14,9 +14,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # qlora_summarization
 
-This model is a fine-tuned version of [Qwen/Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) on an unknown dataset.
+This model is a fine-tuned version of [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 8.3791
+- Loss: 7.7306
 
 ## Model description
 
@@ -39,7 +39,7 @@ The following hyperparameters were used during training:
 - train_batch_size: 2
 - eval_batch_size: 2
 - seed: 42
-- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
 - num_epochs: 3
 - mixed_precision_training: Native AMP
@@ -48,15 +48,15 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 8.4191        | 1.0   | 2025 | 8.4432          |
-| 8.5048        | 2.0   | 4050 | 8.3952          |
-| 8.3704        | 3.0   | 6075 | 8.3791          |
+| 7.6614        | 1.0   | 2025 | 7.7951          |
+| 7.7233        | 2.0   | 4050 | 7.7536          |
+| 7.4209        | 3.0   | 6075 | 7.7306          |
 
 
 ### Framework versions
 
 - PEFT 0.14.0
-- Transformers 4.47.0
-- Pytorch 2.5.1+cu121
-- Datasets 3.3.1
-- Tokenizers 0.21.0
+- Transformers 4.50.0
+- Pytorch 2.6.0+cu124
+- Datasets 3.5.0
+- Tokenizers 0.21.1