taronklm/Qwen2.5-0.5B-Instruct-chatbot

README.md

@@ -37,7 +37,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-05
+- learning_rate: 1e-05
 - train_batch_size: 1
 - eval_batch_size: 8
 - seed: 42
@@ -45,7 +45,7 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 16
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 2
+- num_epochs: 3
 
 ### Training results
 

adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "up_proj",
     "o_proj",
-    "q_proj",
-    "v_proj",
+    "up_proj",
     "k_proj",
-    "down_proj",
-    "gate_proj"
+    "v_proj",
+    "gate_proj",
+    "q_proj",
+    "down_proj"
   ],
   "task_type": "CASUAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cfbc2159c7fbe53e83476546a5700cb2ddb1714027cc026901ef6ce4d1ee8ff4
+oid sha256:2a675bde7dd92fbe4b85b6acde91ab31912b0803415bf5c768ba33fbf0c8e861
 size 8841592

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5fc0a24dce134d9086d64b21071e6dc2df5e075c250426c99b98eba31a13f8fa
+oid sha256:001b207091593c0b9f318ff006bb10d7b8a80e3d8650241075cd155095c4024f
 size 1976163472

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3147a44829aaeded52568334e274104f25be6f4800bb0f2cb2455ee55cb3d918
+oid sha256:a60392f1478e9b3ed1b32c89ae12e2ac0e14a4f9114ceae947f601344d2dfac3
 size 5368