jerseyjerry/task-17-Qwen-Qwen2.5-3B-Instruct

lora

This model is a fine-tuned version of Qwen/Qwen2.5-3B-Instruct on the flock_task17_train dataset. It achieves the following results on the evaluation set:

• Loss: 0.0124

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 2
• total_eval_batch_size: 2
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss
0.1096	0.0513	2	0.1662
0.2943	0.1026	4	0.1425
0.0325	0.1538	6	0.1153
0.3053	0.2051	8	0.0859
0.254	0.2564	10	0.0618
0.0264	0.3077	12	0.0434
0.1261	0.3590	14	0.0319
0.023	0.4103	16	0.0274
0.0062	0.4615	18	0.0281
0.0117	0.5128	20	0.0197
0.008	0.5641	22	0.0190
0.0125	0.6154	24	0.0163
0.0384	0.6667	26	0.0167
0.0283	0.7179	28	0.0150
0.0602	0.7692	30	0.0120
0.0253	0.8205	32	0.0113
0.0119	0.8718	34	0.0114
0.0005	0.9231	36	0.0121
0.0008	0.9744	38	0.0133
0.0281	1.0256	40	0.0121
0.0101	1.0769	42	0.0115
0.0257	1.1282	44	0.0114
0.0207	1.1795	46	0.0114
0.0013	1.2308	48	0.0113
0.0316	1.2821	50	0.0116
0.0016	1.3333	52	0.0117
0.0001	1.3846	54	0.0120
0.0001	1.4359	56	0.0122
0.0088	1.4872	58	0.0123
0.0308	1.5385	60	0.0124
0.0031	1.5897	62	0.0122
0.0334	1.6410	64	0.0127
0.0215	1.6923	66	0.0126
0.0406	1.7436	68	0.0121
0.0157	1.7949	70	0.0117
0.0084	1.8462	72	0.0115
0.0121	1.8974	74	0.0115
0.0025	1.9487	76	0.0116
0.0006	2.0	78	0.0115
0.0104	2.0513	80	0.0117
0.0001	2.1026	82	0.0119
0.0094	2.1538	84	0.0120
0.0013	2.2051	86	0.0124
0.0074	2.2564	88	0.0125
0.0002	2.3077	90	0.0125
0.0051	2.3590	92	0.0128
0.0009	2.4103	94	0.0126
0.0002	2.4615	96	0.0126
0.0003	2.5128	98	0.0128
0.0007	2.5641	100	0.0127
0.008	2.6154	102	0.0129
0.0009	2.6667	104	0.0128
0.0074	2.7179	106	0.0128
0.0003	2.7692	108	0.0126
0.0002	2.8205	110	0.0128
0.0007	2.8718	112	0.0128
0.0003	2.9231	114	0.0126
0.0012	2.9744	116	0.0126
0.0029	3.0256	118	0.0126
0.0003	3.0769	120	0.0124
0.0002	3.1282	122	0.0126
0.0004	3.1795	124	0.0126
0.0001	3.2308	126	0.0125
0.0013	3.2821	128	0.0125
0.0001	3.3333	130	0.0125
0.0003	3.3846	132	0.0126
0.0001	3.4359	134	0.0126
0.0002	3.4872	136	0.0126
0.003	3.5385	138	0.0126
0.0002	3.5897	140	0.0123
0.0003	3.6410	142	0.0125
0.0001	3.6923	144	0.0127
0.0002	3.7436	146	0.0124
0.0003	3.7949	148	0.0126
0.0001	3.8462	150	0.0123
0.0005	3.8974	152	0.0127
0.0006	3.9487	154	0.0125
0.0002	4.0	156	0.0124

Framework versions

• PEFT 0.15.2
• Transformers 4.52.4
• Pytorch 2.9.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1