strict_default_seed-42_1e-3

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1745
• Accuracy: 0.4018

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 256
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 32000
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
6.0064	0.9999	1487	4.4126	0.2930
4.3103	1.9998	2974	3.9046	0.3325
3.6991	2.9997	4461	3.6229	0.3568
3.5294	3.9997	5948	3.4607	0.3715
3.3076	4.9996	7435	3.3641	0.3808
3.2332	5.9995	8922	3.3050	0.3861
3.1276	6.9994	10409	3.2629	0.3906
3.0896	8.0	11897	3.2390	0.3928
3.0274	8.9999	13384	3.2226	0.3948
3.0011	9.9998	14871	3.2100	0.3964
2.9651	10.9997	16358	3.2051	0.3974
2.9434	11.9997	17845	3.1928	0.3986
2.923	12.9996	19332	3.1911	0.3990
2.9066	13.9995	20819	3.1861	0.3999
2.8952	14.9994	22306	3.1889	0.4001
2.8778	16.0	23794	3.1797	0.4002
2.8761	16.9999	25281	3.1794	0.4008
2.8585	17.9998	26768	3.1805	0.4009
2.8646	18.9997	28255	3.1781	0.4016
2.8442	19.9983	29740	3.1745	0.4018

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.20.0