twscrape-prepared-regression-Linq-Embed-Mistral-scorer

This model is a fine-tuned version of Linq-AI-Research/Linq-Embed-Mistral on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 12.6464
• Mse: 12.6428
• Target 0 Mse: 37.8821
• Target 1 Mse: 8.8989
• Target 2 Mse: 2.5619
• Target 3 Mse: 1.2285

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• total_train_batch_size: 192
• total_eval_batch_size: 192
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Mse	Target 0 Mse	Target 1 Mse	Target 2 Mse	Target 3 Mse
428.5	1.0	280	347.7662	347.8059	633.5776	263.5729	370.2541	123.8188
9.9395	2.0	560	20.6962	20.7335	42.0764	18.6294	16.8553	5.3730
11.7539	3.0	840	14.0448	14.0469	37.0272	10.4259	5.5888	3.1457
12.293	4.0	1120	13.0462	13.0442	36.9399	8.8813	4.1221	2.2336
7.3027	5.0	1400	12.8216	12.8139	37.2984	8.7870	3.4468	1.7233
12.8594	6.0	1680	12.7524	12.7500	37.7420	8.7288	3.0093	1.5198
6.7754	7.0	1960	12.7292	12.7237	37.9368	8.8719	2.7380	1.3482
9.9688	8.0	2240	12.6771	12.6815	37.9204	8.9231	2.6111	1.2715
11.9023	9.0	2520	12.6444	12.6388	37.8705	8.8944	2.5619	1.2285
19.6836	10.0	2800	12.6464	12.6428	37.8821	8.8989	2.5619	1.2285

Framework versions

• Transformers 4.49.0
• Pytorch 2.5.1+cu124
• Datasets 3.0.1
• Tokenizers 0.21.0