hallucination_bin_detector_v3.0

This model is a fine-tuned version of yandex/YandexGPT-5-Lite-8B-pretrain on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6244

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 1337
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• total_eval_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss
0.6885	0.3521	16	0.7142
0.6634	0.7043	32	0.6553
0.6475	1.0564	48	0.6706
0.6837	1.4085	64	0.6420
0.6305	1.7607	80	0.6399
0.6719	2.1128	96	0.6503
0.6209	2.4649	112	0.6333
0.5761	2.8171	128	0.6267
0.5386	3.1692	144	0.6331
0.6297	3.5213	160	0.6328
0.6389	3.8735	176	0.6244

Framework versions

• PEFT 0.9.0
• Transformers 4.45.2
• Pytorch 2.3.0a0+6ddf5cf85e.nv24.04
• Datasets 4.4.1
• Tokenizers 0.20.3