w2v-bert-2.0-yoruba_naijavoices_500h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3063
• Wer: 0.2869
• Cer: 0.1773

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 160
• eval_batch_size: 160
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 320
• total_eval_batch_size: 320
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
1.296	0.6418	1000	0.3281	0.8873	0.6682
0.7527	1.2837	2000	0.3026	0.6976	0.5924
0.6799	1.9255	3000	0.2942	0.6570	0.5782
0.6702	2.5674	4000	0.2912	0.6181	0.5547
0.6192	3.2092	5000	0.2850	0.6172	0.5462
0.6272	3.8511	6000	0.2770	0.5867	0.5276
0.6433	4.4929	7000	0.2780	0.5848	0.5342
0.5611	5.1348	8000	0.2736	0.5638	0.5157
0.5624	5.7766	9000	0.2699	0.5587	0.4997
0.536	6.4185	10000	0.2691	0.5348	0.5023
0.5358	7.0603	11000	0.2629	0.5319	0.4882
0.5841	7.7022	12000	0.2657	0.5271	0.4945
0.5352	8.3440	13000	0.2581	0.5158	0.4748
0.4837	8.9859	14000	0.2563	0.5107	0.4783
0.427	9.6277	15000	0.2542	0.4909	0.4629
0.5264	10.2696	16000	0.2576	0.4992	0.4870
0.4708	10.9114	17000	0.2527	0.4783	0.4622
0.4404	11.5533	18000	0.2492	0.4772	0.4601
0.425	12.1951	19000	0.2470	0.4739	0.4476
0.4104	12.8370	20000	0.2438	0.4574	0.4397
0.413	13.4788	21000	0.2454	0.4526	0.4510
0.4014	14.1207	22000	0.2392	0.4400	0.4422
0.413	14.7625	23000	0.2469	0.4476	0.4474
0.3667	15.4044	24000	0.2358	0.4265	0.4299
0.3913	16.0462	25000	0.2284	0.4222	0.4199
0.4004	16.6881	26000	0.2297	0.4135	0.4109
0.3288	17.3299	27000	0.2269	0.4082	0.4117
0.3252	17.9718	28000	0.2257	0.4048	0.4082
0.3338	18.6136	29000	0.2243	0.4015	0.4030
0.3407	19.2555	30000	0.2175	0.3899	0.3926
0.3314	19.8973	31000	0.2165	0.3910	0.3964
0.2828	20.5392	32000	0.2149	0.3730	0.3865
0.2742	21.1810	33000	0.2153	0.3757	0.3834
0.2767	21.8228	34000	0.2118	0.3665	0.3839
0.3309	22.4647	35000	0.2097	0.3653	0.3743
0.2638	23.1065	36000	0.2052	0.3630	0.3628
0.267	23.7484	37000	0.2066	0.3541	0.3623
0.2392	24.3902	38000	0.2044	0.3468	0.3602
0.3291	25.0321	39000	0.2030	0.3448	0.3565
0.3592	25.6739	40000	0.1998	0.3375	0.3473
0.2334	26.3158	41000	0.1995	0.3350	0.3424
0.2268	26.9576	42000	0.1976	0.3278	0.3418
0.1984	27.5995	43000	0.1993	0.3388	0.3430
0.2398	28.2413	44000	0.1953	0.3390	0.3381
0.234	28.8832	45000	0.1950	0.3250	0.3367
0.1869	29.5250	46000	0.1925	0.3234	0.3253
0.1688	30.1669	47000	0.1899	0.3241	0.3257
0.1642	30.8087	48000	0.1901	0.3252	0.3224
0.197	31.4506	49000	0.1879	0.3058	0.3182
0.1315	32.0924	50000	0.1867	0.3200	0.3159
0.1459	32.7343	51000	0.1844	0.3106	0.3092
0.1299	33.3761	52000	0.1881	0.3168	0.3186
0.1485	34.0180	53000	0.1835	0.3055	0.3053
0.1618	34.6598	54000	0.1838	0.3193	0.3077
0.1109	35.3017	55000	0.1853	0.3095	0.3092
0.154	35.9435	56000	0.1816	0.3105	0.3007
0.1351	36.5854	57000	0.3016	0.3007	0.1817
0.1366	37.2272	58000	0.3128	0.3011	0.1803
0.1481	37.8691	59000	0.3161	0.2959	0.1793
0.1239	38.5109	60000	0.3116	0.2988	0.1808
0.1171	39.1528	61000	0.3021	0.2928	0.1782
0.0956	39.7946	62000	0.3063	0.2869	0.1773

Framework versions

• Transformers 4.48.1
• Pytorch 2.7.1+cu126
• Datasets 4.0.0
• Tokenizers 0.21.2