tinycompany/Super-QTK-81K

	Tokenizer	English	Hindi	Code_Python	Code_Java	c++	Math
0	fhai50032/BPE-Multilingual-72k (72k)	343695 reversible	13684 reversible	5367 reversible	2269 reversible	13072 reversible	5026 reversible
1	Random chunk (32k)	369228 irreversible, word count differs (212812 vs 379678)	4439 irreversible, word count differs (7570 vs 906)	7716 irreversible, word count differs (1787 vs 6792)	3435 irreversible, word count differs (1013 vs 2818)	19337 irreversible, word count differs (5016 vs 16170)	6764 irreversible, word count differs (2661 vs 6041)
2	HuggingFaceTB/SmolLM2-1.7B-Instruct (49k)	337029 reversible	44629 reversible	5873 reversible	2548 reversible	15702 reversible	6200 reversible
3	UW/OLMo2-8B-SuperBPE-t80k (80k)	223953 reversible	30642 reversible	4429 reversible	1684 reversible	10978 reversible	4052 reversible
4	tinycompany/Super-QTK-81K-v0.1 (80k)	1227219 reversible	9477 reversible	18892 reversible	9994 reversible	42807 reversible	15502 reversible
5	tinycompany/Super-QTK-81K (80k)	313465 reversible	13286 reversible	4951 reversible	1902 reversible	11517 reversible	3598 reversible
6	fhai50032/QTK-81K (81k)	318160 reversible	10767 reversible	5192 reversible	2241 reversible	12730 reversible	5120 reversible
7	unsloth/phi-4 (100k)	308645 reversible	40456 reversible	4844 reversible	2110 reversible	12088 reversible	5573 reversible
8	krutrim-ai-labs/Krutrim-2-instruct (131k)	343067 reversible	15477 reversible	5246 reversible	2223 reversible	12700 reversible	5361 reversible
9	nvidia/Nemotron-4-Mini-Hindi-4B-Instruct (256k)	332271 irreversible, word count differs (212812 vs 212811)	14327 irreversible, word count differs (7570 vs 7561)	6229 irreversible, word count differs (1787 vs 1781)	2732 irreversible, word count differs (1013 vs 1002)	16306 irreversible, word count differs (5016 vs 4964)	6136 reversible