This is the tokenizer used by the Gemma-3 models, and is used in the EuroEval testing suite. We're using this repo rather than the original ones since the original ones are gated, causing CI problems.
Downloads last month
-
Downloads are not tracked for this model.
How to track
