Languages supported and context length?
#2
by
abpani1994
- opened
Is GLiNER2’s context length of 2,048 based on the number of tokens processed by gliner.data_processor.words_splitter(), or does it use a normal split(' ') word count?
I can see that the vocab size is 250k does that mean it can support 100+ languages like BGE M3?
abpani1994
changed discussion title from
Languages supported
to Languages supported?
abpani1994
changed discussion title from
Languages supported?
to Languages supported and context length?