wrong config in gguf model？ eos_token_id should be 2 instead of 100308
#4
by
davidxifeng
- opened
tokenizer.ggml.bos_token_id 1
tokenizer.ggml.eos_token_id 100308
I run this model and it won't stop
Same. I said 9 words and it ripped into a 30,000 token two-way conversation with me that I wasn't a part of. Also, it replied before it began thinking.
The configuration on the upstream side was incorrect during the quantization process, but it has been repaired now. You can use the following command to fix the issue.
https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-Thinking/commit/004009bb1db14402ae29f253ad9f196673e1a589
--override-kv tokenizer.ggml.eos_token_id=int:2