Multi-super-128K / special_tokens_map.json
fhai50032's picture
Upload tokenizer
cfd142f verified
{
"eos_token": "<|endoftext|>",
"pad_token": "<|endoftext|>"
}