Download tokenizer_config.json from bigscience-data/tokenizer_v0: direct link, hf CLI and curl.
- Browser
- Download file 131 Bytes
-
https://huggingface.co/bigscience-data/tokenizer_v0/resolve/main/tokenizer_config.json
- Command line
-
hf download hf://bigscience-data/tokenizer_v0/tokenizer_config.json
-
curl -L -o tokenizer_config.json https://huggingface.co/bigscience-data/tokenizer_v0/resolve/main/tokenizer_config.json
131 Bytes
| {"unk_token": "<unk>", "eos_token": "</s>", "bos_token": "<s>", "pad_token": "<pad>", "tokenizer_class": "PreTrainedTokenizerFast"} |