Better description and max model size

README.md

@@ -30,7 +30,8 @@ tags:
 
 This is a base (not instruction-tuned) large language model, continually pre-trained on Norwegian data starting from the English [OLMo2-13B](https://huggingface.co/allenai/OLMo-2-1124-13B) model.
 
-The model was trained for 33 000 steps on around 275 billion tokens. Intermediate checkpoints are published here as branches. 
+The model was trained for 33 000 steps on around 275 billion tokens. Maximum sequence length was set to 4 096 in the beginning of training, and then was extended to 16 384 starting from the checkpoint `step_31000`.
+Intermediate checkpoints are published here as branches.
 The main `branch` contains the model's weights after step 33 000 (stage 3).
 
 ## Evaluation

tokenizer_config.json

@@ -8,7 +8,7 @@
   "is_local": true,
   "local_files_only": false,
   "mask_token": "<|mask|>",
-  "model_max_length": 1000000000000000019884624838656,
+  "model_max_length": 16384,
   "pad_token": "<|pad|>",
   "sep_token": "<|endoftext|>",
   "tokenizer_class": "TokenizersBackend",