phi-2-instruct
This model is a fine-tuned version of microsoft/phi-2 on the filtered ultrachat200k dataset using the SFT technique.
Model description
More information about the model architecture and specific modifications made during fine-tuning is needed.
Intended uses & limitations
More information about the intended use cases and any limitations of the model is needed.
Training and evaluation data
More information about the datasets used for training and evaluation is needed.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- training_steps: 51967
Training results
Detailed training results and performance metrics are not provided. It's recommended to reach out to the model creator for more information.
Framework versions
- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0
Evaluation and Inference Example
- For an evaluation of the model and an inference example, refer to the Inference Notebook.
Full Training Metrics on TensorBoard
View the full training metrics on TensorBoard here.
Author's LinkedIn Profile
- Downloads last month
- 104
Model tree for venkycs/phi-2-instruct
Base model
microsoft/phi-2