The dataset is only 250 rows...

#1
by vgoklani - opened

How can you do a high quality fine-tune with only 250 rows?

This was a test to determine:
1 - would this reasoning dataset work on this Llama 3.3 mode.
2 - would it induce reasoning (Claude specific, which has a specific fingerprint) WITHOUT "system prompt help".

This (fine tune) was not designed to update the model, domain information or improve the model beyond this.
This model does require more extensive training to bring it up to date and up to today's SOTA standards.

How can you do a high quality fine-tune with only 250 rows?

With 250 rows you can get high quality "imitation" of the teacher model, not really knowledge transfer though.

Sign up or log in to comment