I've open-sourced the trainer I've been using to build tiny language models from scratch, together with the 95M base model I trained with it.
The trainer runs on Deno (https://deno.com, cross-platform), trains on WebGPU, and it writes GGUF directly. No Python/PyTorch. The weights live in a GGUF file from the first step to the last, so every checkpoint is already something llama.cpp can load.