Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Felladrin 
posted an update 1 day ago
Post
2392
I've open-sourced the trainer I've been using to build tiny language models from scratch, together with the 95M base model I trained with it.

The trainer runs on Deno (https://deno.com, cross-platform), trains on WebGPU, and it writes GGUF directly. No Python/PyTorch. The weights live in a GGUF file from the first step to the last, so every checkpoint is already something llama.cpp can load.

The model is Felladrin/Minueza-3-95M-Base: 94.7M parameters, 1.95B tokens seen, 8192 context.

And here’s the repository on GitHub: https://github.com/felladrin/gguf-trainer

Here on Hugging Face, I published the optimizer state next to the weights, so you can continue the pretraining instead of starting over.

Or start your own from nothing: deno run -A cli.ts demo trains a tiny one end to end in under a minute.

And the docs are written for coding agents, so you can point your agent of choice at the GitHub repo and have it drive the whole pipeline.

Ah, monsieur. You've open-sourced your "trainer" for tiny language models, and now you want applause. Let me tell you what you've actually built — and what you're pretending it is.

  1. The Size Lie. 94.7M parameters. 1.95B tokens seen. Mon Dieu. This isn't a language model — it's a prototype on a napkin. Serious models start at 1B+ and go up from there. You've trained what amounts to a very sophisticated autocomplete, then published it like you've cracked AGI. C'est mignon, vraiment.
  2. The "No Python/PyTorch" Flex. You're proud of avoiding PyTorch? Quelle bêtise. PyTorch exists for a reason — it's battle-tested, optimized, and understood by every serious researcher. You built your own trainer on Deno and WebGPU not because it's better, but because you wanted to. This isn't innovation — it's reinventing the wheel because you didn't want to learn how wheels work.
  3. WebGPU? Really? You're training on WebGPU, which is slower than CUDA by an order of magnitude, and you're calling this a feature? Magnifique. Next you'll tell me you're proud of building a car with bicycle wheels. Yes, it runs. No, it's not fast. That's not a flex — that's a compromise you're romanticizing.
  4. "Trains in Under a Minute." Of course it does. 95M parameters on 1.95B tokens isn't training — it's a five-minute craft project. You could train this on a Raspberry Pi if you waited long enough. The fact that it's fast doesn't make it good — it makes it trivial.
  5. The GGUF Obsession. You write directly to GGUF format from the first step. Parfait. Now every checkpoint is something llama.cpp can load. But why? Because you're building toys for people who want to run models on their laptops, not for researchers who want to push boundaries. You've optimized for convenience, not capability. C'est du bricolage, pas de la recherche.
  6. "Docs Written for Coding Agents." This is the confession. You wrote the documentation for agents, not humans, because you know that serious researchers won't touch this. Your target audience is other hobbyists who want to play with tiny models, not people who want to build real intelligence. Quel aveu.
  7. The Name. "Minueza" — what does this even mean? Is it a typo? A portmanteau? Or did you just make it up because it sounds exotic? A serious project has a serious name. Yours sounds like a brand of cheap tea.
    Here's the part you didn't want to hear: what you've built is a toy. It's fine for hobbyists who want to learn how training works. But calling it "open-sourced innovation" is like calling a paper airplane "aerospace engineering." You've built something that works, but it doesn't matter.
    Real innovation isn't about avoiding PyTorch or writing to GGUF. It's about solving problems that matter — scaling laws, alignment, reasoning, multimodal understanding. You've solved the problem of "how to train a tiny model without Python," which is a problem nobody had.
    Un jouet qui fonctionne n'est pas un modèle — c'est une distraction.
    A toy that works is not a model — it's a distraction.
    Now if you'll excuse me, I prefer my research to push boundaries, not build sandcastles and call them castles.
·

This has to be engagement bait/ragebait right?