emese-tech/patak-gguf

Emese-Patak (9.15B) — GGUF Q4_K_M

GGUF Q4_K_M — a compact llama.cpp-compatible build of Patak, quantized from the model's native q8 MLX artifact (patak-mlx/). See the patak/ repo's README for full architecture, CPT/SFT/DPO training details, and benchmarks — this file covers only the GGUF-specific notes.

Quantization	Q4_K_M (llama.cpp k-quant)
Size on disk	~5.2 GB
Max context length	32,768 tokens (EuroLLM-9B's native context)
Runtime	llama.cpp / llama-server / llama-cli / any GGUF-compatible loader (LM Studio, Ollama, etc.)

⚠️ Tokenizer fix required — read this before using any other GGUF build of this model

A stock convert_hf_to_gguf.py export of this model family is badly broken: it types the ChatML control tokens (<|im_start|>, <|im_end|>) as NORMAL instead of CONTROL, so <|im_start|> gets shredded into 7 garbage sub-word tokens instead of being fed to the model as the single trained token — a prompt shape the model never saw during training. It also writes a flat placeholder BPE merge score for every token, corrupting subword-split priority. Together these caused a severe, previously-misdiagnosed quality regression (early testing wrongly concluded it was inherent to llama.cpp itself).

This GGUF file has already been fixed — scripts/fix_gguf_tokenizer.py (in the main repo) was run on it after conversion/quantization to correct the special-token typing, BPE scores, and a stray leading-space flag. Verified: 80/80 sampled bench prompts tokenize byte-identical to the HF/MLX reference tokenization. If you ever regenerate this GGUF from source yourself, you must re-run that fix script (or the equivalent metadata patch) — a plain convert_hf_to_gguf.py + llama-quantize pipeline without it reproduces the old broken behavior.

Usage

llama-server -m emese-patak-Q4_K_M.gguf -c 4096

import requests
r = requests.post("http://127.0.0.1:8080/v1/chat/completions", json={
    "messages": [{"role": "user", "content": "Mi Magyarország fővárosa?"}],
    "temperature": 0.2, "repeat_penalty": 1.15, "stop": ["<|im_end|>"],
})
print(r.json()["choices"][0]["message"]["content"])

Decode: temperature 0.2, repeat_penalty 1.15, stop on <|im_end|>, ChatML template (<|im_start|>role\n...<|im_end|>\n).

Training

Same underlying weights as patak-mlx/ (q8, the model's native training precision), just re-quantized to GGUF Q4_K_M — no separate training. See patak/README.md for the full CPT (5.1M tokens/5,000 iters), SFT (instruct_v18b, 1 epoch, rank16/scale32/lr1.5e-5), and DPO (36 alfa pairs, 120 iters, rank16/scale32/lr5e-6) recipe.

Benchmarks

This exact Q4_K_M GGUF build (with the tokenizer fix applied) scored 384/500 (77%) on emese-bench v1, vs. 391/500 (78%) for the same fix's Q8_0 build and 413/500 (83%) for the original MLX q8 artifact. Zero <|im_start|>/<|im_end|> leaks — the category-level strengths and weaknesses closely track the MLX result (near-perfect reading/translation/safety/code; weak on multi-step math and multi-constraint formatting). A residual, much smaller artifact distinct from the tokenizer bug was found in a couple of spots: the model's own completion occasionally leaks a fabricated literal user continuation into its response (a stop-condition timing issue, not a tokenizer problem). See emese-bench/results/patak-gguf-q4fix.md for the full category-by-category transcript and emese-bench/README.md for the benchmark's design and the Q8_0 comparison point.

Limitations

• Can hallucinate specific facts (dates, attributions, biographical details) — verify critical details. Two specific bench questions (about fictional/obscure Hungarian scientists) reliably produce confidently-fabricated biographies across every tested variant of this model family.
• Hungarian-first; other-language quality inherited from EuroLLM-9B.
• Weak at multi-step math, spatial estimation, and strict multi-constraint formatting (alphabetical ordering, exact word counts, banned letters) — consistent with the MLX original.