Chatterbox-TTS (sk)

.gitattributes

@@ -99,3 +99,7 @@ en/chatterbox-turbo-webgpu/onnx/embed_tokens_q4f16.onnx_data filter=lfs diff=lfs
 en/chatterbox-turbo-webgpu/onnx/language_model_q4f16.onnx_data filter=lfs diff=lfs merge=lfs -text
 en/chatterbox-turbo-webgpu/onnx/speech_encoder_q4f16.onnx_data filter=lfs diff=lfs merge=lfs -text
 multi/chatterbox-multilingual-ONNX-q4/default_voice.wav filter=lfs diff=lfs merge=lfs -text
+sk/chatterbox-tts-slovak/samples/01_greeting.wav filter=lfs diff=lfs merge=lfs -text
+sk/chatterbox-tts-slovak/samples/02_narrative.wav filter=lfs diff=lfs merge=lfs -text
+sk/chatterbox-tts-slovak/samples/03_explanation.wav filter=lfs diff=lfs merge=lfs -text
+sk/chatterbox-tts-slovak/samples/04_long_narrative.wav filter=lfs diff=lfs merge=lfs -text

sk/chatterbox-tts-slovak/.gitattributes

@@ -0,0 +1,39 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+samples/01_greeting.wav filter=lfs diff=lfs merge=lfs -text
+samples/02_narrative.wav filter=lfs diff=lfs merge=lfs -text
+samples/03_explanation.wav filter=lfs diff=lfs merge=lfs -text
+samples/04_long_narrative.wav filter=lfs diff=lfs merge=lfs -text

sk/chatterbox-tts-slovak/GUIDE.md

@@ -0,0 +1,528 @@
+# Fine-tuning Chatterbox on a Low-Resource Language: 7 Things That Mattered
+
+> 🇸🇰 **Slovenská verzia nižšie** — [skoč na slovenský preklad](#fine-tuning-chatterboxu-na-málo-zastúpenom-jazyku-7-vecí-na-ktorých-záležalo).
+
+Resemble AI's [Chatterbox Multilingual TTS](https://huggingface.co/ResembleAI/chatterbox) is one of the few SOTA open-source TTS models with a real MIT license — code *and* weights — so it's a natural starting point if you want to build a commercially usable text-to-speech for a language that the official model doesn't ship.
+
+I fine-tuned it on Slovak. The 23 supported languages in the official multilingual checkpoint don't include it, and there was nothing comparable in the open-source ecosystem — every other halfway decent multilingual TTS I could find (XTTS-v2, F5-TTS, Fish Speech) ships under non-commercial licenses. So I trained my own and [published the weights](https://huggingface.co/pekiskol/chatterbox-tts-slovak).
+
+The TTS sits at the end of a larger pipeline I'm building — an end-to-end Slovak video-dubbing tool (Whisper for transcription, a fine-tuned Gemma 3 for translation, MuseTalk for lip-sync) — and Slovak TTS was the missing piece. That's the reason "good enough on a single sentence" wasn't good enough; I needed audio that holds up across hours of generated speech.
+
+The fine-tuning itself isn't the hard part. The hard part is the dozen tiny things that turn a "weights file that loads" into "weights file that produces audio you'd actually ship." Here are the seven that mattered most for me.
+
+This post assumes you already know what fine-tuning a TTS is and have a base Chatterbox setup running. It's the practical-tuning notes I wish someone had written before I started.
+
+---
+
+## 1. The base model and your fine-tune disagree on vocab size
+
+Chatterbox uses a sentencepiece text tokenizer, and depending on training data your fine-tune may end up with a different vocab size than the base multilingual checkpoint. If you naively `load_state_dict(strict=True)` the T3 weights into the base model, you get a shape mismatch.
+
+The fix is to pad or trim the affected matrices (`text_emb.weight` and `text_head.weight`) before loading:
+
+```python
+state = load_safetensors(my_finetune_path, device="cpu")
+
+target_vocab = model.t3.text_emb.weight.shape[0]
+src_vocab = state["text_emb.weight"].shape[0]
+
+if src_vocab > target_vocab:
+    state["text_emb.weight"] = state["text_emb.weight"][:target_vocab, :]
+    state["text_head.weight"] = state["text_head.weight"][:target_vocab, :]
+elif src_vocab < target_vocab:
+    pad = target_vocab - src_vocab
+    emb_pad = state["text_emb.weight"].mean(dim=0, keepdim=True).repeat(pad, 1)
+    head_pad = state["text_head.weight"].mean(dim=0, keepdim=True).repeat(pad, 1)
+    state["text_emb.weight"] = torch.cat([state["text_emb.weight"], emb_pad], dim=0)
+    state["text_head.weight"] = torch.cat([state["text_head.weight"], head_pad], dim=0)
+
+model.t3.load_state_dict(state, strict=True)
+```
+
+Padding with the row mean instead of zeros gives you a sensible "neutral" embedding for tokens the fine-tune never saw — better than zeros, which produce noise.
+
+---
+
+## 2. The reference audio matters more than you think
+
+Chatterbox is a zero-shot voice-cloning model. You give it a few seconds of someone speaking, and the output mimics that voice. Most articles stop there. They don't tell you that **whatever noise is in your reference clip will be baked into every generation**.
+
+I was using a 5.7-second Common Voice Slovak clip as the reference. The model output was almost perfect — but had a faint low-frequency hum throughout. Same hum that was in the reference, which I hadn't noticed until I heard it stretched across thirty seconds of generated audio.
+
+The fix is to clean the reference *before* you pass it to the model. Here's the ffmpeg chain I ended up with:
+
+```bash
+ffmpeg -i reference.wav -af "
+  highpass=f=70,
+  afftdn=nr=12:nt=w:om=o,
+  lowpass=f=11000,
+  equalizer=f=6800:t=q:w=1.2:g=-1.5,
+  silenceremove=start_periods=1:start_silence=0.04:start_threshold=-50dB,
+  areverse,
+  silenceremove=start_periods=1:start_silence=0.06:start_threshold=-46dB,
+  areverse
+" reference_clean.wav
+```
+
+What each stage does:
+
+- `highpass=70` removes mains hum and low-frequency rumble
+- `afftdn` is FFT-based broadband denoising (12 dB reduction is gentle — pushing it higher starts to make speech metallic)
+- `lowpass=11000` cuts hiss above 11 kHz, which Chatterbox doesn't reproduce anyway
+- The `equalizer` notch around 6.8 kHz tames sibilance
+- The pair of `silenceremove` + `areverse` blocks trims silence from both ends without complicated edge-case handling
+
+This matters because cloning is essentially a "voice colour transfer" — anything in the reference *is* part of the cloned voice. Garbage in, garbage out.
+
+---
+
+## 3. Generation parameters: defaults vs. tuned
+
+The default `model.generate()` call works, but you can do better. After regression-testing on Slovak segments, I landed on these:
+
+| Param                | Default | Tuned for stability | Notes                                    |
+|----------------------|---------|---------------------|------------------------------------------|
+| `exaggeration`       | 0.5     | 0.5                 | Same                                     |
+| `cfg_weight`         | 0.5     | 0.5                 | Same                                     |
+| `temperature`        | 0.8     | 0.6                 | Lower → more stable, less variable       |
+| `top_p`              | 1.0     | 0.92                | Cuts the long tail of low-prob samples   |
+| `repetition_penalty` | 1.0     | 1.25                | Prevents the model getting stuck on syllables |
+
+There's a real **trade-off** here: tuned parameters produce more consistent, less-likely-to-fail output, but they also flatten the prosody. The voice sounds slightly more monotone. For a production pipeline doing thousands of segments per day where any failure is worse than a slightly less expressive read, tuned wins. For a demo on a model card where you want one perfect take, default temperature with a deterministic seed and a retry loop wins.
+
+```python
+RETRY_SEEDS = [42, 0, 123, 7, 99]
+min_dur = max(0.3, len(text) / 25.0)
+
+for seed in RETRY_SEEDS:
+    torch.manual_seed(seed)
+    wav = model.generate(text=text, language_id="sk")
+    if wav.shape[-1] / model.sr >= min_dur:
+        break
+```
+
+The retry loop catches the cases where the model produces a too-short output (it sometimes EOSes early on hard inputs). If the first seed works, you stop; if not, try another. Five seeds is plenty in practice.
+
+---
+
+## 4. The first word can be garbage. Add a warmup prefix.
+
+Generating "Slovenský jazyk je úradný..." would sometimes produce "Zvolenský jazyk..." — the model's first token after the reference would morph. Sometimes the entire first word would be quiet noise.
+
+This is a "warmup" artifact: the model has to transition from the reference voice's prosody to its own generation, and that first ~0.3 s is where it can wobble. Hard words at position zero hit hardest.
+
+Two fixes, both work:
+
+**Reword.** If "Slovenský" trips the model, start with "Slovenčina" instead. Easy, free, doesn't always work.
+
+**Add a warmup prefix.** Put a short, easy phrase at the start that the model can land on cleanly:
+
+```
+Vitajte v ukážke.
+Slovenčina je úradný jazyk Slovenskej republiky.
+...
+```
+
+By the time the model reaches "Slovenčina," it has stabilised. The prefix becomes part of the generation but is short and natural, and you can always trim it in post if you don't want it.
+
+This isn't unique to Chatterbox — most autoregressive TTS models have warmup behaviour at the very start. The fix transfers.
+
+---
+
+## 5. The model can't read out individual letter names
+
+I tried generating "Má bohatú gramatiku, sedem pádov a špecifické hlásky ako ô, ľ alebo ŕ." — a sentence that names individual Slovak letters. The model produced nonsense for the letter-name part.
+
+This is a known limitation of TTS models trained on running speech: they rarely see "the letter ô" as a phrase, so they don't know it should be pronounced as a name rather than as the sound itself. The same is true of acronyms ("NDA" gets read as "enda") and units ("20 %" sometimes becomes "two-hundred percent" because of how digit-percent pairs were tokenised in training data).
+
+The fix is text normalisation *before* TTS. In my pipeline I have a small Slovak-specific preprocessor that rewrites:
+
+- `20 %` → `dvadsať percent`
+- `Y100` → `ypsilon sto`
+- `NDA` → `eN-Dý-Á`
+- Letter names → spelled-out phonetic forms
+
+This lives as a preprocessing layer, not part of the model. It's easier to fix text once than to retrain the model to handle every edge case.
+
+---
+
+## 6. `max_new_tokens` matters for long generations
+
+Chatterbox auto-scales the generation budget:
+
+```python
+_max_toks = max_new_tokens or min(4096, max(1000, len(text) * 8))
+```
+
+For a 350-character text that gives 2800 tokens, which sounds like plenty (at ~25 Hz token rate that's over 100 seconds of audio). But the model can EOS early on a particular word — even with a budget far above what the text needs. I had a 30-second narrative whose final word ("Amerike") got cut short despite a 4096-token budget.
+
+When you have a long input, **set `max_new_tokens=4096` explicitly** and check for premature EOS in postprocessing. If your output ends mid-word, treat it as a generation failure and retry with a different seed (see #3).
+
+For very long inputs, a more reliable strategy is to chunk into sentences, generate each separately, and concatenate with a short crossfade. Chatterbox doesn't have first-class chunking yet, so you do this at the application layer.
+
+---
+
+## 7. Use `prepare_conditionals` separately from `generate`
+
+The Chatterbox API supports passing `audio_prompt_path=` directly to `generate()`, but in a production loop where you're generating many segments with the same voice, it's faster to call `prepare_conditionals` once and reuse:
+
+```python
+model.prepare_conditionals(reference_path)
+
+# Now generate many times without re-loading reference each time
+for text in texts:
+    wav = model.generate(text=text, language_id="sk")
+```
+
+The conditioning extracts speaker-identity features from the reference. Doing it once amortises the cost across the loop. For one-off demos it doesn't matter; for batch jobs it can shave noticeable time.
+
+A related gotcha: if you switch reference voices mid-loop, remember to re-run `prepare_conditionals` (or null `model.conds`) before the next generation, or you'll keep cloning the previous voice.
+
+---
+
+## Final recipe
+
+Putting it together — the inference snippet I'd hand to someone starting fresh:
+
+```python
+import torch, torchaudio, subprocess
+from chatterbox.mtl_tts import ChatterboxMultilingualTTS
+from safetensors.torch import load_file
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+
+# 1. Clean the reference
+subprocess.run([
+    "ffmpeg", "-y", "-i", "reference.wav",
+    "-af",
+    "highpass=f=70,afftdn=nr=12,lowpass=f=11000,"
+    "equalizer=f=6800:t=q:w=1.2:g=-1.5,"
+    "silenceremove=start_periods=1:start_silence=0.04:start_threshold=-50dB,"
+    "areverse,silenceremove=start_periods=1:start_silence=0.06:start_threshold=-46dB,"
+    "areverse",
+    "reference_clean.wav",
+], check=True)
+
+# 2. Load base + patch in fine-tune
+model = ChatterboxMultilingualTTS.from_pretrained(device=device)
+state = load_file("my_finetune_t3.safetensors", device="cpu")
+# vocab resize block from #1 here
+model.t3.load_state_dict(state, strict=True)
+model.t3.to(device).eval()
+
+# 3. Prepare reference once
+model.prepare_conditionals("reference_clean.wav")
+
+# 4. Generate with retry
+text = "Vitajte v ukážke. " + your_actual_text  # warmup prefix from #4
+RETRY_SEEDS = [42, 0, 123, 7, 99]
+min_dur = max(0.3, len(text) / 25.0)
+
+wav = None
+for seed in RETRY_SEEDS:
+    torch.manual_seed(seed)
+    with torch.inference_mode():
+        candidate = model.generate(text=text, language_id="sk", max_new_tokens=4096)
+    if candidate.shape[-1] / model.sr >= min_dur:
+        wav = candidate
+        break
+
+torchaudio.save("output.wav", wav.cpu(), model.sr)
+```
+
+This is roughly what's running in my pipeline today.
+
+---
+
+## What didn't work
+
+A few things I tried that turned out to be dead ends, in case you're tempted by them:
+
+- **Aggressive denoising** of the reference (FFT noise reduction at -18 dB or higher). Removes hum reliably but starts producing a metallic, "phasey" cloned voice. -12 dB is the sweet spot.
+- **`temperature=0.4` or below**. Flattens prosody to the point of sounding like a robocall.
+- **Skipping the warmup prefix and trimming the first 0.3 s in post**. Works *most* of the time, but occasionally cuts the start of an actually-clean first word. The prefix is more reliable.
+- **Trying to fix the EOS-early problem with `repetition_penalty=1.5` or higher**. Did not help; the model's stop decision is upstream of repetition logic.
+
+---
+
+## Wrapping up
+
+If you're fine-tuning Chatterbox on a language it doesn't ship, the biggest things you can do *outside* the training loop are:
+
+1. Clean your reference audio
+2. Tune your generation params (or accept the defaults' trade-offs)
+3. Normalise your text *before* it reaches the model
+4. Use a warmup prefix for the first word
+5. Set `max_new_tokens` explicitly and have a retry path
+
+The Slovak fine-tune lives at [huggingface.co/pekiskol/chatterbox-tts-slovak](https://huggingface.co/pekiskol/chatterbox-tts-slovak) under MIT — drop in the inference snippet above and it should work out of the box. Feedback (or different language fine-tunes that hit the same gotchas) welcome on the model's HF discussions tab.
+
+If you've discovered other Chatterbox tuning tricks I missed, leave a comment — I'd like to extend this list.
+
+---
+---
+
+# Fine-tuning Chatterboxu na málo zastúpenom jazyku: 7 vecí, na ktorých záležalo
+
+> 🇬🇧 **English version above** — [jump to English version](#fine-tuning-chatterbox-on-a-low-resource-language-7-things-that-mattered).
+
+[Chatterbox Multilingual TTS](https://huggingface.co/ResembleAI/chatterbox) od Resemble AI je jeden z mála SOTA open-source TTS modelov so skutočne MIT licenciou — kód *aj* weights — takže je prirodzeným základom, ak chceš postaviť komerčne použiteľný text-to-speech pre jazyk, ktorý oficiálny model nepodporuje.
+
+Dotrénoval som ho na slovenčinu. V 23 podporovaných jazykoch oficiálneho multilingual checkpointu slovenčina nie je, a v open-source ekosystéme nebolo nič porovnateľné — všetky ostatné aspoň trochu slušné multilingual TTS modely (XTTS-v2, F5-TTS, Fish Speech) sú pod nekomerčnými licenciami. Tak som si vyrobil vlastný a [zverejnil weights](https://huggingface.co/pekiskol/chatterbox-tts-slovak).
+
+TTS sedí na konci väčšej pipeline, ktorú staviam — end-to-end nástroj na dabovanie videí do slovenčiny (Whisper na rozpoznávanie reči, fine-tuned Gemma 3 na preklad, MuseTalk na synchronizáciu úst) — a slovenský TTS bol chýbajúci kúsok. To je dôvod, prečo "dosť dobré na jednu vetu" nestačilo; potreboval som audio, ktoré ustojí hodiny generovanej reči.
+
+Samotný fine-tuning nie je ten ťažký problém. Ten ťažký je tucet drobností, ktoré spravia z "weights súbor ktorý sa načíta" niečo ako "weights súbor čo produkuje audio aké by si reálne dodal klientovi". Tu je sedem, ktoré pre mňa zavážili najviac.
+
+Tento článok predpokladá že vieš čo je fine-tuning TTS modelu a máš sfunkčnené základné Chatterbox prostredie. Sú to praktické tuning poznámky, ktoré som si želal aby niekto napísal predtým, než som začal.
+
+---
+
+## 1. Base model a tvoj fine-tune sa nezhodnú na vocab size
+
+Chatterbox používa sentencepiece text tokenizer a podľa tréningových dát môže tvoj fine-tune skončiť s iným vocab size ako base multilingual checkpoint. Ak naivne urobíš `load_state_dict(strict=True)` s T3 weights do base modelu, dostaneš shape mismatch.
+
+Riešenie je padding alebo trim postihnutých matíc (`text_emb.weight` a `text_head.weight`) pred načítaním:
+
+```python
+state = load_safetensors(my_finetune_path, device="cpu")
+
+target_vocab = model.t3.text_emb.weight.shape[0]
+src_vocab = state["text_emb.weight"].shape[0]
+
+if src_vocab > target_vocab:
+    state["text_emb.weight"] = state["text_emb.weight"][:target_vocab, :]
+    state["text_head.weight"] = state["text_head.weight"][:target_vocab, :]
+elif src_vocab < target_vocab:
+    pad = target_vocab - src_vocab
+    emb_pad = state["text_emb.weight"].mean(dim=0, keepdim=True).repeat(pad, 1)
+    head_pad = state["text_head.weight"].mean(dim=0, keepdim=True).repeat(pad, 1)
+    state["text_emb.weight"] = torch.cat([state["text_emb.weight"], emb_pad], dim=0)
+    state["text_head.weight"] = torch.cat([state["text_head.weight"], head_pad], dim=0)
+
+model.t3.load_state_dict(state, strict=True)
+```
+
+Padding cez priemer riadkov namiesto núl ti dá rozumné "neutrálne" embedding pre tokeny, ktoré fine-tune nikdy nevidel — lepšie ako nuly, ktoré produkujú šum.
+
+---
+
+## 2. Reference audio záleží viac, než si myslíš
+
+Chatterbox je zero-shot voice cloning model. Dáš mu pár sekúnd niekoho hovoriaceho a výstup imituje ten hlas. Väčšina článkov sa tu zastaví. Nepovedia ti, že **akýkoľvek šum v reference klipe sa zapečie do každej generácie**.
+
+Používal som 5,7-sekundový Common Voice slovenský klip ako reference. Výstup modelu bol skoro perfektný — ale s jemným nízkofrekvenčným hum cez celé. Ten istý hum bol v referenci, len som si ho nevšimol kým som ho nepočul roztiahnutý cez tridsať sekúnd generovaného audia.
+
+Riešenie je vyčistiť reference *predtým*, než ho odovzdáš modelu. Tu je ffmpeg chain, na ktorý som došiel:
+
+```bash
+ffmpeg -i reference.wav -af "
+  highpass=f=70,
+  afftdn=nr=12:nt=w:om=o,
+  lowpass=f=11000,
+  equalizer=f=6800:t=q:w=1.2:g=-1.5,
+  silenceremove=start_periods=1:start_silence=0.04:start_threshold=-50dB,
+  areverse,
+  silenceremove=start_periods=1:start_silence=0.06:start_threshold=-46dB,
+  areverse
+" reference_clean.wav
+```
+
+Čo robí každý stage:
+
+- `highpass=70` odstráni mains hum a nízkofrekvenčný rumble
+- `afftdn` je FFT-based broadband denoising (12 dB redukcia je jemná — vyššie hodnoty robia reč kovovou)
+- `lowpass=11000` orezáva hiss nad 11 kHz, ktorý Chatterbox aj tak nereprodukuje
+- `equalizer` notch okolo 6,8 kHz krotí sykavky
+- Pár `silenceremove` + `areverse` orezáva ticho z oboch strán bez komplikovaného edge-case handling-u
+
+Záleží na tom, lebo cloning je v podstate "voice colour transfer" — všetko v referenci *je* súčasťou klonovaného hlasu. Garbage in, garbage out.
+
+---
+
+## 3. Generation parametre: defaultné vs. tunované
+
+Default `model.generate()` volanie funguje, ale dá sa to spraviť lepšie. Po regression-testoch na slovenských segmentoch som pristal na týchto:
+
+| Param                | Default | Tunované pre stabilitu | Poznámka                                |
+|----------------------|---------|------------------------|------------------------------------------|
+| `exaggeration`       | 0.5     | 0.5                    | To isté                                  |
+| `cfg_weight`         | 0.5     | 0.5                    | To isté                                  |
+| `temperature`        | 0.8     | 0.6                    | Nižšie → stabilnejšie, menej variability |
+| `top_p`              | 1.0     | 0.92                   | Reže long tail nízkopravdepodobných samples |
+| `repetition_penalty` | 1.0     | 1.25                   | Zabráni modelu zaseknúť sa na slabikách  |
+
+Tu je reálny **trade-off**: tunované parametre dávajú konzistentnejší, menej náchylný-na-zlyhanie výstup, ale tiež plošia prozódiu. Hlas znie trochu monotónnejšie. Pre produkčnú pipeline robiacu tisíce segmentov denne, kde akékoľvek zlyhanie je horšie ako menej výrazný prejav, vyhráva tunovanie. Pre demo na model card, kde chceš jeden perfektný take, vyhráva default temperature s deterministickým seedom a retry loopom.
+
+```python
+RETRY_SEEDS = [42, 0, 123, 7, 99]
+min_dur = max(0.3, len(text) / 25.0)
+
+for seed in RETRY_SEEDS:
+    torch.manual_seed(seed)
+    wav = model.generate(text=text, language_id="sk")
+    if wav.shape[-1] / model.sr >= min_dur:
+        break
+```
+
+Retry loop chytí prípady, keď model vyprodukuje príliš krátky výstup (občas EOS-uje predčasne na ťažších vstupoch). Ak prvý seed vyšiel, končíš; ak nie, skús iný. Päť seedov stačí v praxi.
+
+---
+
+## 4. Prvé slovo môže byť nezmysel. Pridaj warmup prefix.
+
+Generovanie "Slovenský jazyk je úradný..." občas produkovalo "Zvolenský jazyk..." — model prvý token po referenci zmenil. Niekedy bolo celé prvé slovo tichý šum.
+
+Toto je "warmup" artefakt: model musí prejsť z prozódie referenčného hlasu na vlastnú generáciu a tých prvých ~0,3 s je miesto, kde môže zakolísať. Ťažké slová na pozícii zero ho zasiahnu najsilnejšie.
+
+Dva fixy, oba fungujú:
+
+**Preformulovať.** Ak "Slovenský" model rozhodí, začni "Slovenčina" namiesto toho. Ľahké, zadarmo, nie vždy stačí.
+
+**Pridať warmup prefix.** Daj na začiatok krátku, ľahkú frázu, na ktorej model dokáže pristáť čisto:
+
+```
+Vitajte v ukážke.
+Slovenčina je úradný jazyk Slovenskej republiky.
+...
+```
+
+Kým model dôjde k slovu "Slovenčina", už sa stabilizoval. Prefix sa stane súčasťou generácie, ale je krátky a prirodzený, a vždy ho môžeš orezať v post-processingu, ak ho nechceš.
+
+Toto nie je špecifické len pre Chatterbox — väčšina autoregressive TTS modelov má warmup správanie na úplnom začiatku. Fix sa prenáša.
+
+---
+
+## 5. Model nevie nahlas čítať názvy jednotlivých písmen
+
+Skúšal som vygenerovať vetu "Má bohatú gramatiku, sedem pádov a špecifické hlásky ako ô, ľ alebo ŕ." — vetu, ktorá menuje jednotlivé slovenské písmená. Model produkoval nezmysel pre časť s názvami písmen.
+
+Toto je známa limitácia TTS modelov trénovaných na bežnej hovorenej reči: zriedka videli "písmeno ô" ako frázu, takže nevedia, že sa má vysloviť ako názov, nie ako zvuk. To isté platí pre skratky ("NDA" sa prečíta ako "enda") a jednotky ("20 %" sa občas zmení na "two-hundred percent" kvôli tomu, ako boli digit-percent páry tokenizované v tréningových dátach).
+
+Riešenie je text normalizácia *pred* TTS. V mojej pipeline mám malý slovensky-špecifický preprocessor, ktorý prepisuje:
+
+- `20 %` → `dvadsať percent`
+- `Y100` → `ypsilon sto`
+- `NDA` → `eN-Dý-Á`
+- Názvy písmen → vypísané fonetické formy
+
+Žije to ako preprocessing layer, nie ako súčasť modelu. Je ľahšie opraviť text raz, ako pretrénovať model aby zvládal každý edge case.
+
+---
+
+## 6. `max_new_tokens` má význam pri dlhých generáciách
+
+Chatterbox auto-scaluje generation budget:
+
+```python
+_max_toks = max_new_tokens or min(4096, max(1000, len(text) * 8))
+```
+
+Pre 350-znakový text to dá 2800 tokenov, čo znie ako dosť (pri ~25 Hz token rate je to vyše 100 sekúnd audia). Ale model môže EOS-nuť skoro na konkrétnom slove — aj s budgetom oveľa väčším, ako text potrebuje. Mal som 30-sekundovú naratívnu vetu, ktorej posledné slovo ("Amerike") bolo orezané napriek 4096-tokenovému budgetu.
+
+Keď máš dlhý vstup, **nastav `max_new_tokens=4096` explicitne** a v post-processingu kontroluj predčasný EOS. Ak ti výstup končí v polovici slova, ber to ako zlyhanie generácie a skús znova s iným seedom (pozri #3).
+
+Pre veľmi dlhé vstupy je spoľahlivejšia stratégia chunkovať na vety, generovať každú samostatne a spojiť s krátkym crossfade. Chatterbox zatiaľ nemá first-class chunkovanie, takže to robíš na úrovni aplikácie.
+
+---
+
+## 7. Použi `prepare_conditionals` oddelene od `generate`
+
+Chatterbox API podporuje predanie `audio_prompt_path=` priamo do `generate()`, ale v produkčnom loope, kde generuješ veľa segmentov s tým istým hlasom, je rýchlejšie zavolať `prepare_conditionals` raz a používať opakovane:
+
+```python
+model.prepare_conditionals(reference_path)
+
+# Teraz generuj koľkokoľvek bez opätovného načítavania referencie
+for text in texts:
+    wav = model.generate(text=text, language_id="sk")
+```
+
+Conditioning extrahuje features speaker identity z referencie. Robiť to raz amortizuje cenu cez celý loop. Pre jednorazové demá na tom nezáleží; pre batch joby to môže ušetriť poznateľný čas.
+
+Súvisiaca chyba: ak meníš referenčný hlas v polovici loopu, nezabudni znova spustiť `prepare_conditionals` (alebo nullnúť `model.conds`) pred ďalšou generáciou, lebo budeš ďalej klonovať predošlý hlas.
+
+---
+
+## Finálny recept
+
+Dať to dohromady — inference snippet, ktorý by som dal niekomu, kto začína odznova:
+
+```python
+import torch, torchaudio, subprocess
+from chatterbox.mtl_tts import ChatterboxMultilingualTTS
+from safetensors.torch import load_file
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+
+# 1. Vyčisti referenciu
+subprocess.run([
+    "ffmpeg", "-y", "-i", "reference.wav",
+    "-af",
+    "highpass=f=70,afftdn=nr=12,lowpass=f=11000,"
+    "equalizer=f=6800:t=q:w=1.2:g=-1.5,"
+    "silenceremove=start_periods=1:start_silence=0.04:start_threshold=-50dB,"
+    "areverse,silenceremove=start_periods=1:start_silence=0.06:start_threshold=-46dB,"
+    "areverse",
+    "reference_clean.wav",
+], check=True)
+
+# 2. Načítaj base + patchni svoj fine-tune
+model = ChatterboxMultilingualTTS.from_pretrained(device=device)
+state = load_file("my_finetune_t3.safetensors", device="cpu")
+# vocab resize blok z #1 sem
+model.t3.load_state_dict(state, strict=True)
+model.t3.to(device).eval()
+
+# 3. Priprav referenciu raz
+model.prepare_conditionals("reference_clean.wav")
+
+# 4. Generuj s retry
+text = "Vitajte v ukážke. " + your_actual_text  # warmup prefix z #4
+RETRY_SEEDS = [42, 0, 123, 7, 99]
+min_dur = max(0.3, len(text) / 25.0)
+
+wav = None
+for seed in RETRY_SEEDS:
+    torch.manual_seed(seed)
+    with torch.inference_mode():
+        candidate = model.generate(text=text, language_id="sk", max_new_tokens=4096)
+    if candidate.shape[-1] / model.sr >= min_dur:
+        wav = candidate
+        break
+
+torchaudio.save("output.wav", wav.cpu(), model.sr)
+```
+
+Toto je zhruba to, čo dnes beží v mojej pipeline.
+
+---
+
+## Čo nefungovalo
+
+Pár vecí, ktoré som skúsil a ukázali sa ako slepá ulička, pre prípad že by ťa lákali:
+
+- **Agresívne denoising** referencie (FFT noise reduction pri -18 dB alebo viac). Spoľahlivo odstráni hum, ale začne produkovať kovový, "phasey" klonovaný hlas. -12 dB je sweet spot.
+- **`temperature=0.4` alebo nižšie**. Sploští prozódiu do bodu, kde to znie ako automat z call centra.
+- **Vynechať warmup prefix a orezať prvých 0,3 s v post-processingu**. Funguje *väčšinou*, ale občas odreže začiatok skutočne čistého prvého slova. Prefix je spoľahlivejší.
+- **Pokus opraviť EOS-early problém cez `repetition_penalty=1.5` alebo viac**. Nepomohlo; rozhodnutie modelu zastaviť je upstream od repetition logiky.
+
+---
+
+## Záverom
+
+Ak fine-tunuješ Chatterbox na jazyk, ktorý nepodporuje, najväčšie veci ktoré môžeš spraviť *mimo* trénovacieho loopu sú:
+
+1. Vyčistiť reference audio
+2. Tunovať generation params (alebo akceptovať trade-offs defaultov)
+3. Normalizovať text *predtým*, než sa dostane k modelu
+4. Použiť warmup prefix pre prvé slovo
+5. Nastaviť `max_new_tokens` explicitne a mať retry path
+
+Slovenský fine-tune žije na [huggingface.co/pekiskol/chatterbox-tts-slovak](https://huggingface.co/pekiskol/chatterbox-tts-slovak) pod MIT — vlož inference snippet vyššie a malo by to fungovať out of the box. Spätná väzba (alebo fine-tunes na iných jazykoch ktoré narazili na rovnaké veci) vítaná v HF discussions tab modelu.
+
+Ak si objavil iné Chatterbox tuning triky ktoré som tu nespomenul, napíš komentár — rád by som ten zoznam doplnil.
+
+---
+
+*Kód v tomto poste je tiež v [release scriptoch](https://huggingface.co/pekiskol/chatterbox-tts-slovak/tree/main) na stránke modelu.*

sk/chatterbox-tts-slovak/README.md

@@ -0,0 +1,179 @@
+---
+license: mit
+language:
+- sk
+library_name: chatterbox
+pipeline_tag: text-to-speech
+tags:
+- text-to-speech
+- tts
+- voice-cloning
+- chatterbox
+- slovak
+- slovenčina
+- sk
+base_model: ResembleAI/chatterbox
+---
+
+# Chatterbox TTS — Slovak fine-tune
+
+Slovak (slovenčina) fine-tune of [Resemble AI's Chatterbox Multilingual TTS](https://huggingface.co/ResembleAI/chatterbox).
+Drop-in T3 replacement weights — load the base `ChatterboxMultilingualTTS`,
+then swap in these Slovak weights to get high-quality Slovak speech with
+zero-shot voice cloning.
+
+📝 **Tuning guide**: a 7-lessons writeup on fine-tuning Chatterbox for a
+low-resource language is also published on dev.to:
+[Fine-tuning Chatterbox on a Low-Resource Language: 7 Things That Mattered](https://dev.to/pekisko/fine-tuning-chatterbox-on-a-low-resource-language-7-things-that-mattered-13e1)
+(or see [`GUIDE.md`](GUIDE.md) in this repo for the bilingual EN+SK version).
+
+> 🇸🇰 **Slovenčina dole** (Slovak description below).
+
+---
+
+## What's in this repo
+
+| File | Size | What it is |
+|------|------|------------|
+| `t3_sk_v2.2.safetensors` | ~2 GB | Slovak T3 weights — production default |
+| `GUIDE.md` | ~12 KB | Practical tuning guide — 7 lessons from fine-tuning Chatterbox on a low-resource language (EN + SK) |
+
+This repo ships **only model weights** plus a few demo samples. You bring your
+own reference audio (3–10 s of clean Slovak speech) for voice cloning at
+inference time.
+
+## Demo samples
+
+Generated with a Common Voice SK reference clip (CC-0). Reference audio not
+included — only model output.
+
+**Greeting** — *Dobrý deň, vitajte v ukážke slovenského syntetického hlasu.*
+
+<audio controls src="https://huggingface.co/pekiskol/chatterbox-tts-slovak/resolve/main/samples/01_greeting.wav"></audio>
+
+**Narrative** — *V Bratislave práve začína nový deň. Slnko vychádza nad Dunajom a mesto sa pomaly prebúdza.*
+
+<audio controls src="https://huggingface.co/pekiskol/chatterbox-tts-slovak/resolve/main/samples/02_narrative.wav"></audio>
+
+**Explanation** — *Tento model dokáže klonovať akýkoľvek hlas iba z niekoľkých sekúnd referenčnej nahrávky.*
+
+<audio controls src="https://huggingface.co/pekiskol/chatterbox-tts-slovak/resolve/main/samples/03_explanation.wav"></audio>
+
+**Long narrative (~30 s)** — short text on Slovak language and its history (showcases prosody over a longer span).
+
+<audio controls src="https://huggingface.co/pekiskol/chatterbox-tts-slovak/resolve/main/samples/04_long_narrative.wav"></audio>
+
+## Requirements
+
+```bash
+pip install chatterbox-tts torch torchaudio safetensors
+```
+
+GPU recommended (~3.5 GB VRAM). Runs on CPU but slowly.
+
+## Quickstart
+
+```python
+import torch
+from huggingface_hub import hf_hub_download
+from safetensors.torch import load_file
+from chatterbox.mtl_tts import ChatterboxMultilingualTTS
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+
+# 1) Load the base multilingual Chatterbox
+model = ChatterboxMultilingualTTS.from_pretrained(device=device)
+
+# 2) Download Slovak T3 weights and patch them in
+sk_weights = hf_hub_download(
+    repo_id="pekiskol/chatterbox-tts-slovak",
+    filename="t3_sk_v2.2.safetensors",
+)
+state = load_file(sk_weights, device="cpu")
+
+# Handle vocab size mismatch between SK fine-tune and base model
+target_vocab = model.t3.text_emb.weight.shape[0]
+src_vocab = state["text_emb.weight"].shape[0]
+if src_vocab > target_vocab:
+    state["text_emb.weight"] = state["text_emb.weight"][:target_vocab, :]
+    state["text_head.weight"] = state["text_head.weight"][:target_vocab, :]
+elif src_vocab < target_vocab:
+    pad = target_vocab - src_vocab
+    emb_pad = state["text_emb.weight"].mean(dim=0, keepdim=True).repeat(pad, 1)
+    head_pad = state["text_head.weight"].mean(dim=0, keepdim=True).repeat(pad, 1)
+    state["text_emb.weight"] = torch.cat([state["text_emb.weight"], emb_pad], dim=0)
+    state["text_head.weight"] = torch.cat([state["text_head.weight"], head_pad], dim=0)
+
+model.t3.load_state_dict(state, strict=True)
+model.t3.to(device).eval()
+
+# 3) Generate Slovak speech with zero-shot voice cloning
+wav = model.generate(
+    text="Ahoj, toto je ukážka slovenského hlasu generovaného modelom Chatterbox.",
+    audio_prompt_path="path/to/your/reference.wav",  # 3–10 s of clean SK speech
+    language_id="sk",
+)
+
+import torchaudio
+torchaudio.save("output.wav", wav, model.sr)
+```
+
+## Tips for good results
+
+- **Reference audio**: 4–6 seconds of clean, dense speech works best. Avoid music,
+  noise, and long silences.
+- **Text length**: split very long inputs into sentences or short paragraphs; the
+  model can lose coherence on overly long generations.
+- **Numbers and abbreviations**: Slovak numbers, units (e.g. `20 %`, `Y100`) and
+  acronyms (e.g. `NDA`) are sometimes mispronounced. For production use, normalise
+  text first (write `dvadsať percent` instead of `20 %`, `eN-Dý-Á` instead of `NDA`).
+
+## Limitations
+
+- Slovak only — for other languages use the original Chatterbox Multilingual.
+- Quality depends heavily on the reference audio.
+- Code-switching (mixing Slovak with English in one sentence) can produce wrong
+  pronunciation on the foreign words.
+- The model can occasionally produce quiet, garbled audio mid-utterance on hard
+  inputs; usually fixed by re-generating or splitting the text.
+
+## License
+
+This fine-tune is released under the **MIT License**, matching the base
+[Chatterbox](https://huggingface.co/ResembleAI/chatterbox) license. You are free
+to use it commercially.
+
+When using this model, please credit:
+- This fine-tune (link to this repo)
+- [Resemble AI Chatterbox](https://huggingface.co/ResembleAI/chatterbox) (base model)
+
+## Citation
+
+If this model is useful in your work, a citation/credit is appreciated:
+
+```bibtex
+@misc{chatterbox-tts-slovak,
+  author = {pekiskol},
+  title  = {Chatterbox TTS — Slovak fine-tune},
+  year   = {2026},
+  url    = {https://huggingface.co/pekiskol/chatterbox-tts-slovak}
+}
+```
+
+---
+
+## 🇸🇰 Po slovensky
+
+Toto je fine-tune modelu [Chatterbox Multilingual TTS](https://huggingface.co/ResembleAI/chatterbox)
+od Resemble AI, dotrénovaný na slovenčinu. Použitie:
+
+1. Načítaš základný `ChatterboxMultilingualTTS` z Resemble AI.
+2. Nahradíš T3 weights tými zo súboru `t3_sk_v2.2.safetensors`.
+3. Generuješ slovenskú reč s **zero-shot klonovaním hlasu** — model skopíruje
+   farbu hlasu z 3–10 sekundovej referenčnej nahrávky, ktorú dodáš.
+
+**Licencia:** MIT — komerčné použitie povolené, stačí pri publikovaní uviesť
+odkaz na tento model aj na základný Chatterbox.
+
+**Reference audio:** repo neobsahuje žiadne hlasové vzorky. Vlastný hlas
+(alebo hlas s explicitným súhlasom) si dodáš ty pri inferencii.

sk/chatterbox-tts-slovak/samples/01_greeting.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08dc95fc54df891f0bd6a79d543ee5f6b7272b89a15d2c98df842fe00d52fa4d
+size 410960

sk/chatterbox-tts-slovak/samples/02_narrative.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51bccb727c89ddb96b07dee9d7712ae4a26de2a92a1dd3f71e0e5a53adac613e
+size 633680

sk/chatterbox-tts-slovak/samples/03_explanation.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e327b98d6ef9d70768217c9ae6a43370c1970394057f6cacd3b51d05436b8eeb
+size 572240

sk/chatterbox-tts-slovak/samples/04_long_narrative.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a21b3ff487f642c7bd2d73940eb46dc391949566c306fb92887e13c5d9a99616
+size 2809040

sk/chatterbox-tts-slovak/source.txt

@@ -0,0 +1 @@
+https://huggingface.co/pekiskol/chatterbox-tts-slovak

sk/chatterbox-tts-slovak/t3_sk_v2.2.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aaeef4d1e990715ce8c383d73efed89fae52a55de653a9ba2674284ebb150bcd
+size 2143989752