Fragments · Curiosity Experiment

XTTS: First Training Run, and an Accidental Bulgarian Dialect

A pure side-quest — zero impact on the real Dimitar/Piper assistant voice, which remains untouched by any of this

Tried fine-tuning XTTS (which doesn't support Bulgarian) using Russian as a stand-in on the same Dimitar dataset. Training worked. Feeding Bulgarian text through the Russian-tagged model produced something that sounded like a real, specific Bulgarian dialect — entirely by accident.

Setting up on the new ML Desktop

First real workload on the newly-built machine: daswer123/xtts-finetune-webui. Fresh Ubuntu shipped only with Python 3.14.4 — too new for the toolkit's pinned dependencies (torch==2.1.2 has no build for 3.14 at all). Installed an older Python specifically for this project, same pattern as the earlier Piper/RunPod work:

sudo apt update
sudo apt install -y python3.11 python3.11-venv

mkdir -p ~/xtts-project
cd ~/xtts-project
python3.11 -m venv xtts-env
source xtts-env/bin/activate

git clone https://github.com/daswer123/xtts-finetune-webui.git
cd xtts-finetune-webui
pip install --upgrade pip
pip install -r requirements.txt

Install completed cleanly — no dependency conflicts, just a genuinely heavy download (torch + full CUDA 12.1 toolkit + coqui-tts + spacy + faster-whisper, several GB total).

Hit a real GPU/PyTorch compatibility gap along the way — the RTX 5060 Ti's Blackwell architecture wasn't supported by the toolkit's pinned torch version. Full diagnosis lives with the machine's own documentation, since it's a property of the hardware, not this experiment: ML Desktop: Blackwell GPU Fix.

Getting the dataset there

Located the round 2 Dimitar dataset on the home server (organized under Dimitar_v2, not the naming originally guessed):

ssh r@192.168.2.42
find /mnt/storage1 -iname "*dimitar*" -maxdepth 4
# → /mnt/storage1/learn_bg/Dimitar_v2/wav
# → /mnt/storage1/learn_bg/Dimitar_v2/metadata.csv

Transferred directly server → ML Desktop over the local network, no internet upload bottleneck:

mkdir -p ~/xtts-project/dimitar-data
scp -r r@192.168.2.42:/mnt/storage1/learn_bg/Dimitar_v2/wav ~/xtts-project/dimitar-data/
scp r@192.168.2.42:/mnt/storage1/learn_bg/Dimitar_v2/metadata.csv ~/xtts-project/dimitar-data/

Gotcha caught and fixed: the transfer pulled in a large number of ._block*.wav files — macOS "AppleDouble" resource-fork metadata artifacts (each exactly 4096 bytes), not real audio. These would have silently broken any script that globs *.wav in that folder.

cd ~/xtts-project/dimitar-data/wav
rm -f ._*

Verified clean: 138 real audio files, 138 lines in metadata.csv — exact match, dataset genuinely ready.

Launching the UI

cd ~/xtts-project/xtts-finetune-webui
source ../xtts-env/bin/activate
python3 xtts_demo.py
# → Running on local URL: http://127.0.0.1:5003

Data processing: confirmed Bulgarian genuinely isn't supported

The "Dataset Language" dropdown had no Bulgarian option at all — consistent with what was already known (Bulgarian isn't in XTTS v2's 17 supported languages). This UI is specifically for fine-tuning within existing supported languages, not adding a new one.

Decided to use Russian as a fun stand-in — same spirit as an earlier zero-shot cross-lingual experiment, not aiming for a real "Bulgarian" result.

Three cascading dependency fixes

All stemming from the Blackwell-fix torch upgrade (torch 2.11.0+cu128) creating ripple effects through the rest of the stack — none of these were bugs in the setup itself, just newer-library-version consequences the toolkit's original authors couldn't have anticipated.

1. `ModuleNotFoundError: No module named 'torchcodec'`

Newer torchaudio (2.11+) moved audio loading to depend on a separate torchcodec package.

pip install torchcodec

2. `OSError: libnvrtc.so.13: cannot open shared object file`

Plain pip install torchcodec grabbed a CUDA-enabled build mismatched to the specific CUDA version in use.

pip uninstall -y torchcodec
pip install torchcodec --index-url=https://download.pytorch.org/whl/cu128

3. `RuntimeError: Could not load libtorchcodec` (missing `libnppicc.so.12`)

Root cause: torchcodec's CUDA build wants full GPU video/image decoding infrastructure — total overkill for loading small WAV files. Correct fix: use the CPU-only build instead, since only audio was ever needed, not GPU video decoding:

pip uninstall -y torchcodec
pip install torchcodec --index-url=https://download.pytorch.org/whl/cpu
sudo apt install -y ffmpeg  # also needed, TorchCodec's actual decoding backend

Result: "Dataset Processed!" — 138 clips transcribed via Whisper large-v3 (as Russian) and formatted into the training CSVs the tool needs.

Fine-tuning: one more fix needed

Defaults kept as-is (6 epochs, batch size 2, max audio 11s) for a first "just see if it works" run.

Hit: _pickle.UnpicklingError: Weights only load failed — a PyTorch 2.6+ security policy change (torch.load() now defaults to weights_only=True) that this toolkit's version of coqui-tts predates and doesn't account for.

First attempt — edited the two torch.load() call sites directly in trainer/io.py to add weights_only=False. Didn't fully work — same error resurfaced from a different call site.

Real fix — patched torch.load globally, at the very top of utils/gpt_train.py, before any other imports:

import torch
_original_torch_load = torch.load
def _patched_torch_load(*args, **kwargs):
    kwargs['weights_only'] = False
    return _original_torch_load(*args, **kwargs)
torch.load = _patched_torch_load

# ... rest of the file's original imports follow

Safe to do here since the checkpoint being loaded is XTTS's own official base model, not an untrusted file.

Result: training actually ran. ~0.05–0.1s per step, 43 steps/epoch × 6 epochs, finished in about 2 minutes — genuinely fast, confirming the earlier GPU/Blackwell fix is doing real work. Loss declined steadily each epoch (5.68 → 4.22 average). One accidental interruption via Ctrl+C mid-run required a clean re-run — the second run completed fully: Model training done!, final checkpoint best_model_258.pth (5.3GB).

First listen — genuinely interesting, not just "it worked"

UI auto-populated all four checkpoint paths correctly. Switched Language en → ru to match training, loaded the fine-tuned model, ran inference.

Russian test text: sounded like the actual recorded voice, with a very close accent match — XTTS's voice-cloning conditioning captured real vocal character from the fine-tuning data, largely independent of which language slot was technically used.
Bulgarian text fed through the Russian-tagged model: sounded like Bessarabian Bulgarian — a real, historically-isolated Bulgarian dialect softened by centuries of Slavic-language contact. A genuinely apt comparison: feeding real Bulgarian words through a Russian phonemizer/tokenizer produces exactly the kind of systematic softening and pronunciation drift that dialect actually has — just computationally, via a language-tag mismatch, instead of geography and time.

Status

✅ ML Desktop's xtts-finetune-webui environment fully working end to end: data processing, training, and inference all confirmed functional
✅ First real local GPU training run completed on the new machine
✅ Genuinely interesting (if unintended) result — a Bulgarian-accented voice clone via a Russian-language proxy
Purely a curiosity thread throughout — no impact on the real Dimitar/Piper voice work, which remains the actual assistant voice