Dimitar Fine-Tuning — Round 2 Report

A side branch off the Voice + Vision + Arm project — improving the bg/dimitar Piper voice for the Voice Kit, not blocking core Phase 4/5 work.

What Changed From Round 1

Recording approach — the main experiment this round. Round 1's isolated, disconnected sentences led to more reading mistakes and less natural prosody than hoped. Round 2 used continuous passages read in one flowing take, then split afterward using Audacity (manual splitting by ear, one label per natural pause — auto-detection via Label Sounds was tried but manual felt safer while still learning the tool).

New source text. Five blocks of real excerpted text (historical/biographical, political commentary, social policy debate, theological commentary, evolutionary biology/mimetic rivalry discussion) — richer register and vocabulary diversity than round 1's constructed sentences. Personal reading-practice use only, not published or redistributed.

Dataset grew: 94 clips (round 1) → 138 clips (round 2).

Building the Dataset — What Went Right

Automated diff between the recorded file list and the script caught exactly two real mismatches before upload:

1. A capitalization typo in the script (Line7a vs actual line7a)
2. A Cyrillic "а" vs Latin "a" in one actual filename — invisible to the eye, would have silently failed to match during training. Confirmed and fixed by renaming the file directly.

Five text corrections applied before recording (three typos, two numbers spelled out) — same proofreading process as round 1.

Two Real Bugs Hit This Round

1. Recurring connection drops interrupting preprocessing

Network switches on the user's end repeatedly cut the web terminal connection mid-run, silently truncating preprocessing partway through (130/138, then 96/138 on a retry). Fix: tmux — running long jobs inside a persistent session means a dropped connection no longer kills the process; just tmux attach -t training to resume watching. Should be standard practice for any long-running step going forward, not just an emergency fix.

2. CSV multi-line quoted-field bug (genuinely subtle, worth remembering)

Preprocessing consistently reported 130/138 utterances even with a verified-clean, freshly-uploaded metadata.csv — ruling out corruption. Root-caused by reading piper_train's actual source code: it parses metadata.csv with Python's csv.reader, which has a default behavior where a field starting with " but not closed on the same line causes the parser to keep consuming subsequent rows into that same field, treating them as one multi-line quoted value.

Round 2's text used straight ASCII quotes (") around quoted phrases spanning multiple recorded segments — e.g., a quote opening in block01_line1 and not closing until block01_line2b, two rows later. The CSV reader silently merged all three rows into one entry, explaining exactly the missing count (each such merge event "eats" the extra rows).

Training

Resumed from round 1's output checkpoint (epoch=5999-step=163340.ckpt) rather than the original Dimitar checkpoint — genuine continuation, not a restart. Ran to --max_epochs 6900 (~900 additional epochs, same scale as round 1). Completed cleanly inside tmux, fully protected from the connection issues that had disrupted preprocessing.

Final checkpoint: epoch=6899-step=179540.ckpt
Exported: dimitar-polished-r2.onnx + .onnx.json

Result

Genuinely closer to the user's actual voice than round 1 — noted as sounding similar to a known audiobook narrator's voice character. Pronunciation/stress issues remain in places, but represent real, noticeable improvement over round 1. Full side-by-side comparison against the original unmodified Dimitar voice planned as a follow-up listening test.

Cost: comparable to round 1 (~$2-3 range), same RTX 3090 pod choice — confirmed still the right cost/performance pick, no reason to consider a bigger GPU.

Files Saved Locally

• dimitar-polished-r2.onnx + .onnx.json — round 2's working voice model
• epoch=6899-step=179540.ckpt — checkpoint for round 3, if pursued

What's Next for This Thread

• Local GPU machine ordered (see the new training/inference PC on the roadmap) — future rounds can run entirely locally, no more RunPod dance, no upload-speed bottleneck
• Systematic side-by-side testing (original Dimitar vs. round 1 vs. round 2) not yet done — worth doing before deciding if a round 3 is warranted
• XTTS experiments (zero-shot Russian-via-Bulgarian-reference, and later fine-tuning XTTS itself on the same dataset) — explicitly filed as fun/curiosity side projects, not part of the core assistant work
• Core priority reminder: this whole thread has been a productive side branch — the actual roadmap priority remains Phase 4 (Vision) and Phase 5 (Voice) integration work, which this improved voice model will eventually plug into once "good enough"