Skip to main content
Back to timeline
arXivSource publication:

Japanese polyphones stop being a guess: Irodori-Ruby-OPD reaches 97.13% raw-text pronunciation accuracy with text-only distillation

Synopsis

Japanese speech synthesis has long traded between two paths: a phoneme front end gives pronunciation control but discards the semantic and prosodic cues carried by Kanji orthography, while reading raw text sounds more natural but misreads polyphonic Kanji. Irodori-Ruby-OPD works in two stages on a flow-matching TTS model, first training a pronunciation-controllable teacher on Ruby-annotated text with perturbed surface forms, then distilling that knowledge onto unannotated text with on-policy distillation over text-only corpora. On the Joyo-Kanji-Yomi benchmark it reaches 97.13% accuracy, 2.02% target PER and 0.70% sentence PER from raw text while retaining Ruby-based pronunciation control.

AI-generated editorial illustration: Ruby-OPD: Enhancing Pronunciation Controllability and Accuracy for Flow-Matching Text-to-Speech via On-Policy Distillation

Interpretation

Japanese TTS can now reach 97.13% pronunciation accuracy when fed only raw Kanji text, while still accepting Ruby annotations to specify how a word should be read. Cascaded systems previously converted text into phoneme sequences at a front end to guarantee pronunciation, at the cost of discarding lexical semantics and prosodic cues in the orthography; zero-shot models that read raw text instead made polyphone errors and offered no explicit pronunciation control. On the Joyo-Kanji-Yomi benchmark, which covers 2,136 Joyo Kanji and their 4,512 official pronunciations, Irodori-Ruby-OPD reaches 97.13% accuracy, 2.02% target PER, 0.70% sentence PER and 4.21% sentence CER from raw text, against 94.11% accuracy, 4.68% target PER and 0.97% sentence PER for its starting point Irodori-Base.

Pronunciation control is carried by writing the reading into the text itself, so the Kanji surface form and the target pronunciation appear in one input unit and the model follows the annotation. Two linearized annotation styles exist: implicit Ruby appends the reading in parentheses after the word, while explicit Ruby uses boundary markers to delimit both the target span and its reading, removing left-boundary ambiguity in unsegmented text. Across target-word and full-sentence annotation scopes, explicit Ruby outperforms implicit Ruby for both the SFT and OPD models; with full explicit annotation the OPD model reaches 98.24% accuracy, 1.15% target PER and 0.42% sentence PER.

The gain on unannotated text comes from on-policy distillation: a frozen teacher sees full Ruby text, a student sees only raw text, and the student rolls out an intermediate state from pure Gaussian noise and is trained to match the teacher's velocity field at that timestep. Attaching an external grapheme-to-phoneme module at inference, or synthesizing polyphone speech pairs offline, adds pipeline components or costly offline synthesis; on-policy distillation uses only text-pronunciation pairs and needs no parallel audio. Paired speech data from the SFT stage is interleaved during training to anchor acoustic quality and Ruby controllability, and the distilled model beats the SFT-only version across all four Ruby control modes while matching its teacher on speech quality and speaker similarity.

Perspective

Evaluation is confined to the Japanese Joyo-Kanji-Yomi benchmark, covering 2,136 Joyo Kanji and their 4,512 official pronunciations; at inference, reference audio prompts are sampled at random from 100 speakers in the JVS corpus. Speech quality and speaker similarity are measured by four objective metrics, UTMOSv2, DNSMOS, CAM++ and ERes2Net, rather than by listening tests.

Fine-tuned only on the Emilia Japanese subset, the model shows a slight drop in raw-text accuracy and target PER relative to Irodori-Base, which the report attributes to the limited scale and modest recording quality of the SFT corpus and to capacity allocated to learning Ruby conditioning. After distillation, speech quality and speaker similarity are on par with the teacher rather than improved.