cs.CLSep 25, 2026

Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoring

Authors: Hikaru Asano, Yotaro Kubo, So Kuroki

Organizations: The University of Tokyo · Sakana AI

Abstract

Accurate and efficient pronunciation transcription is essential for preparing text-to-speech training data at scale. Existing approaches have different limitations: grapheme-to-pronunciation (G2P) and speech-to-pronunciation (S2P) methods each capture only partial information, using only text or only speech, while speech-and-text-to-pronunciation (ST2P) methods use both but require costly pronunciation-annotated data. To address this problem, we propose a training-free ST2P pipeline that integrates both lexical and acoustic information at inference time. Lexical resources and G2P tools generate text-constrained candidates, and a left-to-right greedy search selects the best one using whole-sequence negative log-likelihoods from frozen pretrained S2P models. On three Japanese corpora, our method reduces Character Error Rate (CER) from 0.60--1.40% (text-only baseline) to 0.04--0.17% with reference transcripts, and 0.64--1.58% with ASR transcripts. It outperforms all baselines, including a trained ST2P model and commercial multimodal LLMs. Our greedy search method is 3--3.5×\times faster than beam search at similar CER, and the cascade is 2×\times faster than direct decoding ensuring the efficiency and accuracy. In Spanish, French, and preliminary English, it also surpasses four open multimodal LLMs and the best traditional methods.

Figures & tables

Explore similar work

CardsList
  1. Beyond Acoustic Sparsity and Linguistic Bias: A Prompt-Free Paradigm for Mispronunciation Detection and Diagnosis

    Apr 24, 2026Haopeng Geng, Longfei Yang, Xi Chen +3Mispronunciation Detection and DiagnosisSpeech Processing

  2. Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal

    Jun 18, 2026Syeda Faiza Ahmed Sara, Shammur Absar ChowdhurySpeech ProcessingAutomated Spoken Language Assessment

  3. GRAFT: Grafted Reference Audio for Fine-grained Pronunciation in Zero-shot Text-to-Speech

    Jul 2, 2026Antonis Asonitis, Francesco Verdini, Aref Farhadipour +4TTS SynthesisZero-Shot TTS