cs.CLJun 11, 2026

PiDA: Phonetically-Informed Data Augmentation for Robust Vietnamese Speech Translation

Authors: Giang Son NguyenTung X. NguyenHieu Minh TruongNhu VoWray BuntineDung D. Le

Organizations: VinUniversity, Vietnam · University of Technology Sydney, Australia · Monash University, Australia

Abstract

Cascaded speech translation (ST) systems suffer from error propagation when Automatic Speech Recognition (ASR) outputs incorrect transcripts. We present the first systematic categorization of ASR errors for Vietnamese ST, classifying substitution errors by phonetic cause and quantifying their impact on downstream Neural Machine Translation (NMT) performance using Linear Mixed-Effects Modelling. We confirm that most ASR substitution errors arise from phonetic confusions rather than random noise, and that these phonetic errors significantly degrade ST quality. Motivated by this finding, we propose Phonetically-Informed Data Augmentation (PiDA), which generates ASR-like corruptions by substituting words with phonetically similar alternatives using phonetic word embeddings. Fine-tuning on a PiDA-augmented version of FLEURS Vietnamese-English improves translation of erroneous ASR outputs (up to +2.04 BLEU over standard fine-tuning) while also slightly improving clean-text performance.

Explore similar work

CardsList
  1. The Role of Disfluencies in Speech Translation

    Aug 3, 2026Maike Züfle, Maria Teleki, Fabian Retkowski +5Speech TranslationAccented Speech

  2. Automatic Labelling of Speech Translation Errors

    Jun 4, 2026Dominik Macháček, Maike Züfle, Ondrej KlejchSpeech TranslationErrors