Evaluation of forced alignment of code-mixed speech: the case of Hindi-English

Ayushi Pandey; Pamir Gogoi; Kevin Tang

cs.CLJul 28, 2026

Evaluation of forced alignment of code-mixed speech: the case of Hindi-English

Authors: Ayushi Pandey, Pamir Gogoi, Kevin Tang

Organizations: Karya, India · Department of English Language and Linguistics, Institute of English and American Studies, Faculty of Arts and Humanities, Heinrich Heine University Düsseldorf, Germany · Department of Linguistics, University of Florida, United States of America

Abstract

Code-mixed speech poses unique challenges to forced alignment: expanded inventories, orthographic errors, and speaker variation. We evaluate forced alignment of Hindi-English code-mixed speech using the Montreal Forced Aligner. We address 2 problems: (1) free variation involving native vs non-native pairs and (2) phonemic boundary detection for mid-utterance English words. Bootstrapping strategies substantially outperform unmodified lexicons. Acoustic models trained on sentence-level code-mixed data achieve a mean error of 4.15ms, ie. ten times lower than monolingual Hindi (38.18ms) or isolated English (37.58ms) alternatives. Principled lexicon design and code-mixed training data are both essential for reliable alignment of bilingual speech.

Explore similar work

Phonetic forced alignment for low-resource language varieties: Model training and evaluation on Chengdu Mandarin
Jul 23, 2026Zhiheng Qian, Aini Li, Hai Hu +1Cross-Lingual Alignment Multilingual Automatic Speech Recognition
Montreal Forced Aligner and the state of speech-to-text alignment in 2026
Jun 16, 2026Michael McAuliffe, Kaylynn Gunter, Michael Wagner +1Cross-Lingual Alignment Brain Alignment
Multilingual Word-Level Forced Alignment with Self-Supervised Representations and Learned Dynamic Programming
Jun 9, 2026Roy Weber, Meidan Zehavi, Rotem Rousso +1Cross-Lingual Alignment Multilingual Automatic Speech Recognition

Jul 23, 2026 · cs.CLJ/K move · Enter open · S save

Phonetic forced alignment for low-resource language varieties: Model training and evaluation on Chengdu Mandarin

Zhiheng Qian, Aini Li, Hai Hu, Liang Zhao

Shanghai Jiao Tong University · City University of Hong Kong · The Hong Kong Polytechnic University · Beijing Foreign Studies University

Phonetic forced alignment is a key technique in phonetic research, yet existing alignment systems lack specialized models for low-resource language varieties. We address this by training text-dependent and text-independent aligners for Chengdu Mandarin using a 17-hour corpus and a custom G2P dictionary. We trained a text-dependent GMM-HMM model (Chengdu-MFA) and fine-tuned a pretrained audio encoder on frame classification with Chengdu-MFA's pseudo label for text-independent alignment (Chengdu-FC). Evaluation on an expert-annotated test set show that both methods significantly outperform Standard Mandarin baselines. Chengdu-MFA reduced average phone boundary differences by 31.8%, while Chengdu-FC achieved a 61.2% reduction. This work establishes a practical bootstrapping pipeline for developing accurate aligners for under-resourced varieties without labor- and time-intensive manual annotation.

Cross-Lingual Alignment Multilingual Automatic Speech Recognition Low-Resource Languages Brain Alignment Text Corpora Hidden Markov Models Model Training

Open paper Ask about it PDF

Abstract

Explore similar work

Phonetic forced alignment for low-resource language varieties: Model training and evaluation on Chengdu Mandarin

Montreal Forced Aligner and the state of speech-to-text alignment in 2026

Multilingual Word-Level Forced Alignment with Self-Supervised Representations and Learned Dynamic Programming