A Two-Way Street: Surveying the Interplay between Code-Mixing and Native Languages in Text and Speech

Debajyoti Mazumder; Akash Anil

cs.CLOct 20, 2026

A Two-Way Street: Surveying the Interplay between Code-Mixing and Native Languages in Text and Speech

Authors: Debajyoti Mazumder, Akash Anil

Organizations: Indian Institute of Science Education and Research, Bhopal

Abstract

Systems for code-mixed (CM) text and speech rarely start from scratch. CM data is scarce, so most work borrows from the languages being mixed, which we call native languages. This transfer can also run in reverse, when CM data is used to improve models for the native languages. Existing surveys focus on processing CM itself, and none brings the two directions together. This paper fills that gap with a systematic survey of transfer in both directions, across text and speech. We organise the surveyed work in a taxonomy of transfer direction, route and modality. Through this taxonomy, we find that most transfer runs from native languages into the CM setting. The reverse direction is younger, and in it code-switching mainly helps by aligning the two languages. Text and speech work share methods but rarely cite each other. On a shared speech benchmark, native data helps more than new architectures. These findings point to open challenges, such as how much native and CM data to mix in training, and how to evaluate CM text written in different scripts.

Figures & tables

Panel of Figure 1 — Figure 1: Two directions of transfer between native languages and the CM setting. Analysis work measures effects on both sides. The example mixes Hindi (green) and English (purple), with its translation below.

Approach	Mod.	What it needs	Evaluated by	What has been shown
Monolingual and parallel data (§ 3.1 )
Monolingual LM training	T	Monolingual or translated text	CM perplexity	Matches LMs trained on synthetic CM data ( Chuang et al., 2020 )
Native samples and resources	T	Labelled native data, treebanks, lexicons	F1, LAS	Helps as a supplement, but trails annotated CM data ( Mazumder et al., 2025a ; Iliescu et al., 2021 )
Pooling and LM fusion	S	Monolingual speech, text and LMs	WER, MER	Large gains only when test sentences are in the LM text ( Pandey et al., 2017 )
Mixed-lingual TTS	S	Monolingual recordings	MOS	Mixed-lingual voices from monolingual speakers ( Reddy and Rao, 2018 )
Synthetic CM data (§ 3.2 )

Table 1: Native-to-CM approaches compared. Mod.: modality (T text, S speech). Reported numbers come from different setups and are not comparable across rows.

Approach	Mod.	What it needs	Evaluated by	What has been shown
Substitution augmentation	T	Bilingual dictionary	Zero-shot accuracy, F1	Gains on 5 tasks and 19 languages ( Qin et al., 2020 )
Generated augmentation	T	Generator or LLM	Accuracy	Low-resource gains without English loss ( Nagar et al., 2025 )
Contextual switching	T	Model to choose words	POS, NER	Avoids ungrammatical swaps ( Feng et al., 2022 )
Mixed data for speech	S	CM text or speech	WER	Also lowers monolingual WER ( Hu et al., 2023 )
Pretraining objectives	T	Parallel or monolingual corpora	BLEU, XNLI	An alignment signal; parallel data may matter more ( Yang et al., 2020a ; Shao et al., 2026 )
Joint monolingual–CM models	S	CM and monolingual speech	WER	Gains on both ( Madhumani et al., 2020 )

Table 2: CM-to-native approaches compared. Mod.: modality (T text, S speech). Reported numbers come from different setups and are not comparable across rows.

Panel of Figure 2 — Figure 2: Included papers per year and direction. Bidirectional papers count in both. Papers before 2010, without a year or without a direction are omitted. 2026 (shaded) runs to early October.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Survey	Year	No.	Text	Sp.	Dir.	C2N
Sitaram et al. (2019)	2019	–	Y	Y	N	Y
Jose et al. (2020)	2020	–	Y	P	N	N
Doğruöz et al. (2021)	2021	–	Y	P	N	N
Winata et al. (2023)	2023	400+	Y	Y	N	P
Hamed et al. (2025)	2025	–	Y	Y	N	N
Cheng et al. (2025)	2025	–	N	Y	N	N

Appendix

Table 7: Surveys of code-switching compared. No.: number of reviewed papers, if stated. Sp.: speech. Dir.: organised by transfer direction. C2N: covers CM-to-native transfer. Y yes, P partly, N no.

Route (section)	Mechanism labels
Native to CM
Monolingual & parallel data (§ 3.1 )	Monolingual data
Synthetic CM data (§ 3.2 )	Synthetic CM, substitution
Pretrained models & adaptation (§ 3.3 )	Fine-tuning, adapters, pretraining, multitask, prompting, distillation
Mapping to native forms (§ 3.4 )	Pipeline mapping, embedding alignment
CM to native

Appendix

Table 8: Mechanism labels grouped under each route of the taxonomy tree (Figure 3 ), most frequent first. Papers whose main contribution is a resource or an evaluation fall under Analysis, whatever their direction. Papers in the residual class are not in the tree.

Trained from scratch on SEAME
			dev_man			dev_sge
Paper	Change over the baseline	Extra data	Base	Best	$\Delta$	Base	Best	$\Delta$
Zeng et al. (2019) †	LID multitask, LM rescoring	None	34.5	25.0	$-$ 9.5	46.4	34.5	$-$ 11.9
Fan et al. (2023)	Language boundaries	None	16.96	16.29	$-$ 0.7	23.73	22.81	$-$ 0.9
Peng et al. (2022)	Fusion with a CS text LM	CS text for the LM	16.4	15.4	$-$ 1.0	23.2	21.7	$-$ 1.5
Adding native data

Appendix

Table 9: Matched comparisons on SEAME, a Mandarin–English CM speech corpus. Each row gives one paper’s baseline (Base) and best system (Best) in mixed error rate (MER, %).

\Delta

is the change in points. dev_man is dominated by Mandarin, dev_sge by Singapore English. Extra data lists added corpora or the pretrained Whisper model. AISHELL is Mandarin speech, and LibriSpeech and TED-LIUM are English. LID: language identification. † Token error rate, computed like MER. Rows are not comparable with each other.

Paper	Year	Mechanism	Task	Pair
Analysis — speech
Sitaram et al. (2019)	2019	EVAL_ANALYSIS	OTHER	multi
Çetinoğlu et al. (2016)	2016	EVAL_ANALYSIS	OTHER	multi
Lyu et al. (2013a)	2013	EVAL_ANALYSIS	LID	zh-en
Weller et al. (2022)	2022	RESOURCE	ST	es-en
Liu et al. (2021a)	2021	EVAL_ANALYSIS	LID	multi

Appendix

Table 10: Most-cited papers per direction and modality, part 1 of 6.

Paper	Year	Mechanism	Task	Pair
Analysis — text (cont.)
Das et al. (2022a)	2022	EVAL_ANALYSIS	CLS	bn-en
Lee and Wang (2015)	2015	EVAL_ANALYSIS	CLS	zh-en
Kumaresan et al. (2022)	2021	EVAL_ANALYSIS	CLS	ta-en; ml-en
Santy et al. (2021)	2021	EVAL_ANALYSIS	CLS	multi
Yoo et al. (2025b)	2024	EVAL_ANALYSIS	CLS	multi

Appendix

Table 11: Most-cited papers per direction and modality, part 2 of 6.

Paper	Year	Mechanism	Task	Pair
CM to native — text
Qin et al. (2020)	2020	AUG_SUB	NLU	multi
Song et al. (2019)	2019	AUG_SUB	MT	ru-en; zh-en
Liu et al. (2019)	2019	AUG_SUB	NLU	multi
Yang et al. (2020a)	2020	PRETRAIN	LM	multi
Zhang et al. (2021b)	2021	AUG_SUB	CLS	multi

Appendix

Table 12: Most-cited papers per direction and modality, part 3 of 6.

Paper	Year	Mechanism	Task	Pair
Native to CM — speech (cont.)
Winata et al. (2020)	2020	FINETUNE	ASR	zh-en
Zhou et al. (2020a)	2020	MONO_DATA	ASR	zh-en
Dalmia et al. (2021)	2020	MULTITASK	ASR	zh-en
Li and Fung (2013)	2013	AUG_SYN	ASR	–
Luo et al. (2018)	2018	MULTITASK	ASR	zh-en

Appendix

Table 13: Most-cited papers per direction and modality, part 4 of 6.

Paper	Year	Mechanism	Task	Pair
Native to CM — speech (cont.)
Paik et al. (2026b)	2026	ADAPT_MERGE	ASR	multi
Changsong et al. (2023)	2023	FINETUNE	SEQ	id-en; ms-en
Liang et al. (2022b)	2022	ADAPT_MERGE	ASR	zh-en
Lee et al. (2025b)	2025	PRETRAIN	LID	multi
Huang et al. (2021)	2021	MULTITASK	ASR	zh-en

Appendix

Table 14: Most-cited papers per direction and modality, part 5 of 6.

Paper	Year	Mechanism	Task	Pair
Native to CM — text (cont.)
Choudhary et al. (2023)	2018	EMBED_ALIGN	CLS	hi-en
Guellil et al. (2018b)	2018	RESOURCE	CLS	ar-arq
Zhang et al. (2024)	2024	FINETUNE	MT	multi
Sai and Sharma (2021)	2021	PIPELINE_MAP	CLS	multi
Muller et al. (2020)	2020	FINETUNE	SEQ	ar-fr

Appendix

Table 15: Most-cited papers per direction and modality, part 6 of 6.

Cross-Lingual Transfer

Cross-lingual transfer enables models to apply knowledge or capabilities learned in one language to tasks in another.

See all 258 papers

Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech
Jun 14, 2026Yue Heng Yeo, Haoyang Li, Yizhou Peng +6Code-Switching Speech Recognition Synthetic Data Augmentation
College of Computing and Data Science, Nanyang Technological University, Singapore · HLT-COE & CLSP, Johns Hopkins University, USA · Institute for Infocomm Research (I2R), A⋆STAR, Singapore +1
Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders
Mar 20, 2026Debajyoti Mazumder, Divyansh Pathak, Prashant Kodali +1Multilingual Language Models Code-Switching
Indian Institute of Science Education and Research, Bhopal, India · Microsoft Corporation
AfriSwitch: A Benchmark for In-the-Wild African Code-Switched Speech Recognition
Aug 26, 2026Gabrial Zencha Ashungafac, Busayo Awobade, Tobi OlatunjiCode-Switching Speech Recognition Automatic Speech Recognition
Intron Health

Jun 14, 2026J/K move · Enter open · S save

Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech

Yue Heng Yeo, Haoyang Li, Yizhou Peng, Shreyas Gopal+5

College of Computing and Data Science, Nanyang Technological University, Singapore · HLT-COE & CLSP, Johns Hopkins University, USA · Institute for Infocomm Research (I2R), A⋆STAR, Singapore+1

Code-Switching Speech Recognition Synthetic Data Augmentation TTS Synthesis Automatic Speech Recognition Synthetic Data Augmentation for Speech Recognition Preference Learning+1

Open paper AskPDF arXiv

Code-switch (CS) Automatic Speech Recognition (ASR) remains challenging due to limited availability of high quality CS text-speech pairs for training. Although synthetic data augmentation via Text-to-speech (TTS) has been explored, existing CS TTS approaches primarily optimise reconstruction fidelity and do not explicitly enforce language-boundary consistency, thereby limiting their effectiveness for CS ASR augmentation. This paper proposes a code-mixing guided preference-learning framework that steers synthetic speech generation toward improved code-switching fidelity using the Code Mixing Index (CMI). Experiments on the SEAME Mandarin-English conversational corpus demonstrate that the proposed method enhances the utility of synthetic data for ASR fine-tuning. Specifically, when fine-tuning Whisper Large, the proposed approach reduces Mixed Error Rate (MER) from 12.1%/17.8% to 8.9%/14.2% on the DevMAN and DevSGE sets, respectively.

Abstract

Cross-Lingual Transfer

Explore similar work

Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech

Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders

AfriSwitch: A Benchmark for In-the-Wild African Code-Switched Speech Recognition