Improving Language Identification for Code-Switched Utterances with Integer Linear Programming

Joanna Radoła; Josep Maria Crego; François Yvon

cs.CLSep 4, 2026

Improving Language Identification for Code-Switched Utterances with Integer Linear Programming

Authors: Joanna Radoła, Josep Maria Crego, François Yvon

Organizations: Sorbonne Université, CNRS, ISIR, Paris, France · SYSTRAN by ChapsVision, Paris, France

Abstract

Automatic identification of code-switched (CS) utterances remains a challenge for language identification (LID) systems, causing such texts to be underrepresented in the training data of Large Language Models. In this paper, we revisit MaskLID, a state-of-the art approach for CS identification, which requires no training and detects arbitrary language combinations. We make three main contributions: (a) we reveal, and address, a major issue of MaskLID: its overreliance on word-level language association scores; (b) we reformulate the underlying optimization algorithm as an Integer Linear Program, enabling us to experiment with a large set of clear and interpretable constraints; (c) each of these improvements vastly improves the baseline system, as we illustrate in experiments involving 10~diverse languages, where we observe a strong boost in performance on CS benchmarks. We release our code and data for reproducibility.

Figures & tables

	tur-eng	eus-spa	hin-eng	npi-eng	tur-deu	spa-eng	ind-eng	fra-wol
# sent in dev / test	100 / 256	100 / 965	100 / 1928	100 / 6959	100 / 1143	0 / 8269	0 / 627	0 / 4386
mean # char/sent	84	79	72	56	75	57	103	73
mean len L1	57.7[ $\pm$ 29.4]	-	47.0[ $\pm$ 32.0]	36.4[ $\pm$ 18.4]	50.6[ $\pm$ 27.9]	41.1[ $\pm$ 20.7]	64.7[ $\pm$ 36.4]	-
mean len L2	19.0[ $\pm$ 11.9]	-	13.1[ $\pm$ 11.4]	12.4[ $\pm$ 8.6]	21.4[ $\pm$ 14.3]	11.2[ $\pm$ 8.3]	24.0[ $\pm$ 17.0]	-
mean # switches	2.8	-	2.9	2.8	1.9	2.9	2.8	-
type	sm	conv+sm	sm	sm	conv	sm	sm	conv

Table 1: Statistics of the CS datasets used the experiments. The uncertainties reported are standard deviations. The eus-spa and fra-wol datasets do not contain the word-level labels necessary to compute relevant CS-specific statistics. sm=social media corpus, conv=conversational corpus.

Panel of Figure 1 — Figure 1: Top: GlotLID distribution of probabilities assigned to the gold label on a subset of FLORES sentences (left) and words (right). Mean score for a sentence: 0.98. Mean score for a word: 0.31. Bottom: Same figure for LiteLID -v2. Mean score for a sentence: 0.96. Mean score for a word: 0.50.

Figure 2: F1 as a function of the input length (in chars).

		CS		mono		all
Model	config	EM	F1	EM	F1	EM	F1
GlotLID	k=1	0.0	0.55	0.98	0.98	0.49	0.76
GlotLID	k=2	0.22	0.56	0.0	0.66	0.11	0.61
GlotLID	p>0.1	0.09	0.61	0.94	0.97	0.51	0.79
LiteLID -v2	k=1	0.0	0.63	0.99	0.99	0.5	0.81
LiteLID -v2	k=2	0.39	0.68	0.0	0.67	0.19	0.67

Table 2: Exact match and F1 on the development data. (*) marks the best configuration.

		CS		mono		all
Model	config	EM	F1	EM	F1	EM	F1
L-v2 + MaskLID	$\tau$ =5	0.39	0.77	0.96	0.98	0.68	0.88
L-v2 + MaskLID	$\tau$ =10	0.35	0.76	0.98	0.99	0.67	0.87
L-v2 + ILP4LID	AVG	0.60	0.83	0.91	0.97	0.76	0.90
L-v2 + ILP4LID	CS	0.68	0.83	0.06	0.69	0.37	0.76
L-v2 + ILP4LID	MONO	0.47	0.81	0.98	0.99	0.73	0.90

Table 3: Exact Match and F1 on the development data for various ILP configurations, compared to the MaskLID baseline of Section 3 .

		CS				mono				all
Model	config	EM	F1	Pr	Re	EM	F1	Pr	Re	EM	F1	Pr	Re
GlotLID	$k$ =1	0.0	0.58	0.87	0.44	0.97	0.97	0.97	0.97	0.49	0.78	0.92	0.71
GlotLID	$k$ =2	0.20	0.56	0.56	0.56	0.0	0.66	0.49	0.98	0.10	0.61	0.53	0.77
GlotLID	p>0.1	0.06	0.62	0.85	0.49	0.95	0.97	0.97	0.98	0.5	0.8	0.91	0.74
LiteLID-v2	$k$ =1	0.0	0.63	0.95	0.47	0.99	0.99	0.99	0.99	0.50	0.81	0.97	0.73
LiteLID-v2	$k$ =2	0.36	0.67	0.67	0.67	0.0	0.67	0.50	1.0	0.18	0.67	0.58	0.83

Table 4: Results on the monolingual and CS test sets.

k

=1,2 when the LID always predicts

k

languages; p>0.1 when it predicts any language having a probability >0.1. The three highest Exact Match values per column are in bold.

	ILP4LID (CS)				MaskLID ( $\tau{=}5$ )
pair	Pr	Re	F1	Un	Pr	Re	F1	Un
HIN-ENG	0.87	0.77	0.80	1.3	0.90	0.68	0.77	21.5
IND-ENG	0.94	0.90	0.92	0.4	0.92	0.82	0.87	8.3
NPI-ENG	0.88	0.70	0.77	3.3	0.94	0.57	0.71	32.5
SPA-ENG	0.93	0.83	0.88	0.9	0.95	0.75	0.84	15.7
TUR-DEU	0.98	0.95	0.96	0.5	0.99	0.84	0.91	13.5

Table 5: Word-level evaluation results on code-switched test data. "Un" is the percentage of Unassigned words.

Figure 3: Runtime analysis showing a linear dependency between the number of language labels and the execution time on a desktop machine for 80 examples (in seconds). CS configuration (left) - AVG configuration (right). Curves correspond to buckets of different average lengths, ranging from ~25 to ~100 characters.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Algorithm 1 MaskLID: Iterative Masking

Panel of Figure 4 — Figure 4: Percentage of cases when a given language label appears in top-10 languages for GlotLID (left) and LiteLID (right) in the monolingual development set. First row is English, second is Hindi.

v	buckets	n-gram	data
v0	1e6	2-5	sents: GlotLID on 125 labels
v0.1	2e6	2-5	sents
v0.2	1e6	3-6	sents
v0.3	2e6	3-6	sents
v1	2e6	3-6	words
v1.1	1e6	2-5	sents+words

Appendix

Table 6: FastText training configurations explored in training LiteLID .

\begin{array}[]{ccc}\mathbf{C(\mathbf{w})}=\overset{\scriptstyle w_{1}\hskip 8.19447ptw_{2}\hskip 8.19447ptw_{3}}{\begin{bmatrix}c_{11}&c_{12}&c_{13}\\ {\color[rgb]{0,0,1}c_{21}}&{\color[rgb]{0,0,1}c_{22}}&{\color[rgb]{0,0,1}c_{23}}\\ c_{31}&c_{32}&c_{33}\end{bmatrix}}\mathbf{A}=\overset{\scriptstyle w_{1}\hskip 8.19447ptw_{2}\hskip 8.19447ptw_{3}}{\begin{bmatrix}1&0&0\\ 1&1&1\\ 0&1&1\end{bmatrix}}\quad\mathbf{Y}=\overset{\scriptstyle w_{1}\hskip 8.19447ptw_{2}\hskip 8.19447ptw_{3}}{\begin{bmatrix}1&0&0\\ {\color[rgb]{0,0,1}0}&{\color[rgb]{0,0,1}1}&{\color[rgb]{0,0,1}1}\\ 0&0&0\end{bmatrix}}\quad\mathbf{U}=\begin{bmatrix}1\\ {\color[rgb]{0,0,1}1}\\ 0\end{bmatrix}\begin{matrix}\scriptstyle l_{1}\scriptstyle(\text{eng})\\ \scriptstyle l_{2}\scriptstyle(\text{fra})\\ \scriptstyle l_{3}\scriptstyle(\text{spa})\end{matrix}\par\par\end{array}

Appendix

Figure 5: Code-switching LID as an ILP problem. The inputs are

\mathbf{C}(\mathbf{w})

, the cost matrix, and

\mathbf{A}

, which contains information about the presence of each language

l

in the top-M list of each word.

\mathbf{Y}

is a feasible solution, with

\mathbf{U}

the associated language vector.

\mathbf{Y}

satisfies the constraints: at most one language for each word, no more than two languages (

\sum_{l}u_{l}=2

), a minimal length of

10

chars for each span.

\mathcal{O}(\mathbf{Y^{*}})

is the value of the objective function computed for this assignment.

Inputs
$c_{l,t}$	$T\times L$ (costs)
$a_{l,t}$	$T\times L$ (top-M languages)
Variables
$y_{l,t}$	$T\times L$ (word assignments)
$u_{l}$	$L$ (sentence assignments)
All	$(T+1)\times L$

Appendix

Table 7: Counts of variables and constraints in the core model.

Inputs
$c_{l,t}$	$T\times L$ (costs)
$a_{l,t}$	$T\times L$ (top-M languages)
Variables
$y_{k,l,t}$	$K\times T\times L$ (word-level language and rank assignments)
$u_{k,l}$	$K\times L$ (sentence-level language and rank assignments)
$z_{l,t}$	$T\times L$ (word-level language assignment)

Appendix

Table 8: Counts of variables and constraints in the extended model.

	CS		mono		all
config	EM	F1	EM	F1	EM	F1
0 : CS	0.68	0.83	0.06	0.69	0.37	0.76
1 : 0+ $\alpha_{k}=[1,0.95]$	0.68	0.83	0.09	0.70	0.39	0.77
1+ $P=10$	0.67	0.84	0.59	0.88	0.63	0.86
1+ $P=25$	0.57	0.83	0.88	0.96	0.73	0.90
1+ $P=50$	0.33	0.76	0.99	0.99	0.66	0.88

Appendix

Table 9: Exact match and F1 on the development data with various ILP configurations. "C3,C5,P" stands for C3

\tau=5

, C5

M=10

P=10

	G + MaskLID				L-v2 + MaskLID				L-v2+ILP4LID AVG
	(baseline)				(first improvement)				(second improvement)
	EM	F1	Pr	Re	EM	F1	Pr	Re	EM	F1	Pr	Re
eus-spa	0.59	0.86	0.94	0.79	0.55	0.87	0.99	0.77	0.73	0.9	0.95	0.86
hin-eng	0.1	0.56	0.7	0.47	0.17	0.68	0.9	0.55	0.34	0.73	0.82	0.65
ind-eng	0.38	0.74	0.83	0.67	0.47	0.83	0.95	0.73	0.52	0.81	0.87	0.76
npi-eng	0.03	0.32	0.39	0.27	0.16	0.6	0.78	0.49	0.38	0.68	0.71	0.65

Appendix

Table 10: Detailed performances for monolingual and mixed language corpora. G stands for GlotLID, L for LiteLID -v2. Best performances for each code-switched language pair are in bold. Those for which the performance has been improved by a factor of two are underlined.

Language Identification

Language identification determines which language or languages appear in text or speech, including dialects, code-switching, and low-resource varieties.

See all 26 papers

Code-Switching Spoken Language Identification as Multi-Label Set Prediction
Oct 1, 2026Shunsuke Mitsumori, Matthew Wiesner, Shigeo Morishima +1Multi-Label Classification Language Identification
Waseda University, Tokyo, Japan · Johns Hopkins University, Baltimore, USA · Carnegie Mellon University, Pittsburgh, USA
MixLID: Token-Level Language Identification in Mixed-Language Text
Oct 12, 2026Seongtae Hong, Youngjoon Jang, Seungyoon Lee +2Language Identification
Department of Computer Science and Engineering, Korea University
CS-YODAS: A Mined Dataset of In-the-Wild Code-Switched Speech
Jun 9, 2026Brian Yan, Qingzheng Wang, Matthew Wiesner +9Code-Switching Speech Processing
Carnegie Mellon University · Johns Hopkins University · University of Texas at Austin +4

Oct 1, 2026J/K move · Enter open · S save

Code-Switching Spoken Language Identification as Multi-Label Set Prediction

Shunsuke Mitsumori, Matthew Wiesner, Shigeo Morishima, Shinji Watanabe

Waseda University, Tokyo, Japan · Johns Hopkins University, Baltimore, USA · Carnegie Mellon University, Pittsburgh, USA

Multi-Label Classification Language Identification

Open paper AskPDF arXiv

Code-switched (CS) speech leaks through the monolingual language identification (LID) filters used to curate massive speech corpora, calling for CS-aware LID (CS-LID). We formulate utterance-level CS-LID as multi-label language-set prediction and propose a set generator that directly outputs the languages in an utterance, comparing it against atomic-pair and score-based classification baselines. Oracle Top-k is the strongest baseline, but thresholding fails because no single threshold separates CS from monolingual speech. Our set generator predicts the correct language count on unseen pairs without assuming the number of languages, but underperforms oracle Top-k in exact set accuracy. Our analysis identifies the key obstacles to robust CS-LID: oracle cardinality, threshold instability, language bias in CS training data, and the synthetic-to-real gap.

Abstract

Language Identification

Explore similar work

Code-Switching Spoken Language Identification as Multi-Label Set Prediction

MixLID: Token-Level Language Identification in Mixed-Language Text

CS-YODAS: A Mined Dataset of In-the-Wild Code-Switched Speech