Every multilingual speech pipeline begins with a language decision, an error there propagates silently through recognition, translation and routing. In India it spans 22 scheduled languages across four language families, heard over short, noisy, narrowband and code-mixed audio. Foundational speech models increasingly present language identification (LID) as an inherent latent, generalized capability. Autoregressive decoders in speech models emit a special language ID token, and modern LLMs with audio capabilities can name the language when prompted. We test this assumption on identical clips and find that whisper and similar speech models has tokens for only 15 of the 23 languages, on one second inputs Gemini 3.5 Flash reaches 0.52 macro-F1 and GPT-audio 0.14. We present Anuvani LID, a dedicated 115M parameter LID conformer model trained on 54,643 hours of Indian speech spanning 22 Indian languages and Indian accented English. Along with designed and controlled studies on 102 condition benchmark with speaker and corpus held out tests. Across 37 conditions, it reaches 0.914 mean macro-F1, 5.2 points above Gemini 3.5 Flash and ahead of 14 other systems. It is 24 points ahead of Gemini at one second, scores 0.897 on all 23 languages where no other system exceeds 0.713, and is calibrated to an ECE of 0.008. Reliable Indian LID still requires a robust, dedicated and calibrated model.
Figures & tables
Figure 1: Anuvani-LID against the field on SUITE-42 (macro-F1, identical clips). The radar covers nine domains from 32 of the 37 core conditions, with its axis from 0.3; the leaderboard averages all 37; the bars show the four domains with the largest margin. The prompted APIs chose among 10 candidate languages, Anuvani-LID among all 23.
Language
Family: subgroup
Script(s)
L1
Train
F1
T
Language
Family: subgroup
Script(s)
L1
Train
F1
T
Hindi
IA: Hindustani
Devanagari
528.3
8,121
.843
Konkani 92
IA: Southern
Devanagari, Latin
2.3
375
.918
Urdu
IA: Hindustani
Arabic
50.8
691
.925
Gujarati
IA: Guj.–Rajasthani
Gujarati
55.5
1,500
.948
Punjabi
IA: Northwestern
Gurmukhi
33.1
1,339
.973
T
Sanskrit
IA: Old Indo-Aryan
Devanagari
0.02
828
.962
Dogri 04
IA: W. Pahari
Devanagari
2.6
478
.977
T
Tamil
Dravidian: South I
Tamil
69.0
3,560
.974
Kashmiri
IA: Dardic
Arabic, Deva.
6.8
369
.953
Malayalam
Dravidian: South I
Malayalam
34.8
1,544
.965
Sindhi 67
IA: Northwestern
Arabic, Deva.
2.8
390
.903
Kannada
Dravidian: South I
Kannada
43.7
3,287
.975
Table 1: The 23 classes. Family: subgroup follows Glottolog 5.3 ( Hammarström et al., 2026 ) and the Census ( ORGI, 2018 ) (IA: Indo-Aryan; IE: Indo-European; Arabic: Perso-Arabic; Deva.: Devanagari). Script(s) follows CLDR and state official-language acts ( Unicode Consortium, 2026 ; The Unicode Consortium, 2025 ) . L1 : first-language speakers in the 2011 Census (millions). Train : hours in our training pool. F1 : Anuvani-LID on the speaker-disjoint all-23 test. T : documented lexical tone ( Evans et al., 2018 ; Das and Mahanta, 2019 ; Devi and Das, 2024 ) . Superscripts give the year a language entered the Eighth Schedule if not an original member (1967, 1992, 2004; Government of India, 2026 ). ‡ Subgrouping disputed ( Thurgood, 2017 ) . § 128.5M counting second- and third-language speakers ( ORGI, 2011 ) ; all English audio in training and test is Indian-accented.
Figure 2: Speaker population versus training data. Speech resources do not track population: Urdu has more than 50M first-language speakers but less audio than Maithili or Nepali, and English is abundant because most of its 128.5M speakers use it as a second or third language. The diagonals mark 10, 100 and 1,000 training hours per million speakers.
Figure 3: Anuvani-LID. All 17 block outputs of the self-supervised MahaDhwani Conformer are mixed with learned weights, pooled with attentive statistics and classified into 23 languages under focal loss. The whole network is fine-tuned. Blocks 11–12 (blue) carry the most linearly decodable language information (§ 8 ); the panel opens one block. Grey chips summarise the training recipe from Stages I–VI.
Figure 4: Stage I (1,000 h, 20 languages, 4 epochs, one run per arm): all 25 arms, the 11 ECAPA-TDNN back-end variants in the lowest group. Left: cross-corpus macro-F1 with 95% bootstrap intervals. Right: raw ECE (log scale). Pooling choices differ by about 1 pp, but the loss decides calibration. No ECAPA arm improves on attentive pooling with focal loss (blue), despite 23M extra parameters.
Figure 5: Stage II (4,000 h, production skew, 1 epoch). (a) Cross-corpus macro-F1 against sampling temperature, with a 95% band. (b) In-domain F1 by resource tier, with tiers set by hours in the full pool; the in-domain test has no Kashmiri, Santali or Sindhi, so its lowest tier is Dogri and Konkani. Raising T trades head for tail.
Figure 6: Stage III: change in macro-F1 (points) relative to a no-augmentation baseline, on the 44 conditions common to all arms (4,000 h, 2 epochs). The babble row is measured against the full stack without babble (65k steps). Each perturbation mainly moves the axis it simulates. The full stack combines their gains at a 0.4-point cost on clean speech.
Altered
Operation
Setting
p
%
M
k h
Speed
0.95–1.05
0.5
40.0
24.5
41.0
Noise
0–15 dB
0.5
50.0
30.6
51.2
Reverberation
60k RIRs
0.3
30.0
18.4
30.7
Duration crop
0.25–16 s
0.5
44.9
27.5
49.6
Babble
5–18 dB
0.2
20.0
12.2
20.5
Table 2: Augmentation in the final run, in the order of application. p is the probability that a clip is selected for the operation. Altered gives the share of training examples the operation changes, their number in millions and the thousands of hours of source audio they carry, as expected totals over the 61.2M examples of the 425k-step run, computed from the language mix the run’s sampler drew and the clip durations of its training index.
Figure 7: Stage IV: the confusability curriculum as the final run applied it (425k steps, 17 sub-epochs of 25k). Top left: the weight on the 15 Indo-Aryan languages, as logged; the first sub-epoch draws the plain temperature mix. Bottom left: the share of samples by language family. Right: the share of samples per language in the plain mix and in the first scheduled sub-epoch, with each family’s scale factor; Hindi is held at its 11% floor. Shares come from the run’s own sampler on its training index and reproduce every share it logged.
Figure 8: Stage VI (54,643 h). Left: mean macro-F1 on SUITE-102 of the checkpoints saved every 25k steps and of their exponential moving averages (EMA). Right: uniform weight averages (“soups”) of checkpoint subsets. The 7-checkpoint soup ranks first of 31 candidates.
Core-37
All-23
Gap
System
Lang.
F1
F1
[95% CI]
W/L
Anuvani-LID
23
.914
.897
—
—
matched
10
.923
.897
Gemini 3.5 Flash
23
.863
.710
5.2 [4.8, 5.5]
21/9
Sarvam Saaras
23
.847
.346
6.8 [6.5, 7.1]
26/5
GPT-audio
23
.735
.384
18.0 [17.5, 18.4]
34/2
Table 3: SUITE-42 results on identical clips. Lang. counts how many of the 23 classes a system can output. Gap is the paired core macro-F1 difference, Anuvani-LID minus the system (points), with a 95% interval from a bootstrap stratified by condition. W/L counts core conditions on which Anuvani-LID is significantly better or worse (McNemar, Bonferroni-corrected over 37). ∗ Over the covered languages only, which favours partial-coverage systems. Top-1 accuracy and MMS-LID-512/1024 are in Appendix D .
Figure 9: Mean macro-F1 by condition domain on SUITE-42 (identical clips; Anuvani-LID scored over 23 classes). Bold marks the best system in each column, in blue where it is Anuvani-LID. Hindi varieties are rural telephony (GramVaani), dialectal Hindi (Lahaja) and Hinglish (HiACC); see § 7 .
Figure 10: Per-language F1 on the all-23 panel (IndicVoices, speaker-disjoint, 150 clips per language, identical for all systems), grouped by family, with training hours below. Blue connectors show the gain over the better rival.
Figure 11: Calibration. (a) Reliability on the all-23 test before (grey) and after (blue) temperature scaling fitted on the development set. (b) ECE on four sets.
Figure 12: Off-diagonal confusions of Anuvani-LID on the speaker-disjoint all-23 test (6,900 utterances), as row shares on a log scale, grouped by family as in Table 1 . Cells of 3% or more are annotated.
Figure 13: Where language identity lives. (a) Macro-F1 of linear probes on pooled block outputs of the pre-trained encoder, a 1,000 h fine-tune and the 54.6k h model; the band marks blocks 11–12. (b) Softmax-normalised layer weights learned by the 54.6k h model. (c) Cross-corpus macro-F1 after truncating the encoder to its first k blocks (Stage-I baseline recipe, 1,000 h, 2 epochs; parameter counts annotated; two seeds for k≥12 ).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Source tier
Held out by
Lang.
Cond.
Utts
IndicVoices, all-23 test
speaker (0/1,308)
23
12
78.6k
IndicVoices, cross-corpus
speaker
21
13
40.2k
FLEURS
split
13
16
41.4k
Kathbath
split
12
16
41.8k
Common Voice
split
8
12
16.1k
MUCS
split
6
12
14.0k
Appendix
Table 4: Composition of SUITE-102. Utterance counts are in-set clips; the three open-set conditions add 1,900 out-of-set negatives, for 318,861 utterances in total. † HiACC (Hinglish code-switching), Lahaja (dialectal Hindi), Svarah (Indian-accented English), IndicTTS, NPTEL lectures and GramVaani (rural telephony).
Stage
Data (classes)
Arms
Decision carried forward
0 Benchmark
—
—
held-out, multi-axis suite
I Head, loss, adaptation
1k h balanced (20)
25
attentive statistics pooling, focal loss, full fine-tuning
II Rebalancing
4k h production skew (23)
16
T≈2 ; a single rebalancer; learned layer mix
III Augmentation
4k h production skew (23)
8
duration crops, noise, RIR, codec, babble
IV Curriculum
4k h production skew (23)
5
confusability ramp of the 15 Indo-Aryan languages
V Optimisation
4k h proxy (23)
15
peak LR 3×10−4 ; T=1.75 with floors
Appendix
Table 5: The research programme in chronological order.
Setting
Value
Front-end
80-d log-Mel, 25 ms window, 10 ms hop; 4 × convolutional subsampling
pl∝nl1/T , T=1.75 ; EN floor 16%, HI floor 11%; competence curriculum over 17 sub-epochs raising the weight of the 15 Indo-Aryan languages from 0.45 to 1 after a plain-mix first sub-epoch
Appendix
Table 6: Full-scale (Stage VI) training settings.
System
Identifier / snapshot
Notes
Gemini 3.5 Flash
google/gemini-3.5-flash
OpenRouter; card dated 19 May 2026
GPT-audio
openai/gpt-audio
OpenRouter; family snapshot 2025-08-28
GPT-audio-mini
openai/gpt-audio-mini
OpenRouter; snapshots 2025-10-06 / 2025-12-15
Azure Speech LID
Azure AI Speech LID
at most 10 candidates; 11 of 23 locales
Sarvam Saaras v3
saaras:v3
auto-detect; 22 scheduled + en-IN
MMS-LID family
facebook/mms-lid-*
126, 256, 512, 1024, 2048, 4017
Appendix
Table 7: Systems scored on SUITE-42. API runs used the listed OpenRouter or vendor identifiers. Open models run locally. Commercial behaviour may change between versions; these identifiers are the ones we queried.
Domain
Condition
n
Anuvani-LID
matched
Gemini
Sarvam
GPT-aud.
GPT-mini
Azure
Clean read
FLEURS
1,500
.961
.963
.977
.864
.932
.845
.773
Kathbath
1,500
.987
.987
.963
.864
.917
.840
.795
Common Voice
969
.921
.920
.921
.804
.828
.741
.729
IndicTTS (studio)
800
.978
.979
.988
.998
.943
.923
.777
SPRING-INX
1,200
.953
.955
.865
.956
.700
.622
.686
Production mix
1,500
.979
.981
.923
.860
.842
.746
.818
Appendix
Table 8: Per-condition macro-F1 on SUITE-42 for Anuvani-LID and the commercial APIs (identical clips). matched : Anuvani-LID restricted to the candidate set given to the prompted APIs. Bold marks the best of all 16 systems scored, open models included. a The panel’s FLEURS open-set slice contains only in-set clips and duplicates FLEURS clean. Azure supports 11 of the 23 classes and is omitted from the all-23 rows.
Language identification determines which language or languages appear in text or speech, including dialects, code-switching, and low-resource varieties.
Language Identification (LID) is an important component of many multilingual natural language processing pipelines, where it facilitates corpus curation, training data analysis, and cross-lingual evaluation of large language models. Despite near-perfect performance on high-resource languages, existing systems remain brittle in low-resource and closely related language settings. We introduce UniLID, a simple and efficient LID method based on the UnigramLM tokenization algorithm. In short, to predict a string's language label, we simply ask: under which language's unigram distribution is this string most likely? Our formulation is data- and compute-efficient, supports incremental addition of new languages without retraining existing models, and can naturally be integrated into existing language model tokenization pipelines. Empirical evaluations against widely used baselines, including fasttext, GlotLID-M, and CLD3, show that UniLID achieves competitive performance on standard benchmarks, reaches 69% accuracy with five labeled samples per language and 89% with 25, and delivers large gains on fine-grained dialect identification.
India's 22 scheduled languages are spoken by over a billion people in conversations that mix languages, often within one turn, overlap freely, and take place over telephones, in meetings, and in media. Knowing who spoke when, the task of speaker diarization, is essential to speech applications for such conversation: voice agents conversing with several people at once, speaker attributed transcription of meetings and calls, contact center analytics, and media monitoring. We present a systematic study of speaker diarization across all 22 languages on IndicDiarBench and introduce Anuvani-Diarization, which adapts a self-supervised end-to-end segmentation model to Indian conversation with uncertainty aware silver supervision. Evaluated zero shot alongside nine recent diarization systems, it reaches a duration weighted diarization error rate (DER) of 16.78% on all 1,164 recordings, the lowest of all systems we ran and second only to the best published result (16.0%), and leads the three strongest rivals in 14 of 22 languages. It matches its parent on six international benchmarks. We also show that the benchmark's utterance level annotation convention is a hidden variable worth 1.9 to 4.1 DER points, depending on the system, and that a convention learned from other Indian transcriptions does not transfer.
Spoken language identification (LID) aims to recognize the target language regardless of accent. In practice, however, LID models fine-tuned from self-supervised speech representations frequently confuse accents with languages, misclassifying non-native (L2) speech as the speaker's first language (L1). We show that non-native speech representations lie between native target-language and native L1 poles, causing systematic misclassification. To address this, we introduce a geometric projection that estimates an L1-bias direction solely from native speech and removes it before the frozen LID head. Across five MMS-LID models and non-native corpora, this projection substantially improves target language identification for L2-accented speech while preserving predictions for native speech. These results show that accent-induced L1 bias can be corrected directly within the representation space without L2 training data or model adaptation.
Minu Kim, Jihwan Lee, David R. Mortensen +1
Signal Analysis and Interpretation Lab (SAIL), University of Southern California, USA · Language Technologies Institute, Carnegie Mellon University, USA