cs.CLOct 14, 2026
SaveAnuvani-LID: Robust and Calibrated Spoken Language Identification for Indian Languages
Organizations: Central Institute of Indian Languages
Abstract
Every multilingual speech pipeline begins with a language decision, an error there propagates silently through recognition, translation and routing. In India it spans 22 scheduled languages across four language families, heard over short, noisy, narrowband and code-mixed audio. Foundational speech models increasingly present language identification (LID) as an inherent latent, generalized capability. Autoregressive decoders in speech models emit a special language ID token, and modern LLMs with audio capabilities can name the language when prompted. We test this assumption on identical clips and find that whisper and similar speech models has tokens for only 15 of the 23 languages, on one second inputs Gemini 3.5 Flash reaches 0.52 macro-F1 and GPT-audio 0.14. We present Anuvani LID, a dedicated 115M parameter LID conformer model trained on 54,643 hours of Indian speech spanning 22 Indian languages and Indian accented English. Along with designed and controlled studies on 102 condition benchmark with speaker and corpus held out tests. Across 37 conditions, it reaches 0.914 mean macro-F1, 5.2 points above Gemini 3.5 Flash and ahead of 14 other systems. It is 24 points ahead of Gemini at one second, scores 0.897 on all 23 languages where no other system exceeds 0.713, and is calibrated to an ECE of 0.008. Reliable Indian LID still requires a robust, dedicated and calibrated model.
Figures & tables
Figure 1: Anuvani-LID against the field on SUITE-42 (macro-F1, identical clips). The radar covers nine domains from 32 of the 37 core conditions, with its axis from 0.3; the leaderboard averages all 37; the bars show the four domains with the largest margin. The prompted APIs chose among 10 candidate languages, Anuvani-LID among all 23.
| Language | Family: subgroup | Script(s) | L1 | Train | F1 | T | Language | Family: subgroup | Script(s) | L1 | Train | F1 | T |
| Hindi | IA: Hindustani | Devanagari | 528.3 | 8,121 | .843 | Konkani 92 | IA: Southern | Devanagari, Latin | 2.3 | 375 | .918 | ||
| Urdu | IA: Hindustani | Arabic | 50.8 | 691 | .925 | Gujarati | IA: Guj.–Rajasthani | Gujarati | 55.5 | 1,500 | .948 | ||
| Punjabi | IA: Northwestern | Gurmukhi | 33.1 | 1,339 | .973 | T | Sanskrit | IA: Old Indo-Aryan | Devanagari | 0.02 | 828 | .962 | |
| Dogri 04 | IA: W. Pahari | Devanagari | 2.6 | 478 | .977 | T | Tamil | Dravidian: South I | Tamil | 69.0 | 3,560 | .974 | |
| Kashmiri | IA: Dardic | Arabic, Deva. | 6.8 | 369 | .953 | Malayalam | Dravidian: South I | Malayalam | 34.8 | 1,544 | .965 | ||
| Sindhi 67 | IA: Northwestern | Arabic, Deva. | 2.8 | 390 | .903 | Kannada | Dravidian: South I | Kannada | 43.7 | 3,287 | .975 |
Table 1: The 23 classes. Family: subgroup follows Glottolog 5.3 ( Hammarström et al., 2026 ) and the Census ( ORGI, 2018 ) (IA: Indo-Aryan; IE: Indo-European; Arabic: Perso-Arabic; Deva.: Devanagari). Script(s) follows CLDR and state official-language acts ( Unicode Consortium, 2026 ; The Unicode Consortium, 2025 ) . L1 : first-language speakers in the 2011 Census (millions). Train : hours in our training pool. F1 : Anuvani-LID on the speaker-disjoint all-23 test. T : documented lexical tone ( Evans et al., 2018 ; Das and Mahanta, 2019 ; Devi and Das, 2024 ) . Superscripts give the year a language entered the Eighth Schedule if not an original member (1967, 1992, 2004; Government of India, 2026 ). ‡ Subgrouping disputed ( Thurgood, 2017 ) . § 128.5M counting second- and third-language speakers ( ORGI, 2011 ) ; all English audio in training and test is Indian-accented.
Figure 2: Speaker population versus training data. Speech resources do not track population: Urdu has more than 50M first-language speakers but less audio than Maithili or Nepali, and English is abundant because most of its 128.5M speakers use it as a second or third language. The diagonals mark 10, 100 and 1,000 training hours per million speakers.
Figure 3: Anuvani-LID. All 17 block outputs of the self-supervised MahaDhwani Conformer are mixed with learned weights, pooled with attentive statistics and classified into 23 languages under focal loss. The whole network is fine-tuned. Blocks 11–12 (blue) carry the most linearly decodable language information (§ 8 ); the panel opens one block. Grey chips summarise the training recipe from Stages I–VI.
Figure 4: Stage I (1,000 h, 20 languages, 4 epochs, one run per arm): all 25 arms, the 11 ECAPA-TDNN back-end variants in the lowest group. Left: cross-corpus macro-F1 with 95% bootstrap intervals. Right: raw ECE (log scale). Pooling choices differ by about 1 pp, but the loss decides calibration. No ECAPA arm improves on attentive pooling with focal loss (blue), despite 23M extra parameters.
Figure 5: Stage II (4,000 h, production skew, 1 epoch). (a) Cross-corpus macro-F1 against sampling temperature, with a 95% band. (b) In-domain F1 by resource tier, with tiers set by hours in the full pool; the in-domain test has no Kashmiri, Santali or Sindhi, so its lowest tier is Dogri and Konkani. Raising trades head for tail.
Figure 6: Stage III: change in macro-F1 (points) relative to a no-augmentation baseline, on the 44 conditions common to all arms (4,000 h, 2 epochs). The babble row is measured against the full stack without babble (65k steps). Each perturbation mainly moves the axis it simulates. The full stack combines their gains at a 0.4-point cost on clean speech.
| Altered | |||||
| Operation | Setting | % | M | k h | |
| Speed | 0.95–1.05 | 0.5 | 40.0 | 24.5 | 41.0 |
| Noise | 0–15 dB | 0.5 | 50.0 | 30.6 | 51.2 |
| Reverberation | 60k RIRs | 0.3 | 30.0 | 18.4 | 30.7 |
| Duration crop | 0.25–16 s | 0.5 | 44.9 | 27.5 | 49.6 |
| Babble | 5–18 dB | 0.2 | 20.0 | 12.2 | 20.5 |
Table 2: Augmentation in the final run, in the order of application. is the probability that a clip is selected for the operation. Altered gives the share of training examples the operation changes, their number in millions and the thousands of hours of source audio they carry, as expected totals over the 61.2M examples of the 425k-step run, computed from the language mix the run’s sampler drew and the clip durations of its training index.
Figure 7: Stage IV: the confusability curriculum as the final run applied it (425k steps, 17 sub-epochs of 25k). Top left: the weight on the 15 Indo-Aryan languages, as logged; the first sub-epoch draws the plain temperature mix. Bottom left: the share of samples by language family. Right: the share of samples per language in the plain mix and in the first scheduled sub-epoch, with each family’s scale factor; Hindi is held at its 11% floor. Shares come from the run’s own sampler on its training index and reproduce every share it logged.
Figure 8: Stage VI (54,643 h). Left: mean macro-F1 on SUITE-102 of the checkpoints saved every 25k steps and of their exponential moving averages (EMA). Right: uniform weight averages (“soups”) of checkpoint subsets. The 7-checkpoint soup ranks first of 31 candidates.
| Core-37 | All-23 | Gap | |||
| System | Lang. | F1 | F1 | [95% CI] | W/L |
| Anuvani-LID | 23 | .914 | .897 | — | — |
| matched | 10 | .923 | .897 | ||
| Gemini 3.5 Flash | 23 | .863 | .710 | 5.2 [4.8, 5.5] | 21/9 |
| Sarvam Saaras | 23 | .847 | .346 | 6.8 [6.5, 7.1] | 26/5 |
| GPT-audio | 23 | .735 | .384 | 18.0 [17.5, 18.4] | 34/2 |
Table 3: SUITE-42 results on identical clips. Lang. counts how many of the 23 classes a system can output. Gap is the paired core macro-F1 difference, Anuvani-LID minus the system (points), with a 95% interval from a bootstrap stratified by condition. W/L counts core conditions on which Anuvani-LID is significantly better or worse (McNemar, Bonferroni-corrected over 37). ∗ Over the covered languages only, which favours partial-coverage systems. Top-1 accuracy and MMS-LID-512/1024 are in Appendix D .
Figure 9: Mean macro-F1 by condition domain on SUITE-42 (identical clips; Anuvani-LID scored over 23 classes). Bold marks the best system in each column, in blue where it is Anuvani-LID. Hindi varieties are rural telephony (GramVaani), dialectal Hindi (Lahaja) and Hinglish (HiACC); see § 7 .
Figure 10: Per-language F1 on the all-23 panel (IndicVoices, speaker-disjoint, 150 clips per language, identical for all systems), grouped by family, with training hours below. Blue connectors show the gain over the better rival.
Figure 11: Calibration. (a) Reliability on the all-23 test before (grey) and after (blue) temperature scaling fitted on the development set. (b) ECE on four sets.
Figure 12: Off-diagonal confusions of Anuvani-LID on the speaker-disjoint all-23 test (6,900 utterances), as row shares on a log scale, grouped by family as in Table 1 . Cells of 3% or more are annotated.
Figure 13: Where language identity lives. (a) Macro-F1 of linear probes on pooled block outputs of the pre-trained encoder, a 1,000 h fine-tune and the 54.6k h model; the band marks blocks 11–12. (b) Softmax-normalised layer weights learned by the 54.6k h model. (c) Cross-corpus macro-F1 after truncating the encoder to its first blocks (Stage-I baseline recipe, 1,000 h, 2 epochs; parameter counts annotated; two seeds for ).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Source tier | Held out by | Lang. | Cond. | Utts |
| IndicVoices, all-23 test | speaker (0/1,308) | 23 | 12 | 78.6k |
| IndicVoices, cross-corpus | speaker | 21 | 13 | 40.2k |
| FLEURS | split | 13 | 16 | 41.4k |
| Kathbath | split | 12 | 16 | 41.8k |
| Common Voice | split | 8 | 12 | 16.1k |
| MUCS | split | 6 | 12 | 14.0k |
Table 4: Composition of SUITE-102. Utterance counts are in-set clips; the three open-set conditions add 1,900 out-of-set negatives, for 318,861 utterances in total. † HiACC (Hinglish code-switching), Lahaja (dialectal Hindi), Svarah (Indian-accented English), IndicTTS, NPTEL lectures and GramVaani (rural telephony).
| Stage | Data (classes) | Arms | Decision carried forward |
| 0 Benchmark | — | — | held-out, multi-axis suite |
| I Head, loss, adaptation | 1k h balanced (20) | 25 | attentive statistics pooling, focal loss, full fine-tuning |
| II Rebalancing | 4k h production skew (23) | 16 | ; a single rebalancer; learned layer mix |
| III Augmentation | 4k h production skew (23) | 8 | duration crops, noise, RIR, codec, babble |
| IV Curriculum | 4k h production skew (23) | 5 | confusability ramp of the 15 Indo-Aryan languages |
| V Optimisation | 4k h proxy (23) | 15 | peak LR ; with floors |
Table 5: The research programme in chronological order.
| Setting | Value |
| Front-end | 80-d log-Mel, 25 ms window, 10 ms hop; 4 convolutional subsampling |
| Encoder | MahaDhwani Conformer-L, 17 blocks, , 8 heads, kernel 31, rel. pos.; dropout 0.1 |
| Layer mix | learned over all 17 blocks |
| Pooling / embed / classes | ASP (1,024-d) 256-d 23-way |
| Loss | focal , label smoothing |
| Sampling | , ; EN floor 16%, HI floor 11%; competence curriculum over 17 sub-epochs raising the weight of the 15 Indo-Aryan languages from 0.45 to 1 after a plain-mix first sub-epoch |
Table 6: Full-scale (Stage VI) training settings.
| System | Identifier / snapshot | Notes |
| Gemini 3.5 Flash | google/gemini-3.5-flash | OpenRouter; card dated 19 May 2026 |
| GPT-audio | openai/gpt-audio | OpenRouter; family snapshot 2025-08-28 |
| GPT-audio-mini | openai/gpt-audio-mini | OpenRouter; snapshots 2025-10-06 / 2025-12-15 |
| Azure Speech LID | Azure AI Speech LID | at most 10 candidates; 11 of 23 locales |
| Sarvam Saaras v3 | saaras:v3 | auto-detect; 22 scheduled + en-IN |
| MMS-LID family | facebook/mms-lid-* | 126, 256, 512, 1024, 2048, 4017 |
Table 7: Systems scored on SUITE-42. API runs used the listed OpenRouter or vendor identifiers. Open models run locally. Commercial behaviour may change between versions; these identifiers are the ones we queried.
| Domain | Condition | Anuvani-LID | matched | Gemini | Sarvam | GPT-aud. | GPT-mini | Azure | |
| Clean read | FLEURS | 1,500 | .961 | .963 | .977 | .864 | .932 | .845 | .773 |
| Kathbath | 1,500 | .987 | .987 | .963 | .864 | .917 | .840 | .795 | |
| Common Voice | 969 | .921 | .920 | .921 | .804 | .828 | .741 | .729 | |
| IndicTTS (studio) | 800 | .978 | .979 | .988 | .998 | .943 | .923 | .777 | |
| SPRING-INX | 1,200 | .953 | .955 | .865 | .956 | .700 | .622 | .686 | |
| Production mix | 1,500 | .979 | .981 | .923 | .860 | .842 | .746 | .818 |
Table 8: Per-condition macro-F1 on SUITE-42 for Anuvani-LID and the commercial APIs (identical clips). matched : Anuvani-LID restricted to the candidate set given to the prompted APIs. Bold marks the best of all 16 systems scored, open models included. a The panel’s FLEURS open-set slice contains only in-set clips and duplicates FLEURS clean. Azure supports 11 of the 23 classes and is omitted from the all-23 rows.
More in this topic
Language Identification
Language identification determines which language or languages appear in text or speech, including dialects, code-switching, and low-resource varieties.
See all 26 papers