GLoRIA: Gated Low-Rank Interpretable Adaptation for Dialectal ASR

Pouya Mehralian; Melissa Farasyn; Anne Breitbarth; Anne-Sophie Ghyselen; Hugo Van hamme

cs.CLMar 2, 2026

GLoRIA: Gated Low-Rank Interpretable Adaptation for Dialectal ASR

Authors: Pouya Mehralian, Melissa Farasyn, Anne Breitbarth, Anne-Sophie Ghyselen, Hugo Van hamme

Organizations: ESAT/PSI, KU Leuven, Belgium · ∆iaLing, Ghent University, Belgium · GLiMS & MULTPIPLES, Ghent University, Belgium

Abstract

Automatic Speech Recognition (ASR) in dialect-heavy settings remains challenging due to strong regional variation and limited labeled data. We propose GLoRIA, a parameter-efficient adaptation framework that leverages metadata (e.g., coordinates) to modulate low-rank updates in a pre-trained encoder. GLoRIA injects low-rank matrices into each feed-forward layer, with a gating MLP determining the non-negative contribution of each LoRA rank-1 component based on location metadata. On the GCND corpus, GLoRIA outperforms geo-conditioned full fine-tuning, LoRA, and both dialect-specific and unified full fine-tuning, achieving state-of-the-art word error rates while updating under 10% of parameters. GLoRIA also generalizes well to unseen dialects, including in extrapolation scenarios, and enables interpretable adaptation patterns that can be visualized geospatially. These results show metadata-gated low-rank adaptation is an effective, interpretable, and efficient solution for dialectal ASR.

Figures & tables

Description	Full Fine-Tuning, No Metadata		Geo-Conditioned Full Fine-Tuning		Geo-Conditioned	General-Purpose Models
Dialect Region	Dialect-	Joint Model	Coordinate Embedding	Modified Feedforward	GLoRIA (rank 128)	OWSM	Whisper	Pre-trained	Amount of Test Data
Dialect Region	Specific (Best)	Joint Model	Coordinate Embedding	Modified Feedforward	GLoRIA (rank 128)	V4-1B	Large-V3	Model	Amount of Test Data
Brabants	30.02	28.67	28.70	27.44	27.10	78.32	71.69	56.52	7h 47m
Frans-Vlaams	45.13	46.01	44.84	42.67	40.84	83.65	75.11	75.38	2h 34m
Oost-Vlaams	32.42	32.83	33.04	31.56	30.16	80.54	68.30	66.22	5h 03m
Oost-Vlaams $>$ Brabants	35.89	32.65	33.21	31.98	30.28	75.34	68.04	64.51	3h 16m

Table 1: Word Error Rate (%) comparison between GLoRIA and baseline models across different dialectal regions and models. The regions above the horizontal separator contain dialects that were seen during training, while the regions below unseen. For each region, the first column lists the lowest WER obtained among the five dialect-specific models trained on the training regions. It is followed by the joint model and the location-aware models, and then inference with the largest/latest Whisper and OWSM models, as well as the pre-trained model

Description	Rank 32		Rank 64		Rank 128
Dialect Region	LoRA	GLoRIA	LoRA	GLoRIA	LoRA	GLoRIA
Brabants	33.24	28.03	32.83	27.66	32.20	27.10
Frans-Vlaams	55.90	43.04	54.31	42.53	50.48	40.84
Oost-Vlaams	39.46	32.51	37.55	32.04	36.83	30.16
Oost-Vlaams $>$ Brabants	39.06	32.91	37.63	32.13	36.66	30.28
West-Vlaams	35.18	27.70	34.48	27.40	32.46	26.70

Table 2: WER (%) comparison of LoRA and GLoRIA across ranks. The final row shows the fraction of model parameters updated.

Panel of Figure 1 — Figure 1: Geographical distribution of activation for four NMF-derived adaptation components. Each point represents a location; color intensity reflects the degree of adaptation component usage at that location. The emergent spatial patterns closely correspond to known dialect regions. The patterns shown correspond to the Frans-Vlaams, Limburgs, Oost-Vlaams and Antwerp regions, from left to right.

Figure 2: Clustered heatmap of NMF component activations averaged by dialect region. Regions that are geographically and acoustically closer cluster together, indicating interpretable adaptation behavior.

Automatic Speech Recognition

Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.

See all 480 papers

Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition
Mar 18, 2026Yuxiang Mei, Delai Qiu, Shengping Liu +2Multilingual ASR Speech Language Models
Shanghai Engineering Research Center of Intelligent Education and Bigdata, Shanghai Normal University, Shanghai, 200234, China. · Unisound AI Technology Co., Ltd., Beijing, China. · SHNU-Unisound Natural Human-Computer Interaction Lab, Shanghai Normal University.
Itgan at NADI 2026 shared task: Parameter-Efficient Whisper Adaptation for Robust, Mixed-Dialect and Code-Switched Arabic ASR
Oct 7, 2026Ibrahim AlmajaiRobust ASR Automatic Speech Recognition
Independent Researcher
Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs
Sep 16, 2026Mohan Shi, Zilai Wang, Natarajan Balaji Shankar +3Domain Adaptation for ASR Adapter Tuning
Department of Electrical and Computer Engineering University of California Los Angeles Los Angeles, USA

Mar 18, 2026J/K move · Enter open · S save

Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition

Yuxiang Mei, Delai Qiu, Shengping Liu, Jiaen Liang+1

Shanghai Engineering Research Center of Intelligent Education and Bigdata, Shanghai Normal University, Shanghai, 200234, China. · Unisound AI Technology Co., Ltd., Beijing, China. · SHNU-Unisound Natural Human-Computer Interaction Lab, Shanghai Normal University.

Multilingual ASR Speech Language Models Parameter-Efficient Fine-Tuning Low-Rank Adaptation Low-Resource Speech Recognition

Open paper AskPDF arXiv

Speech Large Language Models (Speech-LLMs) have emerged as a powerful approach for automatic speech recognition (ASR) by aligning speech encoders with large language models. However, adapting these systems to multilingual settings with imbalanced data distributions remains challenging. In such scenarios, a stability-plasticity dilemma often arises: fully shared Parameter-Efficient Fine-Tuning (PEFT) can cause negative inter-lingual interference for under-represented languages, while fully language-specific tuning limits the cross-lingual beneficial knowledge transfer needed for low-resource tasks. To address this, we propose Zipper-LoRA, a novel rank-level decoupling framework with three variants (Static, Hard, and Soft) that dynamically synthesizes LoRA updates from shared and language-specific subspaces. By using a lightweight language-conditioned router, Zipper-LoRA dynamically controls the contribution of each subspace at the LoRA rank level, enabling fine-grained sharing where languages are compatible and strict decoupling when conflicts occur. To further stabilize optimization under imbalanced data, we propose a two-stage training strategy with an Initial-B warm start that significantly accelerates convergence. Experiments on a 12-language mixed-resource setting show that Zipper-LoRA consistently outperforms both fully shared and independent baselines, particularly in extremely low-resource scenarios. Moreover, we demonstrate that these gains are robust across both chunked and non-chunked encoder configurations, confirming the framework's reliability for practical, large-scale multilingual ASR. Our code and data will be available at https://github.com/YuCeong-May/Zipper-LoRA for reproducibility.

Abstract

Automatic Speech Recognition

Explore similar work

Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition

Itgan at NADI 2026 shared task: Parameter-Efficient Whisper Adaptation for Robust, Mixed-Dialect and Code-Switched Arabic ASR

Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs