Training-Free Intelligibility-Guided Observation Addition for Noisy ASR

Haoyang Li; Changsong Liu; Wei Rao; Hao Shi; Sakriani Sakti; Eng Siong Chng

eess.ASFeb 24, 2026

Training-Free Intelligibility-Guided Observation Addition for Noisy ASR

Authors: Haoyang Li, Changsong Liu, Wei Rao, Hao Shi, Sakriani Sakti, Eng Siong Chng

Organizations: Nanyang Technological University, Singapore · Independent Researcher · Nara Institute of Science and Technology, Japan

Abstract

Automatic speech recognition (ASR) degrades severely in noisy environments. Although speech enhancement (SE) front-ends effectively suppress background noise, they often introduce artifacts that harm recognition. Observation addition (OA) addressed this issue by fusing noisy and SE enhanced speech, improving recognition without modifying the parameters of the SE or ASR models. This paper proposes an intelligibility-guided OA method, where fusion weights are derived from intelligibility estimates obtained directly from the backend ASR. Unlike prior OA methods based on trained neural predictors, the proposed method is training-free, reducing complexity and enhances generalization. Extensive experiments across diverse SE-ASR combinations and datasets demonstrate strong robustness and improvements over existing OA baselines. Additional analyses of intelligibility-guided switching-based alternatives and frame versus utterance-level OA further validate the proposed design.

Figures & tables

Figure 1: The proposed Confidence-guided OA pipeline.

Table 2

Method	GR-KAN MP-SENet		Demucs
Method	Simu	Real	Simu	Real
Utterance-level OA	16.73	24.03	22.87	35.84
Frame-level OA	16.87	25.30	25.19	37.76

Table 4: WER of utterance- (Section 2.1 ) and frame-level OA (Section 2.3 ) on Chime4 Simu/Real partition. ASR is Wav2Vec2.

Automatic Speech Recognition

Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.

See all 480 papers

Boosting ASR Robustness via Test-Time Reinforcement Learning with Audio-Text Semantic Rewards
Mar 5, 2026Linghan Fang, Tianxin Xie, Li LiuRobust ASR Automatic Speech Recognition
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Technical University of Munich, Munich, Germany
Parallel Time-Band Mixing with Learned Observation-Adding for Robust ASR Front-Ends
Aug 31, 2026Xingyu Shen, Runze Wang, Wei-Ping Zhu +1Automatic Speech Recognition Speech Enhancement
Department of Electrical and Computer Engineering, Concordia University, Canada · Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology, China · Department of Electrical and Computer Engineering, McGill University, Canada
Loud and Clear: Dynamic Activation Steering for Improving Speech Intelligibility in Noisy Environments
Oct 6, 2026Seymanur Akti, Alexander WaibelTTS Synthesis Controllable Speech Generation
Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany · KIT Campus Transfer (KCT), Karlsruhe, Germany · Carnegie Mellon University (CMU), Pittsburgh, USA

Mar 5, 2026J/K move · Enter open · S save

Boosting ASR Robustness via Test-Time Reinforcement Learning with Audio-Text Semantic Rewards

Linghan Fang, Tianxin Xie, Li Liu

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Technical University of Munich, Munich, Germany

Robust ASR Automatic Speech Recognition Test-Time Adaptation Domain Adaptation for ASR

Open paper AskPDF arXiv

Recently, Automatic Speech Recognition (ASR) systems (e.g., Whisper) have achieved remarkable accuracy improvements but remain highly sensitive to real-world unseen data (data with large distribution shifts), including noisy environments and diverse accents. To address this issue, test-time adaptation (TTA) has shown great potential in improving the model adaptability at inference time without ground-truth labels, and existing TTA methods often rely on pseudo-labeling or entropy minimization. However, by treating model confidence as a learning signal, these methods may reinforce high-confidence errors, leading to confirmation bias that undermines adaptation. To overcome these limitations, we present ASR-TRA, a novel Test-time Reinforcement Adaptation framework inspired by causal intervention. More precisely, our method introduces a learnable decoder prompt and utilizes temperature-controlled stochastic decoding to generate diverse transcription candidates. These are scored by a reward model that measures audio-text semantic alignment, and the resulting feedback is used to update both model and prompt parameters via reinforcement learning. Comprehensive experiments on LibriSpeech with synthetic noise and L2 Arctic accented English datasets demonstrate that our method achieves higher accuracy while maintaining lower latency than existing TTA baselines. Ablation studies further confirm the effectiveness of combining audio and language-based rewards, highlighting our method's enhanced stability and interpretability. Overall, our approach provides a practical and robust solution for deploying ASR systems in challenging real-world conditions.

Abstract

Automatic Speech Recognition

Explore similar work

Boosting ASR Robustness via Test-Time Reinforcement Learning with Audio-Text Semantic Rewards

Parallel Time-Band Mixing with Learned Observation-Adding for Robust ASR Front-Ends

Loud and Clear: Dynamic Activation Steering for Improving Speech Intelligibility in Noisy Environments