Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models

Lok-Lam Ieong; Chia-Chien Chen; Chih-Kai Yang; Yu-Han Huang; An-Yu Cheng; Hung-yi Lee

cs.SDMar 15, 2026

Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models

Authors: Lok-Lam Ieong, Chia-Chien Chen, Chih-Kai Yang, Yu-Han Huang, An-Yu Cheng, Hung-yi Lee

Organizations: National Taiwan University, Taiwan

Abstract

Chain-of-thought (CoT) prompting has been extended to large audio-language models (LALMs) to elicit reasoning, yet enhancing its effectiveness without training remains challenging. We study inference-time model steering as a training-free approach to improve LALM reasoning. We introduce three strategies using diverse information sources and evaluate them across four LALMs and four benchmarks. Results show general accuracy gains up to 4.4% over CoT prompting. Notably, we identify a cross-modal transfer where steering vectors derived from few text samples effectively guide speech-based reasoning, demonstrating high data efficiency. We also examine hyperparameter sensitivity to understand the robustness of these approaches. Our findings position model steering as a practical direction for strengthening LALM reasoning.

Figures & tables

Figure 1: Example of reasoning enhanced by steering.

Figure 2: Overview of our three proposed methods in the extraction phase, along with the subsequent injection phase.

Model	Method	Hyperparams. ( $k$ , $\alpha$ )	College ( $\uparrow$ )	High School ( $\uparrow$ )	Elementary ( $\uparrow$ )	ReveAL-CoT ( $\uparrow$ )	ALL ( $\uparrow$ )	$\Delta$ ( $\uparrow$ )
Voxtral	Normal	–	29.0	36.7	56.9	53.5	47.5	–
	CoT	–	26.3	42.0	67.4	43.0	50.7	–
	Vanilla (Ours)	5, 0.025	32.0	43.3	68.8	56.0	55.0	+4.3
	SGS (Ours)	3, 0.1	29.3	43.9	65.8	57.0	53.8	+3.1
	TGS (Ours)	4, 0.025	26.3	41.3	67.1	54.5	52.8	+2.1
Phi4-mm	Normal	–	26.0	24.8	33.1	38.5	31.1	–

Table 1: Accuracies (%) on four evaluation benchmarks and their micro-average (ALL). Bold values denote the best result among the five settings for each model on each benchmark. “Hyperparams.” specifies the last

k

layers and the scaling factor

\alpha

used for each configuration.

\Delta

indicates the difference in average accuracy between our steering methods and the CoT baseline.

Model	Method	Col	HS	Elem	RC	ALL
Voxtral	Self-Con	25.0	36.3	66.9	51.0	50.4
Voxtral	Vanilla	32.0	43.3	68.8	56.0	55.0
Phi4-mm	Self-Con	43.0	45.2	65.6	65.0	57.3
Phi4-mm	Vanilla	35.0	47.0	69.1	69.5	59.3
Qwen2.5	Self-Con	58.0	74.1	81.8	66.5	73.8
Qwen2.5	Vanilla	56.0	79.6	83.9	74.0	77.6

Table 2: Self-consistency (Self-Con) vs. vanilla steering on College (Col), High School (HS), Elementary (Elem) math, and ReveAL-CoT (RC), with overall accuracy (ALL).

Panel of Figure 3 — Figure 3: Hyperparameter sensitivity of the steering methods.

Figure 4: Effect of external dataset size for SGS/TGS on Voxtral.

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 275 papers

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models
Apr 20, 2026Xiang He, Chenxing Li, Jinting Wang +5RL for Language Model Reasoning Audio Reasoning
Tencent AI Lab, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
ALARM: Audio-Language Alignment for Reasoning Models
Mar 10, 2026Petr Grinberg, Hassan ShahmohammadiAudio-Language Models Audio Reasoning
EPFL, Switzerland · Sony Europe Ltd., Germany
Learning When to Think While Listening in Large Audio-Language Models
May 26, 2026Zhiyuan Song, Weici Zhao, Yang Xiao +3Audio QA RL for Language Model Reasoning
University of Pennsylvania

Apr 20, 2026J/K move · Enter open · S save

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

Xiang He, Chenxing Li, Jinting Wang, Yan Rong+4

Tencent AI Lab, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

RL for Language Model Reasoning Audio Reasoning Audio Understanding Audio-Language Models Multimodal CoT Reasoning

Open paper AskPDF arXiv

Large Audio-Language Models (LALMs) have made significant progress in audio understanding, yet they primarily operate as perception-and-answer systems without explicit reasoning processes. Existing methods for enhancing audio reasoning rely either on supervised chain-of-thought (CoT) fine-tuning, which is limited by training data quality, or on reinforcement learning (RL) with coarse rewards that do not directly evaluate reasoning quality. As a result, the generated reasoning chains often appear well-structured yet lack specific acoustic grounding. We propose Audio-DeepThinker, a framework built on two core ideas. First, we introduce a hybrid reasoning similarity reward that directly supervises the quality of generated reasoning chains by combining an LLM evaluator assessing logical path alignment, key step coverage, and analytical depth with an embedding similarity component enforcing semantic alignment with reference reasoning chains. Second, we propose a progressive two-stage curriculum that enables high-quality CoT reasoning to emerge through pure RL exploration, without any supervised reasoning fine-tuning, from an instruction-tuned model that possesses no prior chain-of-thought capability. Stage 1 trains on foundational audio QA with the hybrid reward to foster basic reasoning patterns, while Stage 2 shifts to acoustically challenging boundary cases with an LLM-only reward for greater reasoning diversity. Audio-DeepThinker achieves state-of-the-art results on MMAR (74.0%), MMAU-test-mini (78.5%), and MMSU (77.26%), winning 1st Place in the Interspeech 2026 Audio Reasoning Challenge (Single Model Track). Interpretability analyses further reveal that RL training primarily reshapes upper-layer MoE gating mechanisms and that reasoning tokens crystallize progressively in the upper transformer layers, offering mechanistic insights into how audio reasoning emerges through exploration.

Abstract

Language Model Steering

Explore similar work

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

ALARM: Audio-Language Alignment for Reasoning Models

Learning When to Think While Listening in Large Audio-Language Models