ALARM: Audio-Language Alignment for Reasoning Models

Petr Grinberg; Hassan Shahmohammadi

cs.CLMar 10, 2026

ALARM: Audio-Language Alignment for Reasoning Models

Authors: Petr Grinberg, Hassan Shahmohammadi

Organizations: EPFL, Switzerland · Sony Europe Ltd., Germany

Abstract

Large audio language models (ALMs) extend LLMs with auditory understanding. A common approach freezes the LLM and trains only an adapter on self-generated targets. However, this fails for reasoning LLMs (RLMs) whose built-in chain-of-thought traces expose the textual surrogate input, yielding unnatural responses. We propose self-rephrasing, converting self-generated responses into audio-understanding variants compatible with RLMs while preserving distributional alignment. We further fuse and compress multiple audio encoders for stronger representations. For training, we construct a 6M-instance multi-task corpus (2.5M unique prompts) spanning 19K hours of speech, music, and sound. Our 4B-parameter ALM outperforms similarly sized models and surpasses most larger ALMs on related audio-reasoning benchmarks, while preserving textual capabilities with a low training cost. Notably, we achieve the best open-source result on the MMAU-speech and MMSU benchmarks and rank third among all the models.

Figures & tables

Figure 1: Overview of the corpora collection pipeline.

Audio Type	$\#$ Elements (M)	$\#$ Hours (K)	$\#$ Unique Prompts (M)
Speech	$2.91/2.60/0.29$	$9.88/8.83/0.98$	$1.40/1.27/0.16$
Sound	$2.01/1.80/0.20$	$5.54/4.98/0.55$	$0.36/0.33/0.06$
Music	$0.59/0.53/0.06$	$2.45/2.21/0.24$	$0.16/0.14/0.03$
Instruction	$0.57/0.56/0.01$	$1.02/1.01/0.01$	$0.57/0.56/0.01$
Total	$6.08/5.49/0.56$	$18.89/17.03/1.78$	$2.49/2.3/0.26$
DeSTA [ 5 ]	$4.96/4.96/0.00$	$7.00/7.00/0.00$	$0.007/0.007/0$

Table 1: Overview of our compiled dataset (total / train / validation) in comparison with DeSTA.

Type	Model response
Original	We are given an audio snippet with metadata : [00:00-00:01] I’m on my way to the meeting. (duration: 1s, emotion: Neutral, gender: Male, age: 34, language: English) The prompt asks: ”What is the tone of the speaker’s voice in the audio?” From the metadata provided , we see: emotion: Neutral…
Rephrased (no limit)	Okay, I listen to the audio clip. The speaker says ’I’m on my way to the meeting’ during the first second (00:00-00:01). I perceive the emotional state as neutral, the gender as male, the age as 34, and the language as English. The prompt asks: ’What is the tone of the speaker’s voice in the audio?’…
Rephrased (limit: $1536$ )	I hear an audio clip that lasts 1 second, starting at 00:00 and ending at 00:01. The speaker says: ”I’m on my way to the meeting.” The prompt asks: ”What is the tone of the speaker’s voice in the audio?” From the audio I perceive : emotional tone: neutral…

Table 2: Original and rephrased responses with and without limits from Qwen3-4B-Thinking-2507

Figure 2: Overview of ALARM training pipeline. A pre-trained RLM first generates an initial response $R_{0}$ from textual metadata, then rephrases it into $R_{\text{text}}$ . Finally, the RLM is equipped with the audio-fusion module and trained to generate $R_{\text{text}}$ from the corresponding audio inputs.

Model	Size	No LLM Modification	$\#$ Samples $\|$ $\#$ Hours $\|$ $\#$ Tokens	MMSU
Model	Size	No LLM Modification	$\#$ Samples $\|$ $\#$ Hours $\|$ $\#$ Tokens	Perception $\uparrow$	Reasoning $\uparrow$	Overall $\uparrow$
GPT-4o Audio [ 62 ]	$-$	$-$	$-$	$39.7$	$71.2$	$56.4$
Gemini-1.5-Pro [ 63 ]	$-$	$-$	$-$	46.3	$76.1$	$60.7$
LTU [ 64 ]	7B	$\times$	$5.6$ M $\|$ $-$ $\|$ $-$	$20.8$	$24.4$	$22.6$
SALMONN [ 8 ]	7B	$\times$	$2.3$ M $\|$ $4.4$ K $\|-$	$29.8$	$30.0$	$30.1$
Baichuan-Omni-1.5 [ 65 ]	7B	$\times$	$-\|$ $887$ K $\|-$	$35.4$	$67.2$	$50.6$

Table 3: Accuracies on the MMSU benchmark. We further report the text LLM backbone size, whether it is kept frozen (

\checkmark

) or modified (

\times

), and the training data volume. Best results are in bold, second best underlined, third best are in italic.

Model	Size	MMLU-Pro [ 69 ] $\uparrow$	MMLU-Redux [ 70 ] $\uparrow$	GPQA [ 71 ] $\uparrow$	GSM8K [ 72 ] $\uparrow$	HumanEval [ 73 ] $\uparrow$
Megrez-3B-Instruct [ 55 ]	2B	$46.1$	$-$	$-$	$65.5$	$78.7$
Megrez-3B-Omni [ 55 ]	2B	$45.2$	$-$	$-$	$63.8$	$72.6$
Qwen2.5 [ 74 ]	7B	$56.3$	$75.4$	$36.4$	$91.6$	$84.8$
Qwen2.5-Omni [ 7 ]	7B	$47.0$	$71.0$	$30.8$	$88.7$	$78.7$
Qwen3-4B-Thinking-2507 [ 16 ]	4B	$74.0$	$86.1$	$65.8$	$-$	$-$
ALARM (ours, any variant)	4B	$74.0$	$86.1$	$65.8$	$-$	$-$

Table 4: Comparison of text LLMs/RLMs and their corresponding multimodal variants on different text benchmarks.

Model	Size	No LLM Modification	$\#$ Samples $\|$ $\#$ Hours $\|$ $\#$ Tokens	MMAU-v05.15.25 (Test-mini / Test)				MMAR
Model	Size	No LLM Modification	$\#$ Samples $\|$ $\#$ Hours $\|$ $\#$ Tokens	Sound $\uparrow$	Music $\uparrow$	Speech $\uparrow$	Mean $\uparrow$	Mean $\uparrow$
GPT-4o mini Audio [ 62 ]	$-$	$-$	$-$	$50.8$ / $49.7$	$39.2$ / $36.0$	$69.1$ / $67.5$	$53.0$ / $51.0$	$50.6$
GPT-4o Audio [ 62 ]	$-$	$-$	$-$	$64.6$ / $63.2$	$56.3$ / $49.9$	$66.7$ / $69.3$	$62.5$ / $60.8$	$63.5$
LTU [ 64 ]	7B	$\times$	$5.6$ M $\|$ $-$ $\|$ $-$	$20.4$ / $20.7$	$16.0$ / $15.7$	$15.9$ / $15.3$	$17.4$ / $17.2$	$19.2$
GAMA [ 57 ]	7B	$\times$	$6.3$ M $\|-\|-$	$31.8$ / $30.7$	$17.7$ / $17.3$	$12.9$ / $16.7$	$20.8$ / $21.7$	$26.5$
GAMA-IT [ 57 ]	7B	$\times$	$6.5$ M $\|-\|-$	$30.9$ / $32.7$	$26.7$ / $22.4$	$10.8$ / $11.6$	$22.8$ / $22.2$	$17.4$

Table 5: Results on MMAU and MMAR benchmarks. We further report the text LLM backbone size, whether it is kept frozen (

\checkmark

) or modified (

\times

), and the training data volume. Best results are in bold, second best underlined.

Figure 3: Comparison of our models on AIR-Bench benchmark across different tasks.

Explore similar work

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models
Apr 20, 2026Xiang He, Chenxing Li, Jinting Wang +5RL for Language Model Reasoning Audio Reasoning
Tencent AI Lab, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models
Sep 9, 2026Yupei Li, Qiyang Sun, Mohamed Mady +4Audio-Language Model Evaluation Audio Reasoning
Imperial College London, UK · Technische Universität München München, Germany · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, AE +2
Listen-to-Reason: Listen with Experts, Retrieve over a Graph, Reason with LLMs
Oct 7, 2026Pooneh Mousavi, Mirco Ravanelli, Cem SubakanAudio-Language Models Audio Reasoning
Concordia University · Mila – Quebec AI Institute · Laval University

Apr 20, 2026J/K move · Enter open · S save

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

Xiang He, Chenxing Li, Jinting Wang, Yan Rong+4

Tencent AI Lab, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

RL for Language Model Reasoning Audio Reasoning Audio Understanding Audio-Language Models Multimodal CoT Reasoning

Open paper AskPDF arXiv

Large Audio-Language Models (LALMs) have made significant progress in audio understanding, yet they primarily operate as perception-and-answer systems without explicit reasoning processes. Existing methods for enhancing audio reasoning rely either on supervised chain-of-thought (CoT) fine-tuning, which is limited by training data quality, or on reinforcement learning (RL) with coarse rewards that do not directly evaluate reasoning quality. As a result, the generated reasoning chains often appear well-structured yet lack specific acoustic grounding. We propose Audio-DeepThinker, a framework built on two core ideas. First, we introduce a hybrid reasoning similarity reward that directly supervises the quality of generated reasoning chains by combining an LLM evaluator assessing logical path alignment, key step coverage, and analytical depth with an embedding similarity component enforcing semantic alignment with reference reasoning chains. Second, we propose a progressive two-stage curriculum that enables high-quality CoT reasoning to emerge through pure RL exploration, without any supervised reasoning fine-tuning, from an instruction-tuned model that possesses no prior chain-of-thought capability. Stage 1 trains on foundational audio QA with the hybrid reward to foster basic reasoning patterns, while Stage 2 shifts to acoustically challenging boundary cases with an LLM-only reward for greater reasoning diversity. Audio-DeepThinker achieves state-of-the-art results on MMAR (74.0%), MMAU-test-mini (78.5%), and MMSU (77.26%), winning 1st Place in the Interspeech 2026 Audio Reasoning Challenge (Single Model Track). Interpretability analyses further reveal that RL training primarily reshapes upper-layer MoE gating mechanisms and that reasoning tokens crystallize progressively in the upper transformer layers, offering mechanistic insights into how audio reasoning emerges through exploration.

Abstract

Explore similar work

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models

Listen-to-Reason: Listen with Experts, Retrieve over a Graph, Reason with LLMs