SWEAR: Section-Wise Efficient Audio Reasoning via Self-Rated Difficulty

Yixuan Li; Minghao Lv; Xuenan Xu; Zihao Zheng; Mengyue Wu

cs.AIOct 9, 2026

SWEAR: Section-Wise Efficient Audio Reasoning via Self-Rated Difficulty

Authors: Yixuan Li, Minghao Lv, Xuenan Xu, Zihao Zheng, Mengyue Wu

Organizations: MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University, China · Shanghai AI Lab, China · KlingAI Research, China

Abstract

Recent work on large audio-language models (LALMs) explores adaptive reasoning to balance accuracy and efficiency. However, existing approaches often rely on coarse-grained difficulty estimation, overlooking the diverse factors that determine reasoning effort. We address this limitation by deriving five-dimensional difficulty scores and compressed reasoning traces from complementary offline assessments. Based on this characterization, we propose SWEAR, a difficulty-aware reasoning framework that learns difficulty signals during supervised fine-tuning and uses reinforcement learning to adapt reasoning allocation across reasoning stages. Experiments on MMAU-test-mini and MMAR show that SWEAR improves accuracy while substantially reducing generated tokens, shortening reasoning for easy instances without sacrificing performance on difficult ones.

Figures & tables

Figure 1: Overview of our multi-dimensional difficulty-aware audio reasoning framework. (a) Offline linguistic and acoustic annotations estimate Planning, Caption, and Reasoning difficulty, which determine section ratings and compression budgets for structured targets. (b) Structured SFT teaches the model to produce concise, rated reasoning traces. (c) Curriculum RL progresses from easy to hard while jointly optimizing task quality, format, rating-conditioned section-length rewards, and teacher-anchored rating calibration under reference-policy KL regularization. Offline annotators are used only during training.

	MMAU-test-mini								MMAR
Model	Easy	Medium	Hard	Sound	Music	Speech	Overall	Token	Easy	Medium	Hard	Sound	Music	Speech	Overall	Token
Qwen2.5-Omni-7B [ 17 ]	73.87	70.96	61.26	71.47	66.47	68.17	68.7	14.34	60.18	60.36	56.16	58.18	50.00	62.93	58.9	13.18
Public Baselines
Audio-Reasoner-7B [ 16 ]	69.37	69.16	61.26	66.67	64.67	68.77	66.7	504.36	49.40	43.84	43.84	45.45	39.81	48.98	45.6	537.18
Ke-Omni-R-7B [ 22 ]	79.88	75.75	64.86	78.08	70.36	72.07	73.5	76.64	58.38	56.46	49.85	55.15	42.23	58.84	55.0	70.34
Audio-Flamingo 3 [ 2 ]	78.68	76.65	65.17	77.48	69.46	73.27	73.4	199.90	61.68	55.26	45.95	49.70	48.06	58.16	54.5	189.10

Table 1: Accuracy (%) across predicted difficulty levels and audio categories on MMAU-test-mini and MMAR. Overall denotes accuracy on the full test set, and Token denotes the average number of generated tokens. The best and second-best results are highlighted in bold and underlined , respectively. Token rankings include only models with explicit CoT.

Panel of Figure 2 — Figure 2: Accuracy (bars) and mean output tokens (lines) across difficulty under DICoTA standard on MMAU and MMAR.

		MMAU		MMAR
Setting	$K$	Acc. $\uparrow$	Token $\downarrow$	Acc. $\uparrow$	Token $\downarrow$
SWEAR	4	73.5	197.1	62.0	195.9
Component ablations
w/o rating calibration	4	70.3	188.5	59.4	193.6
w/o length shaping	4	74.2	216.6	60.4	225.5
w/o adaptive KL	4	71.6	198.4	61.5	197.7

Table 2: Curriculum RL component ablations and sensitivity to the number of curriculum buckets.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 373 papers

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models
Apr 20, 2026Xiang He, Chenxing Li, Jinting Wang +5RL for Language Model Reasoning Audio Reasoning
Tencent AI Lab, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models
Sep 9, 2026Yupei Li, Qiyang Sun, Mohamed Mady +4Audio-Language Model Evaluation Audio Reasoning
Imperial College London, UK · Technische Universität München München, Germany · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, AE +2
ALARM: Audio-Language Alignment for Reasoning Models
Mar 10, 2026Petr Grinberg, Hassan ShahmohammadiAudio-Language Models Audio Reasoning
EPFL, Switzerland · Sony Europe Ltd., Germany

Apr 20, 2026J/K move · Enter open · S save

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

Xiang He, Chenxing Li, Jinting Wang, Yan Rong+4

Tencent AI Lab, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

RL for Language Model Reasoning Audio Reasoning Audio Understanding Audio-Language Models Multimodal CoT Reasoning

Open paper AskPDF arXiv

Large Audio-Language Models (LALMs) have made significant progress in audio understanding, yet they primarily operate as perception-and-answer systems without explicit reasoning processes. Existing methods for enhancing audio reasoning rely either on supervised chain-of-thought (CoT) fine-tuning, which is limited by training data quality, or on reinforcement learning (RL) with coarse rewards that do not directly evaluate reasoning quality. As a result, the generated reasoning chains often appear well-structured yet lack specific acoustic grounding. We propose Audio-DeepThinker, a framework built on two core ideas. First, we introduce a hybrid reasoning similarity reward that directly supervises the quality of generated reasoning chains by combining an LLM evaluator assessing logical path alignment, key step coverage, and analytical depth with an embedding similarity component enforcing semantic alignment with reference reasoning chains. Second, we propose a progressive two-stage curriculum that enables high-quality CoT reasoning to emerge through pure RL exploration, without any supervised reasoning fine-tuning, from an instruction-tuned model that possesses no prior chain-of-thought capability. Stage 1 trains on foundational audio QA with the hybrid reward to foster basic reasoning patterns, while Stage 2 shifts to acoustically challenging boundary cases with an LLM-only reward for greater reasoning diversity. Audio-DeepThinker achieves state-of-the-art results on MMAR (74.0%), MMAU-test-mini (78.5%), and MMSU (77.26%), winning 1st Place in the Interspeech 2026 Audio Reasoning Challenge (Single Model Track). Interpretability analyses further reveal that RL training primarily reshapes upper-layer MoE gating mechanisms and that reasoning tokens crystallize progressively in the upper transformer layers, offering mechanistic insights into how audio reasoning emerges through exploration.

Abstract

Efficient Language Model Reasoning

Explore similar work

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models

ALARM: Audio-Language Alignment for Reasoning Models