Audio Reasoning

Follow this topic Ask about this topic

Audio reasoning asks models to answer questions and make decisions from speech, music, and environmental sound, including cues such as timing, source location, emotion, and overlapping events. Common approaches pair audio encoders with language models, then use structured chain-of-thought, source separation, acoustic tools, or reinforcement learning to connect answers to sound evidence. Benchmarks such as MMAU and MMAR test broad audio question answering, while PolyBench and long-audio and spatial-audio evaluations probe compositional relations, temporal grounding, and overlapping sources. A central challenge is keeping extended reasoning grounded in acoustics rather than language priors, while preserving accuracy across long recordings, accents, and noisy or polyphonic audio.

Latest papers

64 papers18 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

AudioRSI: Recursively Self-Improving Audio Reasoners
Oct 27, 2026Jiajun Fan, Roger Ren, Jingyuan Li +6Audio Reasoning Audio-Language Models
UIUC
Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks
Oct 25, 2026Jihoon Jeong, Mirco Ravanelli, Zhepei Wang +1Audio Reasoning Audio-Language Models
Mila-Québec AI Institute · Université Laval · Concordia University +1
Derivation Depth Exposes Reasoning Failures in Tool-Augmented Music Analysis
Oct 21, 2026Rupali Rajesh, Sethuraman T, Savya Khosla +4Music Understanding Audio-Language Model Evaluation
IIT Madras · UIUC · Google +2
Think Before You Transcribe: Context-Steered Reasoning for Hotword-Aware Speech Recognition
Oct 19, 2026Qihao Wang, Hang Su, Fan Cui +3Automatic Speech Recognition Contextual ASR
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · MiLM Plus, Xiaomi Inc.
Long Listening Thoughts: Eliciting Open Auditory Reasoning with Deliberative Perception and Cognitive Refinement
Oct 17, 2026Jaeyeon Kim, Chao-Han Huck Yang, Luoyi Zhang +5Audio Reasoning Audio-Language Models
Carnegie Mellon University · NVIDIA
RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can
Oct 9, 2026Yi-Jen Shih, Puyuan Peng, Haohe Liu +2Full-Duplex Spoken Dialogue Systems Positional Encoding
The University of Texas at Austin · FAIR, Meta Superintelligence Labs
SWEAR: Section-Wise Efficient Audio Reasoning via Self-Rated Difficulty
Oct 9, 2026Yixuan Li, Minghao Lv, Xuenan Xu +2Efficient Language Model Reasoning Audio Reasoning
MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University, China · Shanghai AI Lab, China · KlingAI Research, China
STEMMA: Song-to-Stem Multi-Audio Reasoning for Large Audio Language Models
Oct 8, 2026Hoyeol Sohn, Wonil Kim, Keunhyoung Kim +7Audio Reasoning Music Understanding
Graduate School of Culture Technology, KAIST, Daejeon, Republic of Korea · Neutune, Seoul, Republic of Korea
Listen-to-Reason: Listen with Experts, Retrieve over a Graph, Reason with LLMs
Oct 7, 2026Pooneh Mousavi, Mirco Ravanelli, Cem SubakanAudio-Language Models Audio Reasoning
Concordia University · Mila – Quebec AI Institute · Laval University
AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Models
Oct 3, 2026Lee Seung-woo, Bowen QiAudio-Language Models Audio Reasoning
Seoul National University · Shanghai Jiao Tong University
Hallucination Reduction for LLM-Based Audio Understanding via Multimodal Direct Preference Optimization
Oct 2, 2026Bebe Cosgrove, Aaron Isidore Grace, Weiran WangAudio Understanding Audio-Language Models
Smith College, Northampton, USA · University of Iowa, Iowa City, USA
AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models
Oct 1, 2026Yuxiang Wang, Kunyu Feng, Yuancheng Wang +12RL for Language Model Reasoning Audio Reasoning
The Chinese University of Hong Kong, Shenzhen · Tencent Hunyuan · Tsinghua University +2
AudioJev: Direct Audio Decisions with Order-Calibrated Probabilities
Oct 1, 2026Sihan Lv, Zhen Li, Zhiqi Cao +4Audio QA Audio Reasoning
Zhejiang University, Hangzhou, China · China Academy of Space Technology, Beijing, China · Innovation and Management Center of the School of Software (Ningbo), Zhejiang University, Ningbo, China +2
RMS-AQA: A Two-Stage Spatial Audio Question Answering Benchmark for Real-World Domestic Environments
Oct 1, 2026Peihao Chen, Qing Wang, Lichun Fan +12Audio-Language Model Evaluation Audio QA
University of Science and Technology of China, China · Xiaomi Corporation, China · DataoceanAI, China +2
Thinking in Depth, Speaking Directly: Recurrent Latent Reasoning for Paralinguistically Grounded Spoken Dialogue
Sep 29, 2026Shengbo Cai, Yuxiang Wang, Jingran Xie +5Audio Reasoning Speech Language Models
Tsinghua University · Tencent Hunyuan · The Chinese University of Hong Kong, Shenzhen
Probing Large Audio-Language Models for Compositional Understanding of Sounding Actions
Sep 28, 2026Michel Olvera, Paraskevas Stamatiadis, Changhong Wang +1Audio-Language Model Evaluation Audio Reasoning
S2A, LTCI, IDS · S2A, IDS
Almost Human, Except When It Matters: VoxParity and the Decisions a Voice Should Change
Sep 28, 2026Bhavik ManglaSpoken Dialogue Systems Audio Reasoning
Independent Research
On Temporal Binding in Large Audio Language Models
Sep 28, 2026Paul Primus, Gerhard WidmerLLM Interpretability Audio Reasoning
Institute of Computational Perception, Johannes Kepler University Linz, Austria · LIT Artificial Intelligence Lab, Johannes Kepler University Linz, Austria
SAIL: Spatial Audio Intelligence with Large Language Models via Disentangled Acoustic-Spatial Encoding and Dual-Stream Q-Former
Sep 28, 2026Zhengding Luo, Jinyang Wu, Haozhe Ma +3Audio Reasoning Direction-of-Arrival Estimation
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Singapore Management University · Tencent Hy Frontier Lab, Singapore +2
Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction
Sep 21, 2026Lujia Bao, Qian Chen, Luyao Cheng +15Tool-Augmented Language Model Agents Speech Foundation Models
Alibaba Token Foundry, Alibaba Group
Listen, Critique, and Refine: RL-Based Self-Refinement for Instruction-Following Speech Synthesis
Sep 21, 2026Chee-En Yu, Yi-Cheng Lin, Sung-Feng Huang +4TTS Synthesis Audio Reasoning
Graduate Institute of Electrical Engineering, National Taiwan University, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taiwan · NVIDIA Research 4 Artificial Intelligence Center of Research Excellence, National Taiwan University, Taiwan
Listen Then Reason: Perception-Grounded Test-Time Reinforcement Learning for Large Audio-Language Models
Sep 20, 2026Jiaheng Dong, Xiaofeng Yu, Jean Honorio +3Audio Reasoning Audio-Language Models
The University of Melbourne, Australia · University of Auckland, New Zealand · ARC OPTIMA, Australia +1
ParA-LLM: A Unified Approach to Paralinguistic and Acoustic Speech Understanding
Sep 19, 2026Nishit Anand, Jiaqi Su, Ke Chen +5Audio Understanding Audio-Language Models
Adobe Research, USA · University of Maryland, College Park, USA · OpenAI, USA
SEABED: SouthEast Asian Benchmark for Evaluating Audio Reasoning
Sep 18, 2026Harshit Rajgarhia, Asif Shaik, Rachuri Lokesh +2Audio Reasoning Audio-Language Model Evaluation
Centific AI Research
StepAudio 3 Realtime Technical Report
Sep 12, 2026Bin Lin, Bo Zhao, Boyang Zhang +87Audio-Language Models Full-Duplex Spoken Dialogue Systems
Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models
Sep 9, 2026Yupei Li, Qiyang Sun, Mohamed Mady +4Audio-Language Model Evaluation Audio Reasoning
Imperial College London, UK · Technische Universität München München, Germany · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, AE +2
ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection
Sep 3, 2026Taewoo Kim, Young Han Lee, Nam In Park +1Audio Deepfake Detection Explainable Deepfake Detection
Multi-Modal Research Center, KETI, South Korea · Digital Analysis Section, National Forensic Service, South Korea · Department of Artificial Intelligence, Korea University, South Korea
VoiceLongMemEval: Do Assistants Remember How You Sounded?
Sep 1, 2026Ramit Pahwa, Parivesh Priye, Apoorva BeeduAudio-Language Model Evaluation Long-Context Language Model Evaluation
Rivian VW Tech
LongAudioSpan: Spanning the Duration and Depth of Audio Comprehension
Aug 26, 2026Wen Huang, Yunfei Chu, Meng Gao +2Audio-Language Model Evaluation Audio Reasoning
Qwen Team, Alibaba Group · Tsinghua University · The Chinese University of Hong Kong
AudioLens: Multi-Perspective Speech Clustering with Reasoning Audio-Language Models
Aug 25, 2026Wenjun Huang, Qiaosong Chu, Tiger Shao +11Audio-Language Models Audio Reasoning
University of California, Irvine · Independent Researcher · Dartmouth College +1

Oct 27, 2026J/K move · Enter open · S save

AudioRSI: Recursively Self-Improving Audio Reasoners

Jiajun Fan, Roger Ren, Jingyuan Li, Rahul Pandey+5

UIUC

Audio Reasoning Audio-Language Models RL for Language Model Reasoning Language Model Self-Improvement Process Reward Models

Open paper AskPDF arXiv

Reinforcement learning has made Audio Large Language Models (Audio LLMs) better reasoners, but the reward that drives it still comes from outside the model: a final-answer check, a hand-crafted process score, or an external LLM judge that stays fixed while the policy improves. We show that an Audio LLM can supply this reward itself, and can do so recursively: each improved model becomes the critic that trains its successor, so the training signal improves along with the model. We introduce AudioRSI (Recursively Self-Improving Audio Reasoners), a reinforcement learning framework in which a single model plays two roles: as an actor it generates reasoning paths, and as a critic it compares them pairwise, turning its own judgments into a process-level reward. The critic is periodically replaced by the current policy, which closes the recursive loop. Because the critic hears the same recording as the actor, it can reject reasoning that cites acoustic evidence the audio does not contain, rather than reward reasoning that merely sounds plausible. The recursion is what sustains progress: under a frozen external judge, out-of-distribution accuracy plateaus during training, while a critic refreshed from the improving policy keeps it rising. Across four out-of-distribution benchmarks, all disjoint from the AVQA training distribution, AudioRSI outperforms strong baselines including Audio Flamingo 3 and Gemini 2.5 Pro on MMAU test-mini, ranks first on MMSU, achieves a clear lead on the challenging Multi-Audio split of MMAU-Pro, and leads the 7B open-source tier on MMAR among methods without external reward models. With no reward specifying how to reason, AudioRSI also develops structured reasoning behaviors (logical reasoning, perception-aware analysis, and reduced hallucination), corroborated by an external LLM-as-Judge evaluation and a keyword-based behavior analysis.