Audio Understanding

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 325% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 170

All topics

MiDashengLM-Spatial: Unifying General Audio Understanding and Spatial Awareness
Oct 8, 2026Jinbo Hu, Hang Su, Lichun Fan +10Audio-Language Models Audio Understanding
Xiaomi Inc., China
CARES: A Controlled Synthetic Benchmark of Speaker Reactions to Sound
Oct 7, 2026Marcel Gibier, Thomas Thebaud, Olivier Boëffard +1Audio Understanding Audio Captioning
Inria, Paris, France · AMIAD, France
ARIA: Audio-Driven Melody-Tone Relation Modeling for Cantonese Lyric Authoring
Oct 6, 2026Shengyu Li, Jinting Wang, Li LiuAudio Understanding Text Generation
The Hong Kong University of Science and Technology (Guangzhou)
SkillFormer: Skill-Decomposed Adaptation for Audio Language Models
Oct 6, 2026Lee Seung-woo, Bowen Qi, Kim Min-jun +1Multi-Task Learning Audio Understanding
Pusan National University · Shanghai Jiao Tong University · Hanyang University
Logbook: Extremely Long-form Audio Event Understanding
Oct 5, 2026Kwanghee Choi, Suwon Shon, Dmitriy Serdyuk +7Audio-Language Model Evaluation Audio Understanding
UT Austin, USA · Meta Reality Labs, USA
Revisiting Frame-Wise Saliency for Audio Moment Retrieval
Oct 5, 2026Tatsuya Komatsu, Hokuto MunakataDetection Transformer Audio Understanding
LY Corporation, Tokyo, Japan
Revisiting Input Time-frequency Representations in Multi-pitch Estimation for Vocal Ensembles
Oct 2, 2026Junyoung Koh, Hao-Wen DongMusic Information Retrieval Audio Understanding
Yonsei University · University of Michigan
LAST: Looped Audio Spectrogram Transformer
Oct 1, 2026Haider Al-Tahan, Sean O'Brien, Anastasia Razdaibiedina +1Audio Understanding Recurrent Transformers
Georgia Institute of Technology · University of California San Diego · Google DeepMind
Multi-agent Auditory Scene Analysis: Improved Localization Speed and Robustness by Multi-beamformed Speech Quality Feedback
Sep 30, 2026Caleb RasconMulti-Agent System Optimization Direction-of-Arrival Estimation
Instituto de Investigaciones en Matematicas Aplicadas y en Sistemas, Universidad Nacional Autonoma de Mexico, Circuito Escolar 3000, Coyoacan, 03740, Ciudad de Mexico, Mexico.
UniAE-MoE: A Unified Audio Encoder via Mixture of Experts
Sep 30, 2026Shengbo Cai, Zhisheng Zhang, Zichao Nie +3Audio Representation Learning Mixture of Experts
Tsinghua University, China
Audio Token Attention Is Predictable Before the Language Model Runs
Sep 30, 2026Kyoungjun Park, Yunzhe Li, Lili QiuAudio Token Compression Audio Understanding
The University of Texas at Austin
Probing Large Audio-Language Models for Compositional Understanding of Sounding Actions
Sep 28, 2026Michel Olvera, Paraskevas Stamatiadis, Changhong Wang +1Audio-Language Model Evaluation Audio Reasoning
S2A, LTCI, IDS · S2A, IDS
NSV-Shift: A Contrastive Benchmark for Non-Speech Vocalization Understanding and Response Adaptation in Speech-to-Speech Models
Sep 27, 2026Ziwei ChenSpeech Generation Evaluation Audio Understanding
University of California San Diego
When Does the Concept of "Dog" Emerge in an Audio LLM?
Sep 27, 2026Zhe Wang, Shiqi Liu, Ruiyun Zhong +2Audio Understanding Audio-Language Models
Huazhong University of Science and Technology, Wuhan, China
EvoAudio: Recursive Self-Improvement for Audio Understanding
Sep 23, 2026Yuxiang Wang, Shengbo Cai, Yingda Shen +6Synthetic Data Generation Audio Understanding
The Chinese University of Hong Kong, Shenzhen · Tencent Hunyuan · Tsinghua University +1
When Entanglement Lower-Bounds Disparity: Auditing and Repairing Demographic Fairness in Audio Understanding Models
Sep 23, 2026Kian Shamsaie, Iman ModarressiDisentangled Representation Learning Audio Representation Learning
People Make Things
MuLA-Bench: A Multilingual Long-Form Audio Understanding Benchmark via Multi-Tier Auditing
Sep 20, 2026Zeyu Yang, Xinyu Zhang, Zibo Bi +5Audio-Language Model Evaluation Audio Understanding
The Chinese University of Hong Kong, Shenzhen · Alibaba Token Hub, Alibaba Group · Xi’an Jiaotong University
Discrete vs. Continuous: A Comprehensive Study of Unified Audio Understanding in LALMs
Sep 19, 2026Jing Peng, Zichao Nie, Zhisheng Zhang +2Audio-Language Model Evaluation Audio Representation Learning
Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception
Sep 14, 2026Yanfeng Shi, Yan Song, Junhui Li +4Audio Understanding Audio-Language Models
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China, Hefei, China · ICT Cluster, Singapore Institute of Technology, Singapore
Unifying Score and Performance for Fine-Grained Music Understanding in Audio-Language Models
Sep 9, 2026Milan Liessens Dujardin, Song-Ze Yu, Kevin MiaoAudio Understanding Audio-Language Models
NVV-Locator: From Transcript Tags to Acoustic Boundaries for Fine-Grained Nonverbal Vocalization Grounding
Sep 9, 2026Yuang Cao, Bingshen Mu, Zhennan Lin +7Audio Understanding Sound Event Detection
School of Intelligence Science and Technology, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China
Source-Adaptive Data Curation for Bilingual NVV-Aware ASR
Sep 9, 2026Yuang Cao, Qirui Zhan, Jingbin Hu +7Speech Processing Audio Understanding
Shenzhen Pimei Technology, China · Yutuzhineng, China · Nanjing University, China
Geometry-Informed Distributed Acoustic Scene Understanding
Sep 7, 2026Yiyuan Yang, Shitong Xu, Niki Trigoni +1Scene Graph Generation Audio Understanding
Department of Computer Science, University of Oxford, United Kingdom
Auditory Illusion Benchmark for Large Audio Language Models
Sep 2, 2026Hayoon Kim, Eunice Hong, Kyogu LeeAudio Understanding
Music and Audio Research Group, Seoul National University · Department of Intelligence and Information, Seoul National University · AIIS, Seoul National University +1
TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models
Aug 30, 2026Apoorva Kulkarni, Kaousheik Jayakumar, Sreyan Ghosh +3Audio Understanding Audio-Language Models
University of Maryland, College Park, USA · CNH Industrial India
LongAudioSpan: Spanning the Duration and Depth of Audio Comprehension
Aug 26, 2026Wen Huang, Yunfei Chu, Meng Gao +2Audio-Language Model Evaluation Audio Reasoning
Qwen Team, Alibaba Group · Tsinghua University · The Chinese University of Hong Kong
Robust Multi-Tier Infant-Centered Audio Understanding with Whisper via Structured Speaker Conditioning
Aug 12, 2026Xulin Fan, Jialu Li, Mohammad Nur Hossain Khan +4Speech Foundation Models Domain Generalization
University of Illinois Urbana-Champaign, USA · University of Arizona, USA · Worcester Polytechnic Institute, USA
From Prompting to Describing: A Cross-Cultural Study of Language for AI-Generated Music
Aug 6, 2026Sangheon Park, Claire ArthurText-to-Music Generation Audio Understanding
Georgia Institute of Technology School of Music
The Learning Objective Governs Perceptual Narrowing: A Cross-Lingual, Layer-Wise, Ten-Seed Study of Self-Supervised Speech Encoders
Aug 1, 2026Sejin YooAudio Understanding Self-Supervised Speech Representation Learning
Independent Researcher
TORUS: A Test of Rendering-Understanding Self-Coherence for Unified Audio Models
Jul 30, 2026Aryan Vijay Bhosale, Harshit Rajgarhia, Abhishek Mukherji +1Audio-Language Model Evaluation Audio Editing
Centific Global Solutions Inc. · University of Maryland

Oct 8, 2026, cs.SDJ/K move · Enter open · S save

MiDashengLM-Spatial: Unifying General Audio Understanding and Spatial Awareness

Jinbo Hu, Hang Su, Lichun Fan, Heinrich Dinkel+9

Xiaomi Inc., China

Audio-Language Models Audio Understanding Spatial Audio Sound Event Detection

Open paper AskPDF arXiv

Large audio-language models (LALMs) have achieved strong performance in general audio understanding, yet most are designed for monaural input and discard the inter-channel cues essential for spatial perception. In contrast, existing spatial audio-language models are purpose-built for spatial tasks and fail to capitalize on the general understanding capabilities of monaural LALMs. We present MiDashengLM-Spatial, the first open-source end-to-end unified audio-language model, to our knowledge, which supports both general audio understanding and spatial awareness within a single architecture. It extends MiDashengLM with a spatial audio encoder, Spatial-Dasheng, integrated through a hierarchical semantic-to-spatial conditioning module that injects intermediate semantic representations into the spatial branch at multiple depths while preserving the original semantic pathway. To provide spatial audio-language supervision at scale, we develop a data synthesis pipeline that renders diverse spatial acoustic scenes with scene-level spatial descriptions and question-answer pairs. Experiments show that Spatial-Dasheng achieves strong performance on sound event localization and detection in real-world scenes, and that MiDashengLM-Spatial substantially outperforms existing LALMs on spatial understanding and reasoning benchmarks. Meanwhile, it remains competitive with state-of-the-art 8B-scale LALMs on diverse monaural benchmarks, demonstrating that spatial awareness can be acquired without compromising general audio understanding. The source code and model checkpoint are available at https://github.com/xiaomi-research/midashenglm-spatial and https://huggingface.co/mispeech/midashenglm-spatial.

Momentum

Related topics

Latest papers 170