Audio-Language Model Evaluation

Follow this topic Ask about this topic

Momentum

25 papers in the last four weeks, up 108% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 126

All topics

MusTBench: Benchmarking and Advancing Temporal Grounding in Music LLMs
May 28, 2026Daeyong Kwon, Qiyu Wu, Shinobu Kuriya +6Audio-Language Model Evaluation Audio Understanding
1Seoul National University · 2Sony Group Corporation · 3Sony AI
ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood
May 28, 2026Tiantian Feng, Anfeng Xu, Xuan Shi +10Audio-Language Model Evaluation Speech Quality Assessment
University of Southern California · The Ohio State University · University of California, Los Angeles +3
KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs
May 27, 2026Haechan Kim, Seungjun Chung, Inkyu Park +2Audio-Language Model Evaluation Question Answering
1KRAFTON · Department of Mathematical Sciences, Seoul National University
VoiceGiraffe: A Benchmark for Extreme Long-Context Audio-Language Understanding
May 27, 2026Jashin Ye, Dongxiao Wang, Yixuan Ye +10Audio-Language Model Evaluation Audio Understanding
Future Living Lab, Alibaba
Do Audio LLMs Listen or Read? Analyzing and Mitigating Paralinguistic Failures with VoxParadox
May 26, 2026Jiacheng Pang, Ashutosh Chaubey, Mohammad SoleymaniAudio-Language Model Evaluation Speech Processing
Institute for Creative Technologies, University of Southern California, Los Angeles, USA
Why Can't They Remember? Uncovering Representation and Retrieval Bottlenecks in Multi-Turn Acoustic Memory
May 26, 2026Yang Xiao, Siyi Wang, Han Yin +4Audio-Language Model Evaluation Memorization in Language Models
1The University of Melbourne · 2KAIST · 3The University of Auckland +1
PitchBench: Measuring Pitch Hearing in Audio-Language Models
May 25, 2026Milan Liessens Dujardin, Song-Ze Yu, Craver Corbyn Thomas-Smith +2Audio-Language Model Evaluation Audio Understanding
University of California, Berkeley · Thoughtful Lab
Zero-Shot Parkinson's Disease Detection from Speech: Comparing Large Audio and Language Models
May 24, 2026Muhammad Ashad Kabir, Sirajam MuniraAudio-Language Model Evaluation Zero-Shot Learning
School of Computing, Mathematics and Engineering, Charles Sturt University, NSW, Australia · Department of Computer Science, Rensselaer Polytechnic Institute, NY, USA
Beyond Acoustic Emotion Recognition: Multimodal Pathos Analysis in Political Speech Using LLM-Based and Acoustic Emotion Models
May 21, 2026Juergen DietrichAudio-Language Model Evaluation Political Discourse Analysis
Democracy Intelligence gGmbH, Germany
Analyzing Error Propagation in Korean Spoken QA with ASR-LLM Cascades
May 17, 2026Donghyuk Jung, Youngwon ChoiAudio-Language Model Evaluation Question Answering
Korea Culture Technology Institute, Republic of Korea · Maum AI Inc., Republic of Korea
CBT-Audio: Evaluating Audio Language Models for Patient-Side Distress Intensity Estimation in CBT Session Recordings
May 17, 2026Qixuan Hu, Shuchang Ye, Xumou Zhang +8Audio-Language Model Evaluation Mental Health Counseling
School of Computer Science, Faculty of Engineering, University of Sydney, Australia · School of Psychology, Faculty of Science, University of Sydney, Australia · CHeBA (Centre for Healthy Brain Ageing), School of Clinical Medicine, Discipline of Psychiatry & Mental Health, The University of New South Wales, Australia +2
From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agents
May 14, 2026Md Tahmid Rahman Laskar, Xue-Yong Fu, Seyyed Saeed Sarfjoo +3Audio-Language Model Evaluation Voice Agent Evaluation
Dialpad Inc.
When Vision Speaks for Sound
May 13, 2026Xiaofei Wen, Wenjie Jacky Mo, Xingyu Fu +6Audio-Language Model Evaluation Audio-Visual Understanding
dUniversity of California, Davis · pPrinceton University · wUniversity of Wisconsin–Madison +1
Afrispeech Semantics: Evaluating Audio Semantic Reasoning in Spoken Language Models Across Domains and Accents
May 11, 2026Chibuzor Okocha, Christan GrantAudio-Language Model Evaluation Spoken Language Understanding
University of Florida
Benchmarking LLMs on the Massive Sound Embedding Benchmark (MSEB)
May 6, 2026Cyril Allauzen, Tom Bagby, Georg Heigold +2Audio-Language Model Evaluation LLM Evaluation
Google USA & Germany
JASTIN: Aligning LLMs for Zero-Shot Audio and Speech Evaluation via Natural Language Instructions
May 6, 2026Leying Zhang, Bowen Shi, Haibin Wu +2Audio-Language Model Evaluation Zero-Shot Learning
Auditory Cognition and Computational Acoustics Lab, School of Computer Science& MoE Key Laboratory of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, 200240 P. R. China · independent researchers, United States
AfriVox-v2: A Domain-Verticalized Benchmark for In-the-Wild African Speech Recognition
May 5, 2026Busayo Awobade, Gabrial Zencha Ashungafac, Tobi OlatunjiAudio-Language Model Evaluation ASR Evaluation
Intron Health
When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition
May 4, 2026Pehuén Moure, Niclas Pokel, Bilal Bounajma +4Audio-Language Model Evaluation ASR Evaluation
Institute of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland · Department of Mathematics, ETH Zurich, Switzerland · Department of Computational Linguistics, University of Zurich, Switzerland
All That Glitters Is Not Audio: Rethinking Text Priors and Audio Reliance in Audio-Language Evaluation
Apr 27, 2026Leonardo Haw-Yang Foo, Chih-Kai Yang, Chen-An Li +2Audio-Language Model Evaluation Audio QA
National Taiwan University · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)
SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech Generation
Apr 22, 2026Ruohan Liu, Shukang Yin, Tao Wang +6Audio-Language Model Evaluation LLM Evaluation
1Nanjing University · 2Xiaomi
HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models
Apr 21, 2026Feiyu Zhao, Yiming Chen, Wenhuan Lu +3Audio-Language Model Evaluation Audio-Language Model Hallucination Detection
College of Intelligence and Computing, Tianjin University, China · ASUS Intelligent Cloud Services, Singapore
Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval
Apr 20, 2026HaeJun Yoo, Yongseop Shin, Insung Lee +2Audio-Language Model Evaluation Multimodal IR
Sogang University
Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment
Apr 20, 2026Woody Haosheng Gan, William Held, Diyi YangAudio-Language Model Evaluation Human Preference Modeling
University of Southern California · Stanford University · OpenAthena
VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech
Apr 19, 2026Yi-Cheng Lin, Yusuke Hirota, Sung-Feng Huang +1Audio-Language Model Evaluation Gender Bias in Language Models
Graduate Institute of Communication Engineering National Taiwan University Taipei, Taiwan · NVIDIA Research NVIDIA Taipei, Taiwan · AI Center of Research Excellence National Taiwan University Taipei, Taiwan
Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Use
Apr 17, 2026Ramit Pahwa, Apoorva Beedu, Parivesh Priye +4Audio-Language Model Evaluation Voice Agent Evaluation
Rivian and Volkswagen Group Technologies
KoALa-Bench: Evaluating Large Audio Language Models on Korean Speech Understanding and Faithfulness
Mar 30, 2026Jinyoung Kim, Hyeongsoo Lim, Eunseo Seo +4Audio-Language Model Evaluation Speech Processing
Department of Artificial Intelligence, Chung-Ang University · Upstage AI
When Demonstrations Fail: Diagnosing the Limits of In-Context Learning in Large Audio-Language Models with Progressive Cue Removal
Mar 20, 2026Yen-Ting Piao, Jay Chiehen Liao, Wei-Tang Chien +5Audio-Language Model Evaluation LLM Evaluation
National Taiwan University, Taiwan
PARSA-Bench: A Comprehensive Persian Audio-Language Model Benchmark
Mar 15, 2026Mohammad Javad Ranjbar Kalahroodi, Mohammad Amini, Parmis Bathayan +2Audio-Language Model Evaluation Audio Reasoning
School of Electrical and Computer Engineering, University of Tehran, Iran · Institute for Research in Fundamental Sciences (IPM), Tehran, Iran
MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Models
Mar 10, 2026Chih-Kai Yang, Yun-Shao Tsai, Yu-Kai Guo +7Audio-Language Model Evaluation Audio Understanding
Graduate Institute of Communication Engineering, National Taiwan University, Taiwan · National Taiwan University, Taiwan · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE), Taiwan
RA-QA: A Benchmarking System for Respiratory Audio Question Answering Under Real-World Heterogeneity
Feb 4, 2026Gaia A. Bertolino, Yuwei Zhang, Tong Xia +2Audio-Language Model Evaluation Audio QA
University of Cambridge · Cambridge, United Kingdom · Tsinghua University +3

May 28, 2026, cs.CLJ/K move · Enter open · S save

MusTBench: Benchmarking and Advancing Temporal Grounding in Music LLMs

Daeyong Kwon, Qiyu Wu, Shinobu Kuriya, Junghyun Koo+5

1Seoul National University · 2Sony Group Corporation · 3Sony AI

Audio-Language Model Evaluation Audio Understanding Audio-Language Models Temporal Audio Grounding

Open paper AskPDF arXiv

Recent Large Audio-Language Models (LALMs) have demonstrated promising abilities in understanding musical content. However, whether their responses are grounded in the correct temporal regions of the audio remains underexplored. This limitation is particularly critical for music understanding, where key information often occurs as temporally localized events, such as instrument entries and rhythmic transitions. To address this gap, we introduce MusTBench, a music-expert-validated benchmark designed to evaluate temporal grounding in LALMs through five temporally grounded question-answering tasks. To further improve temporal grounding in existing models, we propose MusT, a novel four-stage temporal optimization recipe spanning music encoder adaptation, LLM adaptation, LLM supervised fine-tuning, and RL-based optimization. Experiments on MusTBench show that existing LALMs struggle with precise temporal grounding, while MusT brings significant improvements over strong baselines. These results establish temporal grounding as a key missing capability in current LALMs and position MusTBench as a challenging benchmark for future research in temporally grounded music understanding.

Momentum

Related topics

Latest papers 126