Temporal Audio Grounding

Follow this topic Ask about this topic

Temporal audio grounding studies how models identify the time intervals that support descriptions, answers, or other claims about a recording. Approaches include frame- and event-level supervision, timestamp tokens and time-aware audio-language alignment, plus reinforcement learning with interval-based rewards; synthetic and pseudo-labeled data help scale training. Benchmarks cover dense sound-event captioning, speech and music, and long recordings where relevant moments must be found among overlapping or redundant audio. Models can recognize an event yet miss its timing, and precise localization remains difficult for brief events and long-form audio; gains from grounding training do not always transfer to broader understanding.

Latest papers

36 papers11 papers in the last four weeks, up 57% on the four weeks before. 0.1% of all new papers.

TAC2: Timestamped Audio Captioning as composable functions
Oct 27, 2026Swapnil Bhosale, Prem Seetharaman, Ke Chen +3Audio-Language Models Audio Captioning
University of Surrey, UK · Adobe Research, USA
RVG-Bench: A Benchmark for Reasoning-Driven Audio-Visual Temporal Grounding
Oct 23, 2026Yue Feng, Boshen Xu, Zihao Yue +5Temporal Video Grounding Multimodal Grounding
AIM3 Lab, Renmin University of China · Kling AI
Query-Conditioned Liquid Temporal Grounding for Long-Form Audio
Oct 22, 2026Yuelan Cheng, Yonggang Zhu, Yongqiang Chen +3Temporal Audio Grounding
Centre for Vision, Speech and Signal Processing, University of Surrey, UK · School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China · College of Information and Electrical Engineering, China Agricultural University, China
Heard But Not Said: Prompting Understates Temporal Information of Musical Moments in Frozen Large Audio-Language Models
Oct 21, 2026Eunsik ShinAudio-Language Models Audio-Language Model Evaluation
Music and Audio Research Group (MARG), Department of Intelligence and Information, Seoul National University
OmniGround: Strengthening Omni-Modal Temporal Grounding via Attention-Consistency Training
Oct 19, 2026Zixuan Chen, Depeng Wang, Hao Lin +5Temporal Video Grounding Multimodal Grounding
Shanghai Jiao Tong University · Ant Group · Independent Researcher
MUB: An Identity-Masked Benchmark for Music Understanding in Multimodal Language Models
Oct 16, 2026Kun Liu, Lyra TaoAudio-Language Model Evaluation Music Understanding
ATTEST: Evidence-Grounded Reinforcement Learning for Long-Audio Temporal Understanding
Oct 12, 2026Ashutosh Chaubey, Jinxi Guo, Minh Tran +2Temporal Audio Grounding Audio Understanding
Meta Superintelligence Labs · University of Southern California
Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucination
Oct 9, 2026Mohammad Khalooei, Mohammad SabokrouAudio-Language Models Audio-Language Model Hallucination Detection
Sharif University of Technology, Iran · Amirkabir University of Technology, Iran · New Uzbekistan University, Uzbekistan +1
TiTok: Audio-Visual LLM for Multi-Segment Temporal Grounding
Oct 7, 2026Eunji Shin, Dahyun Choi, Seungyeon Jo +2Multimodal Grounding Multimodal Large Language Models
Department of Artificial Intelligence, Ewha Womans University · Division of Artificial Intelligence & Software, Ewha Womans University
Revisiting Frame-Wise Saliency for Audio Moment Retrieval
Oct 5, 2026Tatsuya Komatsu, Hokuto MunakataDetection Transformer Audio Understanding
LY Corporation, Tokyo, Japan
RMS-AQA: A Two-Stage Spatial Audio Question Answering Benchmark for Real-World Domestic Environments
Oct 1, 2026Peihao Chen, Qing Wang, Lichun Fan +12Audio-Language Model Evaluation Audio QA
University of Science and Technology of China, China · Xiaomi Corporation, China · DataoceanAI, China +2
On Temporal Binding in Large Audio Language Models
Sep 28, 2026Paul Primus, Gerhard WidmerLLM Interpretability Audio Reasoning
Institute of Computational Perception, Johannes Kepler University Linz, Austria · LIT Artificial Intelligence Lab, Johannes Kepler University Linz, Austria
Finding Emotions Where They Belong: Rethinking Audio Emotion Recognition through Masked Temporal Affective Grounding
Sep 26, 2026Abdelrahman Mohamed, Lars Kai Hansen, Zheng-Hua TanSpeech Emotion Recognition Temporal Audio Grounding
Aalborg University Pioneer Centre for AI · Technical University of Denmark Pioneer Centre for AI
TEMA: Evidence-Grounded Temporal Question Answering in Multi-Turn Multi-Audio Dialogs
Sep 24, 2026Kaidi Yang, Hualei Wang, Zhaohui Wang +3Audio QA Temporal Audio Grounding
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China. · University of Chinese Academy of Sciences, Beijing, China.
MuLA-Bench: A Multilingual Long-Form Audio Understanding Benchmark via Multi-Tier Auditing
Sep 20, 2026Zeyu Yang, Xinyu Zhang, Zibo Bi +5Audio-Language Model Evaluation Audio Understanding
The Chinese University of Hong Kong, Shenzhen · Alibaba Token Hub, Alibaba Group · Xi’an Jiaotong University
If You Hear It, Help Find It: Orthogonal Knowledge Distillation for Open-Vocabulary Audio-Visual Event Localization
Sep 20, 2026Yi Xu, Cheng Chen, Wenzhuo LeiAudio-Visual Understanding Knowledge Distillation
Tongji University Shanghai, China
Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception
Sep 14, 2026Yanfeng Shi, Yan Song, Junhui Li +4Audio Understanding Audio-Language Models
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China, Hefei, China · ICT Cluster, Singapore Institute of Technology, Singapore
What Did the MLLM Hear? Token-Level Spectro-Temporal Grounding for Audio MLLM Explainability
Sep 11, 2026Lucia Cascone, Valeria Fraenza, Michele Nappi +2Multimodal Grounding Audio-Language Models
Department of Computer Science, University of Salerno, Italy
NVV-Locator: From Transcript Tags to Acoustic Boundaries for Fine-Grained Nonverbal Vocalization Grounding
Sep 9, 2026Yuang Cao, Bingshen Mu, Zhennan Lin +7Audio Understanding Sound Event Detection
School of Intelligence Science and Technology, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China
TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models
Aug 30, 2026Apoorva Kulkarni, Kaousheik Jayakumar, Sreyan Ghosh +3Audio Understanding Audio-Language Models
University of Maryland, College Park, USA · CNH Industrial India
Note-Level Temporal Grounding of Musical Concepts in Large Audio-Language Models
Aug 30, 2026Kun Fang, Ziyu Wang, Ichiro FujinagaMusic Understanding Audio-Language Models
Schulich School of Music, McGill University, Montr´eal, QC, Canada · CIRMMT (Centre for Interdisciplinary Research in Music Media and Technology), Montr´eal, Canada · Courant Institute, New York University, New York, USA +1
LongAudioSpan: Spanning the Duration and Depth of Audio Comprehension
Aug 26, 2026Wen Huang, Yunfei Chu, Meng Gao +2Audio-Language Model Evaluation Audio Reasoning
Qwen Team, Alibaba Group · Tsinghua University · The Chinese University of Hong Kong
CoSTALA: Compositional Spatio-Temporal Audio-Language Alignment via Multi-Grain Hierarchical Contrastive Learning
Aug 25, 2026Peiwei Ren, Jinbo Hu, Fang Kang +2Audio-Language Models Audio Understanding
Xi’an Jiaotong Liverpool University, China · MiLM Plus, Xiaomi Inc., China · Center for Machine Vision and Signal Analysis, University of Oulu, Finland +1
From Semantics to Readout: Mechanistic Understanding of Audio Tokens after Fine-Tuning for Temporal Audio Grounding
Jul 28, 2026Yujian Ma, Jinqiu Sang, Ruizhe Li +2LLM Grounding LLM Fine-Tuning
1Shanghai Institute of Artificial Intelligence for Education, East China Normal University, China · School of Computer Science and Technology, East China Normal University, China · School of Computer Science, University of Birmingham, UK +1
GigaChat Audio: Time-aware Large Audio Language Model
Jul 11, 2026Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov +4Audio QA Audio Understanding
SaluteDevices, Russia
Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding
Jul 5, 2026Zihan Zhang, Xize Cheng, Wenhao Yan +9Audio Understanding Audio-Language Models
Zhejiang University · Tsinghua University
MOSS-Audio Technical Report
Jun 1, 2026Chen Yang, Chufan Yu, Hanfu Chen +27Audio Understanding Audio-Language Models
MusTBench: Benchmarking and Advancing Temporal Grounding in Music LLMs
May 28, 2026Daeyong Kwon, Qiyu Wu, Shinobu Kuriya +6Audio-Language Model Evaluation Music Understanding
1Seoul National University · 2Sony Group Corporation · 3Sony AI
Towards Fine-grained Temporal Perception: Post-Training Large Audio-Language Models with Audio-Side Time Prompt
Apr 15, 2026Yanfeng Shi, Pengfei Cai, Jun Liu +5Audio-Language Models Temporal Audio Grounding
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China, Hefei, China · ICT Cluster, Singapore Institute of Technology, Singapore
SpotSound: Enhancing Large Audio-Language Models with Fine-Grained Temporal Grounding
Apr 14, 2026Luoyi Sun, Xiao Zhou, Zeqian Li +3Temporal Audio Grounding Audio-Language Models
Zhejiang University Hangzhou, Zhejiang, China · SAI, Shanghai Jiao Tong University Shanghai, China

Oct 27, 2026J/K move · Enter open · S save

TAC2: Timestamped Audio Captioning as composable functions

Swapnil Bhosale, Prem Seetharaman, Ke Chen, Haonan Chen+2

University of Surrey, UK · Adobe Research, USA

Audio-Language Models Audio Captioning Temporal Audio Grounding Speech Processing

Open paper AskPDF arXiv

Real-world audio applications require precise, queryable representations grounded in physical time. Yet modern Large Audio-Language Models (LALMs) increasingly favor conversational generation, producing free-form descriptions that conflate acoustic perception with higher-level reasoning and lack precise temporal grounding. We introduce TAC2, a foundation model that casts timestamped audio captioning as composable functions to construct granular, speaker-attributed acoustic timelines directly from in-the-wild audio. To avoid prohibitively expensive manual annotation, we programmatically fuse complementary predictions from specialized weak teachers through time-registered IoU matching. We model the resulting multi-granularity schema of timestamped events, speaker turns, and fine-grained speech attributes using an Instruction-Input-Output (IIO) completion format, in which each task is a function of the audio, an instruction, and an optional input. A single call decodes the full description autonomously, while composing calls lets earlier outputs or external priors, such as a reference transcript, condition later ones. We further extend TAC2 with expressive speech attributes to obtain TAC2-S, using parameter-efficient specialists to elicit latent paralinguistic capabilities from the pre-trained backbone and distill them into a unified model. We validate the resulting representation in two complementary settings. Paired with frozen text-only LLMs, it yields strong spoken audio understanding across diverse reasoning backends, demonstrating the utility of decoupling fine-grained acoustic perception from downstream reasoning. Used as paired supervision for text-to-audio generation, it supports semantically and temporally grounded reconstruction, together with explicit zero-shot control over event timing, speaker attributes, and word-level emphasis.