Audio-Visual QA

QA: Question Answering

AVQA studies how models use synchronized audio, video, and a question to identify relevant evidence and answer about what happens, what is heard, and how the modalities relate. Common approaches include pretrained audio and video encoders, question-guided attention or cross-modal fusion, and methods that retrieve or temporally ground evidence before reasoning. Benchmarks range from music and sound-event questions to long videos, spatial reasoning, and tests of whether answers truly depend on both modalities. Models are improving, but benchmark shortcuts, cross-modal interference, long-context costs, and reliable spatial and temporal grounding remain open challenges.

Latest papers

28 papers9 papers in the last four weeks, with none the four weeks before. 0.1% of all new papers.

EvaOmni: An Evidence-Grounded Benchmark for Audio-Visual Reasoning
Oct 20, 2026Xiaoxing Hu, Wenxi Yao, Chaoyuan Shi +17Audio-Visual Understanding Audio-Visual QA
Shanghai Jiao Tong University · Shanghai Innnovation Institute · Inner Mongolia University of Technology +8
MLLM Frame Subset Ensembling for Audio-Visual Video QA and MLLM-based Reranking for Ad-hoc Video Search in TRECVID 2025
Oct 16, 2026Andreas Goulas, Damianos Galanopoulos, Ioannis Patras +1Audio-Visual QA Video Retrieval
Information Technologies Institute (ITI), Centre of Research and Technology Hellas (CERTH), Thessaloniki, Greece · Queen Mary University of London, London, UK
ANCHOR: Audio-Visually Grounded Chain-of-Thought Reasoning Benchmark
Oct 16, 2026Joel Julin, Souraja Kundu, Liza Dahiya +6Multimodal Model Evaluation Multimodal Grounding
Carnegie Mellon University · Amazon
Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-Visual LLMs
Oct 15, 2026Kim Sung-Bin, Lee Jung-Mok, Jinwoo Jung +4Audio-Visual Understanding Multimodal Large Language Models
POSTECH · KAIST · Hankuk University of Foreign Studies +1
Designing Textual Interfaces To Enhance Llm Reasoning On Audio-Visual Question Answering
Oct 14, 2026Yi Wang, Tianzi Wang, Tong Lei +2Multimodal Reasoning Automatic Prompt Optimization
School of Informatics, University of Edinburgh, UK · Department of Systems Engineering and Engineering Management, Chinese University of Hong Kong, Hong Kong SAR · Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, China
LEAP: Learned Block-wise Evidence Retrieval for Long Audio-Video Perception
Sep 30, 2026Juyi Lin, Zhiqiang Lao, Jiali Cui +9Audio-Visual QA Long-Video QA
Northeastern University · Futurewei Technologies
OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
Sep 30, 2026Junming Lin, Yuxuan Wang, Zhenxin Lei +11On-Policy Self-Distillation Multimodal Reasoning
School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, Peking University · Alibaba Token Hub, Alibaba Group
OP-CAD: On-Policy Clean-Audio Distillation for Robust Audio-Visual Reasoning
Sep 30, 2026Xingming Shui, Dapeng Chen, Bowei Liu +5Audio-Visual Understanding Multimodal Robustness
Tsinghua Shenzhen International Graduate School, Tsinghua University · Independent Researcher
SyncRA: Learning Temporal Correspondence in Omni-Modal Models
Sep 28, 2026Zelong Xu, Yan Li, Wenhe Hu +1Audio-Visual Representation Learning Audio-Visual Synchronization
University of Wisconsin–Madison · University of Alberta · Arizona State University
TAKE 85: Testing Audiovisual filmmaKer's intEnt across 85 Hours of Film
Aug 30, 2026Kaishuu Shinozaki-Conefrey, Olivier Pascaud, Robin Courant +3Audio-Visual Understanding Video-Language Model Evaluation
LIX, Ecole Polytechnique, IP Paris, Palaiseau, France · New York University, New York, NY, USA · Ecole nationale supérieure Louis-Lumière, Saint-Denis, France +1
Reasoning for Social Audio-Visual Question Answering: Where Do We Stand?
Aug 13, 2026Koen P. de Vries, Xavier Alameda-Pineda, Estefanía Talavera +1Supervised Fine-Tuning Multimodal Large Language Models
Inria at Univ. Grenoble Alpes, CNRS, LJK, France · University of Twente, The Netherlands
Qwen-MusicAVQA-7B: A Multimodal Model for Music Audio-Visual QA
Aug 11, 2026Maryam DehdashtiAudio-Visual QA
Inference Matter Labs
Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models
Aug 10, 2026Zhi Zeng, Cheng Zhang, Zesheng Yang +9Unified Multimodal Models Audio-Visual QA
Xi’an Jiaotong University · The Hong Kong Polytechnic University · National University of Singapore +2
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
Aug 10, 2026Puneet Mathur, Manan Suri, Dinesh ManochaEfficient Multimodal Inference Multimodal Memory
University of Maryland, College Park
Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos
Jul 17, 2026Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar +19Audio-Visual Understanding Multimodal Large Language Models
1NVIDIA, USA · University of Maryland, USA
Empowering Long-form Omni-modal Understanding with Robust Audio Perception
Jul 11, 2026Kaiying Yan, Luoyi Sun, Xiao Zhou +1Audio-Visual Understanding Multimodal Pretraining
SAI, Shanghai Jiao Tong University, shanghai, China · Zhejiang University, Hangzhou, Zhejiang, china · Shanghai AI Lab, shanghai, China
Q-TriM: Question-Guided Tri-Modal Attention for Audio-Visual Question Answering
Jul 4, 2026SungHun Kim, SeungJun BaekMultimodal QA Cross-Modal Attention
Dept. of Computer Science and Engineering Korea University Seoul, South Korea
OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains
Jun 12, 2026Xinyue Cai, Chaoyou Fu, Yi-Fan Zhang +2Multimodal QA Audio-Visual QA
1Nanjing University · 2CASIA
V-LynX: Token Interface Alignment for Video+X LLMs
May 30, 2026Jungin Park, Jiyoung Lee, Kwanghoon SohnCross-Modal Alignment Multimodal QA
Yonsei University, Seoul, South Korea · Ewha Womans University, Seoul, South Korea
O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding
May 26, 2026Peiran Wu, Yunze Liu, Chi-Hao Wu +2Audio-Visual Understanding Efficient Multimodal Inference
University of Bristol · Memories.ai Research · University of Central Florida
Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought
May 11, 2026Xuanchen Li, Yuheng Lu, Chenrui Cui +6Cross-Modal Learning Multimodal QA
Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, Tianjin, China · Tencent, China · Huiyan Technology Company, Ltd., Tianjin, China +1
TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos
May 8, 2026Hengyi Feng, Hao Liang, Mingrui Chen +6Audio-Visual Understanding Multimodal Hallucination
University of Electronic Science and Technology of China · Peking University · Zhongguancun Academy +1
KARMA-MV: A Benchmark for Causal Question Answering on Music Videos
May 5, 2026Archishman Ghosh, Abhinaba Roy, Dorien HerremansAudio-Visual Understanding Video QA
AMAAI Lab, Singapore University of Technology and Design
Retrieving to Recover: Towards Incomplete Audio-Visual Question Answering via Semantic-consistent Purification
Apr 12, 2026Jiayu Zhang, Shuo Ye, Qilang Ye +3Audio-Visual QA Missing-Modality Learning
School of Computing and Information Technology, Great Bay University · Tsinghua Shenzhen International Graduate School, Tsinghua University · College of Computer Science, Nankai University +2
LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs
Mar 19, 2026Keda Tao, Yuhua Zheng, Jia Xu +13Audio-Visual Understanding Long-Video Understanding
Zhejiang University · Westlake University · Ant Group +4
JAEGER: Joint 3D Audio-Visual Grounding and Reasoning in Simulated Physical Environments
Feb 20, 2026Zhan Liu, Changli Tang, Yuxin Wang +73D Spatial Reasoning 3D VLMs
Tsinghua University · Tencent AI Lab · HKUST +2
Query-Guided Spatial-Temporal-Frequency Interaction for Music Audio-Visual Question Answering
Jan 27, 2026Kun Li, Michael Ying Yang, Sami Sebastian BrandtMultimodal QA Audio-Visual QA
University of Twente, NL · University of Bath, UK · IT University of Copenhagen, DK
Do Modern Video-LLMs Need to Listen? A Benchmark Audit and Scalable Remedy
Sep 22, 2025Geewook Kim, Minjoon SeoVLM Evaluation Audio-Visual Understanding
NAVER Cloud AI, South Korea · KAIST AI, South Korea

Oct 20, 2026J/K move · Enter open · S save

EvaOmni: An Evidence-Grounded Benchmark for Audio-Visual Reasoning

Xiaoxing Hu, Wenxi Yao, Chaoyuan Shi, Ziqi Ye+16

Shanghai Jiao Tong University · Shanghai Innnovation Institute · Inner Mongolia University of Technology+8

Audio-Visual Understanding Audio-Visual QA Multimodal Model Evaluation Temporal Video Grounding

Open paper AskPDF arXiv

Omni LLMs can naturally locate key audio and visual evidence and integrate information across modalities for audiovisual reasoning. Existing audio-visual reasoning benchmarks that mainly rely on multiple-choice questions usually score models solely based on answer correctness, making it difficult to assess whether a model truly forms a reasoning chain from evidence to the correct answer. In addition, textual or unimodal shortcuts may allow models to answer correctly without analyzing the necessary audio-visual evidence, further weakening the reliability of evaluation benchmarks. To address these issues, we introduce EvaOmni, an evidence-grounded benchmark containing 392 videos and 1,027 purely human-annotated question-answer pairs, covering 15 capabilities and three levels: retrieval and localization, basic understanding, and complex reasoning. We design a series of rigorous annotation procedures in which experts annotate audio and visual evidence and construct video-based distractors; humans review whether questions meet strict inspection criteria, and finally models filter textual and unimodal shortcuts. Beyond high-quality question-answer pairs, we require models to simultaneously return the answer and supporting evidence intervals in order to evaluate the complete reasoning chain. We propose the Coverage Grounded Score (C.G.) to measure how much annotated evidence a model identifies, and the Bottleneck Grounded Score (B.G.) to penalize poor temporal localization in either modality; both are incorporated into answer correctness. Across 7 open-weight and 17 proprietary Omni LLMs, Gemini3.7-Flash achieved the highest overall accuracy (79.4%), C.G. (70.8%), and B.G. (35.1%). At Level 3, it reached 79.1% accuracy, but its B.G. was only 12.3%. These results show that even the most advanced models still have room for development in grounded audio-visual reasoning.