cs.CVNov 4, 2026

SleuthVLM: Do We Really Need Audio-in-Video for Omni Video Understanding?

Authors: Peiran Wu, Fangfei Gou, Zhaoxi Mu, Changhao Zhang, Jian Liu, Weiqiang Wang, Yuntao Wang, Yuanchun Shi, 

Organizations: University of Bristol · Ant Group · Tsinghua University

Abstract

Omni models answer questions about videos by jointly processing sampled frames and audio in one turn, with progress driven largely by scaling model size and omni training data. We ask whether a model needs to hear the audio directly at all, and present SleuthVLM, an agent harness built on a VLM that never hears the audio directly: speech will be transcribed into text, and the VLM will sleuth the video on its own without any predefined workflow. On four omni benchmarks with videos up to 90 minutes, SleuthVLM achieves the best average accuracy: 75.0%, 2.6 points above Gemini 3.8 Flash, and it also tops the Video-MME-v2 leaderboard with 77.8%; with an open-source 27B VLM, it beats all open-source omni models. We draw three findings: SleuthVLM beats a model that glances; hearing is not the bottleneck, agentic ability is; omni understanding needs better agentic ability, not just more omni data for modality alignment. So we provide a simple but effective starting point for doing omni video understanding. Omni understanding needs better agents, not just bigger models that hear.

Figures & tables

Explore similar work

CardsList
  1. Native Active Perception as Reasoning for Omni-Modal Understanding

    Jun 17, 2026Zhenghao Xing, Ruiyang Xu, Yuxuan Wang +8Audio-Visual UnderstandingActive Perception

  2. OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understanding

    Oct 13, 2026Xuyang Liu, Shihao Wang, Xinyu Wei +5Video-Language Model EvaluationMultimodal Grounding

  3. When Vision Speaks for Sound

    May 13, 2026Xiaofei Wen, Wenjie Jacky Mo, Xingyu Fu +6Audio-Language Model EvaluationAudio-Visual Understanding