Omni models answer questions about videos by jointly processing sampled frames and audio in one turn, with progress driven largely by scaling model size and omni training data. We ask whether a model needs to hear the audio directly at all, and present SleuthVLM, an agent harness built on a VLM that never hears the audio directly: speech will be transcribed into text, and the VLM will sleuth the video on its own without any predefined workflow. On four omni benchmarks with videos up to 90 minutes, SleuthVLM achieves the best average accuracy: 75.0%, 2.6 points above Gemini 3.8 Flash, and it also tops the Video-MME-v2 leaderboard with 77.8%; with an open-source 27B VLM, it beats all open-source omni models. We draw three findings: SleuthVLM beats a model that glances; hearing is not the bottleneck, agentic ability is; omni understanding needs better agentic ability, not just more omni data for modality alignment. So we provide a simple but effective starting point for doing omni video understanding. Omni understanding needs better agents, not just bigger models that hear.
Figures & tables
Figure 1: SleuthVLM never hears the audio directly, yet beats the omni models that do. (a) Average accuracy on four omni benchmarks against model size; closed-source models are on the right, and the blue band spans the closed-source omni models. (b) Accuracy on the four omni benchmarks, Video-MME-v2 and their average, with every axis spanning 40 points; numbers give SleuthVLM (Opus 5.5) and, in parentheses, its difference from the best closed-source omni model.
Figure 2: Overview of SleuthVLM. (a) Once per video, an ASR model transcribes the speech and a forced aligner assigns timestamps to each word, preserving non-speech intervals as gaps. (b) A VLM gathers evidence by calling tools over the muted video and transcript in a sandbox; only omni reasoning models also have access to listen_audio . (c) The model answers once it has sufficient evidence, subject to fixed budgets. The illustrated trajectory is a real example from Daily-Omni, answered correctly in 13 turns.
Model
Input
Daily-Omni
WorldSense
OmniVideoBench
LVOmniBench
Avg.
Closed-Source Omni Models
Gemini 3.8 Flash
A+V
84.0
69.6
65.2
70.7
72.4
Qwen3.8-Omni-Flash ( Team, 2026a )
A+V
85.1
68.5
63.4
63.3
70.1
Qwen3.5-Omni-Plus ( Team, 2026b )
A+V
85.1
63.9
53.8
53.2
64.0
Seed 2.0 Lite
A+V
81.4
67.3
58.5
New A
New A
Muse Spark 1.2
A+V
79.6
65.0
62.2
New A
New A
Table 1: Comparison with omni models and agents on four omni benchmarks (accuracy, %). A+V: audio as input; V+ASR: speech as a timed transcript, no direct audio. Bold : best; underline : second best; New A: not reported; Avg.: mean over the four benchmarks. In parentheses: difference from the best open-source omni model ( green ) or the best closed-source omni model or Qwen-code agent ( red ).
Model
DO
WS
OVB
LV
VM2
Avg.
SleuthVLM (Opus 5.5)
84.4
70.1
69.9
75.5
77.8
75.5
Gemini 3.8 Flash
84.0
69.6
65.2
70.7
71.0
72.1
Qwen3.8-Omni-Flash
85.1
68.5
63.4
63.3
65.0
69.1
SleuthVLM (Qwen3.8-27B)
80.4
64.5
60.4
61.7
62.0
65.8
Qwen3.5-Omni-Plus
85.1
63.9
53.8
53.2
47.9
60.8
Gemini 3.8 Flash (QC)
New A
New A
70.1
70.7
72.7
New A
Table 2: SleuthVLM (Opus 5.5) achieves the best average over the four omni benchmarks and Video-MME-v2 (accuracy, %). DO: Daily-Omni; WS: WorldSense; OVB: OmniVideoBench; LV: LVOmniBench; VM2: Video-MME-v2; Avg.: mean over the five benchmarks. QC: the model run as an agent in the Qwen Code harness. Bold : best; underline : second best; New A: not reported.
Difficulty
Ability
Model
Input
Low
Medium
High
Underst.
Percept.
Infer.
Logical
Overall
(314)
(441)
(259)
(230)
(386)
(289)
(109)
(1014)
SleuthVLM (Opus 5.5)
V+ASR
82.5
76.0
66.4
80.4
74.4
70.6
82.6
75.5
SleuthVLM (Qwen3.8-Omni-Flash)
V+ASR+Audio
80.6
71.4
57.1
73.9
68.7
71.6
67.9
70.6
SleuthVLM (Gemini 3.8 Flash)
V+ASR+Audio
82.8
72.6
67.2
77.4
74.6
70.6
77.1
74.4
Qwen3.8-Omni-Flash
A+V
New A
New A
New A
New A
New A
New A
New A
63.3
Table 3: LVOmniBench accuracy (%) by difficulty and ability for reasoning models in our harness. V+ASR+Audio denotes omni models with access to listen_audio ; A+V denotes closed-source omni models without our harness. Ability groupings and per-type results appear in Sections 7 and 8 , respectively. Bold and underline mark the best and second-best results; question counts are in parentheses.
Figure 3: (a) Lead of SleuthVLM over the best omni model (closed-source for Opus 5.5, open-source for Qwen3.8-27B), with benchmarks ordered by average video length. (b) Turns and images viewed per question by SleuthVLM (Qwen3.8-27B, medium). (c) Accuracy of SleuthVLM (Qwen3.8-27B, medium) as each part of the audio input is added.
Type
Speech
Music
Sound
Non-speech
Attribute Perception
145 (63)
3 (67)
6 (67)
5.8%
Counting
120 (60)
11 (55)
12 (42)
16.1%
Spatial Inference
122 (66)
5 (60)
2 (50)
5.4%
Event Understanding
115 (68)
7 (43)
5 (80)
9.4%
Logical Reasoning
94 (61)
10 (70)
5 (60)
13.8%
Human-Centric Underst.
87 (60)
2 (50)
14 (43)
15.5%
Table 4: LVOmniBench questions by question type and audio type, with the accuracy (%) of SleuthVLM (Qwen3.8-27B, medium) in parentheses. Non-speech: share of Music and Sound questions in the type.
Figure 4: Four trajectories of SleuthVLM: (a, b) answered from the transcript and the picture, (c) with the audio_caption tool, (d) one question with Claude Opus 5.5, a VLM, and Gemini 3.8 Flash, an omni model, as the reasoning model. Highlighted rows hold the decisive evidence, and the line under a call is the model’s decision; green : transcript, red : picture, blue : audio. Every step of (a–d) is detailed in Section 10 .
Setting
Daily-Omni
LVOmniBench
A0 Full system
80.3
60.2
A1 w/o ASR transcript
62.9 ( − 17.4)
43.6 ( − 16.6)
A2 w/o word-level times
79.0 ( − 1.3)
58.9 ( − 1.3)
Table 5: Ablations of SleuthVLM (Qwen3.8-27B, medium) on a short-video (Daily-Omni, 30–60 s) and a long-video benchmark (LVOmniBench, 10–90 min), accuracy (%) with the change from the full system A0 in parentheses.
Model
Input
AV Align
Comp.
Ctx. Und.
Evt. Seq.
Infer.
Reas.
30 s
60 s
Avg.
SleuthVLM (Opus 5.5)
V+ASR
81.51
86.26
84.97
82.35
86.36
88.00
83.15
85.82
84.38
SleuthVLM (Qwen3.8-27B, medium)
V+ASR+Audio Caption
79.83
85.50
80.31
77.78
84.42
84.57
80.06
82.73
81.29
SleuthVLM (Qwen3.8-27B)
V+ASR
76.89
81.68
78.76
79.08
83.77
85.14
77.28
84.00
80.37
SleuthVLM (Qwen3.8-27B, medium)
V+ASR
77.31
80.92
76.68
80.39
84.42
84.00
78.05
82.91
80.28
video-SALMONN 2+ (72B)
Open, A+V
New A
New A
New A
New A
New A
New A
New A
New A
79.4 †
video-SALMONN-R3 (8B)
Open, A+V
New A
New A
New A
New A
New A
New A
New A
New A
78.7 ¶
Table 6: Full comparison on Daily-Omni (1197 questions) by question type and video duration (accuracy, %). Baselines are the “With Visual and Audio” rows of the Daily-Omni paper; both the Instruct and Thinking versions of Qwen3-Omni are listed. Overall scores only: † from the video-SALMONN 2 paper, § from the TFO paper, ¶ from the video-SALMONN-R3 paper.
Model
Input
Tech.
Culture
Daily
Film & TV
Perf.
Games
Sports
Music
Avg.
SleuthVLM (Opus 5.5)
V+ASR
71.2
72.2
69.9
74.9
72.7
71.2
66.0
64.8
70.1
Gemini 2.5 Pro
Closed, A+V
64.9
66.0
65.8
68.1
69.7
65.7
63.5
61.3
65.1
SleuthVLM (Qwen3.8-27B)
V+ASR
69.6
69.9
66.6
67.8
66.7
65.7
56.5
53.9
64.5
SleuthVLM (Qwen3.8-27B, medium)
V+ASR+Audio Caption
70.2
68.6
61.9
66.0
65.9
63.9
57.9
63.1
64.4
SleuthVLM (Qwen3.8-27B, medium)
V+ASR
69.6
67.6
60.8
67.0
65.9
62.7
59.3
52.2
62.8
Omni-Decision
Agent, A+V
63.7
60.5
53.3
66.8
52.4
56.7
50.0
64.0
58.3
Table 7: Full comparison on WorldSense (3172 questions) by domain (accuracy, %): Tech & Science, Culture & Politics, Daily Life, Film & TV, Performance, Games, Sports, Music. Baselines are the audio-visual models of the WorldSense paper; Omni-Decision, a GPT-5.2 planner over Gemini-3.1-Pro perception, is from its paper. Overall scores only: † from the video-SALMONN 2 paper, § from the TFO paper.
Model
Input
Music
Sound
Speech
(0,1]
(1,5]
(5,10]
(10,30]
Avg.
SleuthVLM (Opus 5.5)
V+ASR
57.14
68.03
71.78
64.91
74.85
67.69
68.70
69.90
Gemini-3.0-Pro
Closed, A+V
52.81
55.17
64.13
62.42
66.18
57.02
59.76
61.80
SleuthVLM (Qwen3.8-27B)
V+ASR
51.65
57.14
62.07
63.16
62.13
57.21
59.16
60.40
SleuthVLM (Qwen3.8-27B, medium)
V+ASR+Audio Caption
47.25
56.46
62.07
61.99
61.54
55.02
60.69
59.90
Gemini-2.5-Pro
Closed, A+V
38.46
57.72
61.66
57.83
64.43
55.02
55.94
58.90
SleuthVLM (Qwen3.8-27B, medium)
V+ASR
45.05
55.10
60.50
59.06
59.17
58.95
56.11
58.30
Table 8: Full comparison on OmniVideoBench (1000 questions) by audio type and video duration in minutes (accuracy, %). Baselines are the “With Visual and Audio” rows of the OmniVideoBench paper. Duration buckets follow its intervals, with the five videos just over 30 min in (10,30]. ¶ Overall score only, from the video-SALMONN-R3 paper.
Model
Input
Low
Medium
High
Underst.
Percept.
Infer.
Logical
Avg.
SleuthVLM (Opus 5.5)
V+ASR
82.5
76.0
66.4
80.4
74.4
70.6
82.6
75.5
Gemini-3.0-Pro
Closed, A+V
79.3
68.1
45.0
73.3
60.1
65.4
67.5
65.8
SleuthVLM (Qwen3.8-27B, medium)
V+ASR+Audio Caption
70.4
60.3
54.4
67.4
61.7
58.5
60.6
61.9
SleuthVLM (Qwen3.8-27B)
V+ASR
68.8
62.6
51.7
66.5
59.8
60.6
61.5
61.7
SleuthVLM (Qwen3.8-27B, medium)
V+ASR
69.4
61.7
46.3
62.6
58.5
59.9
61.5
60.2
Code-X (Qwen3.6-27B) ∗
Agent, V+ASR
75.0
53.8
48.0
New A
New A
New A
New A
60.0
Table 9: Full comparison on LVOmniBench (1014 questions) by difficulty and ability (accuracy, %). Baselines are the A+V rows of the LVOmniBench paper. We group our question types into its four abilities: Perception = Counting + Attribute Perception + Music Perception; Understanding = Human-Centric + Event Understanding; Inference = Sound + Spatial + Temporal Inference; Logical = Logical Reasoning. ¶ Overall score only, from the video-SALMONN-R3 paper. ∗ Sandboxed coding agents and their trained Code-X model, from their paper, tested on the first 100 questions only.
Model
Input
Attr.
Count.
Spat.
Event
Logic.
Human
Temp.
Music
Sound
Overall
(154)
(143)
(129)
(127)
(109)
(103)
(79)
(89)
(81)
(1014)
SleuthVLM (Opus 5.5)
V+ASR
80.5
71.3
76.0
81.9
82.6
78.6
74.7
68.5
58.0
75.5
SleuthVLM (Qwen3.8-Omni-Flash)
V+ASR+Audio
73.4
65.7
74.4
78.0
67.9
68.9
72.2
65.2
66.7
70.6
SleuthVLM (Gemini 3.8 Flash)
V+ASR+Audio
80.5
69.2
70.5
78.0
77.1
76.7
73.4
73.0
67.9
74.4
Qwen3.8-Omni-Flash
A+V
New A
New A
New A
New A
New A
New A
New A
New A
New A
63.3
Gemini 3.8 Flash
A+V
New A
New A
New A
New A
New A
New A
New A
New A
New A
70.7
Table 10: The same harness with other reasoning models on LVOmniBench, by question type (accuracy, %); Table 3 groups the same runs by difficulty and ability. V+ASR+Audio: an omni reasoning model that can also call listen_audio . A+V rows: closed-source omni models without our harness, with their reported overall accuracy. Bold : best; underline : second best. Question counts in parentheses.
Figure 5: Step-by-step trajectories answered from the timed transcript. (a) A count at the moment a phrase is said in a 38-minute video. (b) A sound that never appears on screen and is named by the speaker.
Figure 6: Step-by-step trajectories answered from the timed transcript. (a) A gap in the speech that times the music. (b) Lyrics that carry the emotion of a song.
Figure 7: Step-by-step trajectories answered from the timed transcript. (a) A gap that rules out the spoken options. (b) Word times that reveal one sustained voice.
Figure 8: Step-by-step trajectories with the audio_caption tool. (a) How the music changes. (b) Which instrument drives the rhythm, settled by two differently worded calls.
Figure 9: Step-by-step trajectories with the audio_caption tool. (a) A non-speech sound that explains the plot. (b) The start of the first music in a 41-minute video, located by the transcript gaps and confirmed by the tool.
Figure 10: The same question in the same harness with two reasoning models. (a) SleuthVLM (Opus 5.5), a VLM, binds the comment to the sentence spoken at that moment in five turns. (b) SleuthVLM (Gemini 3.8 Flash), an omni model, finds the same evidence and loses it over 27 turns.
Opus 5.5
Gemini 3.8 Flash
Answer
A (correct)
B (wrong)
Turns
5
27
Tool calls (without answer)
6
26
Images viewed
2
6
Audio listens
0
2
Tokens
27,378
589,458
Table 11: The question of Figure 10 in the same harness: the VLM answers correctly with a fraction of the turns, tokens and time of the omni model.
Model
Non-Lin
Level 1
Level 2
Level 3
Cons.
Coh.
Acc.
Human Expert
90.7
94.8
91.1
87.9
91.7
88.9
94.9
SleuthVLM (Opus 5.5)
63.5
79.0
66.2
52.6
64.5
61.7
77.8
Gemini 3.8 Flash (Qwen-code) †
New A
New A
New A
New A
New A
New A
72.7
Qwen3.8-Omni-Flash (Qwen-code) †
New A
New A
New A
New A
New A
New A
71.3
Gemini 3.8 Flash †
New A
New A
New A
New A
New A
New A
71.0
Gemini-3-Pro
49.4
64.0
50.0
40.6
50.8
47.0
66.1
Table 12: Video-MME-v2 leaderboard with subtitles (w. sub; for omni models, with audio input), sorted by average accuracy. SleuthVLM reads the official subtitles as its timed transcript. Non-Lin: grouped non-linear score; Level 1–3: the three levels of the benchmark; Cons.: capability consistency; Coh.: reasoning coherence; Acc.: average accuracy. Models evaluated with several frame counts are listed with their best setting. † : overall accuracy reported in the Qwen3.8-Omni technical report; Qwen-code: the model run as an agent in the Qwen Code harness. Bold : best; underline : second best; New A: not reported.
audio caption
Benchmark
Questions
w/o
w/
Δ
Daily-Omni
34
64.7
70.6
+5.9
WorldSense
193
43.5
62.2
+18.7
OmniVideoBench
85
51.8
51.8
0.0
LVOmniBench
136
50.7
55.1
+4.4
Total
448
48.9
58.7
+9.8
Table 13: The audio_caption tool raises SleuthVLM (Qwen3.8-27B, medium) by 9.8 points on the 448 music questions (accuracy, %).