While multimodal large language models (MLLMs) have advanced video understanding, they remain highly prone to hallucinations in dynamic scenes. We argue this stems from a failure in spatio-temporal monitoring, the ability to persistently track object identities, states, and relations over time. Existing benchmarks obscure this deficit by relying on single final-answer evaluations for queries that can often be resolved via local visual cues or statistical priors. To rigorously diagnose this, we introduce STRAND, a benchmark of human-verified object-centric facts that evaluates intermediate reasoning by decomposing queries into sub-questions, distinguishing genuine temporal understanding from coincidental correctness. Crucially, we score models with Faithful Accuracy, an unconditional joint metric that credits a prediction only when the target answer and every prerequisite sub-question are correct, so that a model cannot inflate its score by being selectively consistent on the small subset of targets it happens to answer correctly. To address failure modes exposed by STRAND, we further propose an object-centric framework that explicitly constructs and reasons over structured object trajectories via chunk-wise state extraction and temporal aggregation. Extensive experiments, including backbone-, frame-, call-, and token-matched comparisons against both end-to-end MLLMs and modular video harnesses, demonstrate that our object-centric framework significantly reduces hallucinated answers and improves spatio-temporal reasoning consistency over state-of-the-art MLLMs. The code, model, and data have been made available at nguyentthong.github.io/strand.
Figures & tables
Figure 1: Existing benchmarks and MLLM architectures versus ours. Top : a single answer records only that a model is wrong, whereas sub-questions name the facts it rests on. Bottom : we answer from object trajectories rather than raw frames alone.
Figure 2: Overview of the STRAND construction pipeline. Annotators anchor timestamps, register entities, extract typed facts, and link identities across time. Target questions and sub-questions are composed from that record and verified.
Figure 3: Overview of our framework. Stages 1 to 5 run once per video. Stages 6 and 7 run once per question.
Method
STRAND
Afaith
Atarget
Asub
Acons
Ours
Ours (Gemini-3-Flash)
59.4
77.1
76.3
81.3
Ours (Qwen3-VL-235B)
54.2
74.0
72.6
78.1
Proprietary end-to-end MLLMs
Gemini-3.1-Pro
38.9
61.4
64.9
69.2
Table 1: STRAND results (%). Best and second-best per column. Four further open-weight models are in Appendix H . Afaith is the primary metric and Acons is reported for reference only (Section 4.3 ).
Method
VideoHallucer
Video-MME
EgoSchema
Ours (Gemini-3-Flash)
76.8
76.8
78.4
Ours (Qwen3-VL-235B)
71.4
73.6
76.4
Gemini-3.1-Pro
63.1
77.3
76.8
GPT-5
67.5
76.2
75.4
Qwen3-VL-235B
59.8
72.6
72.1
LLaVA-OV
48.5
66.3
60.1
Table 2: Condensed external-benchmark results (%). Full sub-scores in Appendix D.1 .
Ablation Focus
Pipeline
Chunk (s)
Frames/chunk
Frames@ Gη
Atarget (%)
Reference Base Config
Base Configuration
Ours (Gemini-3-Flash)
15
60
64
77.1
Base Configuration
Ours (Qwen3-VL-235B)
15
60
64
74.0
Query-based retrieval
No filter (full timeline)
Gemini / Qwen
15
60
64
77.0 / 64.6
Random filter (top- k )
Gemini / Qwen
15
60
64
76.5 / 68.0
Table 3: Ablation studies on STRAND Target Accuracy (%).
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Model
State Change
Identity Tracking
Action Sequ.
Multi-Obj Interact.
Target count ( nk )
200
277
300
200
Ours (Gemini-3-Flash)
70.00
49.82
52.00
73.00
Gemini-3.1-Pro
50.00
31.05
32.67
48.00
Qwen3.5-27B
58.00
37.91
38.00
55.00
Appendix
Table 4: Faithful Accuracy ( Afaith ) breakdown across STRAND question categories (%). The four categories partition the 977 target questions.
Model
Obj. Hallucination
State Misattribution
Temporal Order Error
Ours (Gemini-3-Flash)
12.4
34.1
53.5
Gemini-3.1-Pro
38.2
36.5
25.3
Qwen3.5-27B
33.7
38.4
27.9
Appendix
Table 5: Sub-question error types (%), as a share of each model’s incorrect sub-questions. Rows sum to 100.
Method
VideoHallucer
Video-MME (w/o subs)
EgoSchema
Basic
Halluc.
Overall
Short
Med.
Long
Over.
Acc.
Ours
Ours (Gemini-3-Flash)
83.5
79.9
76.8
85.8
78.3
66.2
76.8
78.4
Ours (Qwen3-VL-235B)
80.6
75.1
71.4
82.2
76.0
62.7
73.6
76.4
Frontier end-to-end MLLMs
Gemini-3.1-Pro †
85.2
68.4
63.1
84.1
76.5
71.3
77.3
76.8
Appendix
Table 6: Results on external benchmarks. Basic and Hallucinated sub-columns are unmarked because Video-LLaVA’s yes-bias inflates Basic. VideoHallucer Overall is the paired both-correct score, so Overall≤min(Basic,Halluc.) . Video-MME Overall is the mean of the three subset columns.
Ablation Focus
Pipeline
Chunk (s)
Frames/chunk
Frames@ Gη
Atarget (%)
Reference Base Config
Base Configuration
Ours (Gemini-3-Flash)
15
60
64
77.1
Base Configuration
Ours (Qwen3-VL-235B)
15
60
64
74.0
Cluster A: Visual Budget at the Answerer
Frames@ Gη = 32
Gemini / Qwen
15
60
32
76.3 / 66.4
Frames@ Gη = 16
Gemini / Qwen
15
60
16
76.5 / 65.5
Appendix
Table 7: Full ablation studies on STRAND Target Accuracy (%).
Constraint
End-to-end baseline
Ours
Δ
Equal total frame budget (64 frames)
41.5
54.2
+12.7
Equal number of model calls (single pass)
39.8
49.3
+9.5
Equal token budget
45.1
52.8
+7.7
Appendix
Table 8: Budget-matched comparisons on Faithful Accuracy Afaith (%). All configurations use the Qwen3-VL backbone, so only the representation changes within each control.
Configuration
Atarget (%)
Afaith (%)
Base: Δtmax=15 s, τconf=0.75 , bipartite
74.0
54.2
Temporal constraint Δtmax
5 s (strict)
71.2
50.4
30 s (relaxed)
72.8
51.1
Confidence threshold τconf
0.90 (high precision)
70.5
48.9
Appendix
Table 9: Ablation of identity-linking thresholds and conflict resolution on STRAND. The base row reproduces Ours (Qwen3-VL-235B) in Table 1 .
Model
Full video
Restricted input
Atarget
Afaith
Atarget
Afaith
Single centre frame
Gemini-3.1-Pro
61.4
38.9
31.4
14.1
Gemini-3-Flash
48.7
28.7
29.8
12.5
GPT-5
32.5
20.0
28.2
10.2
Qwen3.5-27B
68.4
45.5
32.7
13.8
Appendix
Table 10: Single-frame solvability and shortcut audit. Full-video columns reproduce Table 1 . The last two rows receive no visual input at all rather than a single frame.
System
Latency (s)
Total cost per video ( Q=39.7 )
Pre-proc.
Per-query
Calls
Input tokens
Output tokens
Ours (Gemini-3-Flash) ∗
14.2
3.2
91.6
962,400
18,100
Ours (Qwen3-VL-235B)
18.4
2.9
91.6
958,157
17,450
Gemini-3.1-Pro ∗
0.0
3.5
39.7
654,415
1,985
Qwen3.5-27B
0.0
2.8
39.7
654,415
1,985
Qwen3-VL-32B Instruct
0.0
2.9
39.7
654,415
1,985
Appendix
Table 11: Efficiency and cost per video ( Q=39.7 queries). Measurement protocol in Appendix D.7 .
Figure 4: Qualitative analysis of target and sub-question behavior.
Figure 5: Example of the STRAND dataset. To rigorously evaluate model faithfulness, STRAND requires models to correctly answer all underlying supporting sub-questions in addition to the complex target question. This ensures the model accurately tracks temporal action sequences, from passing ( t0 ) to scoring ( t2 ), and grounds specific visual attributes (e.g., “yellow shirt”, “number 10”) rather than merely guessing the final answer.
Figure 6: STRAND evaluation on multi-step object interactions. Building on our faithfulness criteria, models must resolve all supporting sub-questions to demonstrate true comprehension. In this scenario, the model must accurately follow a complex sequence, from the initial display of all artifacts ( t0 ) to cutting machine 1 ( t3 ), while distinguishing specific visual attributes (“number 1”, “number 2”, “cake”) to prevent shortcut learning.
Figure 7: STRAND evaluation of temporal ordering. This example further illustrates how mandatory sub-questions enforce reasoning faithfulness. To succeed, the model must track the precise sequence of events, from placing ingredients ( t0 ) to smashing both items ( t3 ), and ground specific visual objects (“strawberries”, “cookies”). This verifies that the model understands the correct temporal ordering of interactions rather than relying on educated guesses.
Figure 8: STRAND evaluation of actor disambiguation over time. This scenario highlights how STRAND tests complex temporal tracking across multiple subjects. To accurately answer the target question, the model must resolve the chronological sequence of events, from the first student raising his hand ( t0 ) and the second student raising his hand ( t1 ) to the eventual celebration ( t3 ). Mandatory supporting sub-questions force the model to explicitly differentiate between the two actors (e.g., verifying that the second student celebrated, while the first did not) to confirm precise spatiotemporal grounding rather than relying on spurious correlations.
Temporal video understanding studies how models track actions, events, and changes over time to answer questions, locate moments, and predict what happens next.