While LVLMs rapidly improve, long-video question answering still remains challenging: relevant evidence is sparse, and question-relevant context often fails to provide cues that discriminate the correct answer from plausible alternatives. Diagnostic analysis on a manually annotated subset of MMR-V shows that prior agentic systems substantially improve cue retrieval over direct VLM inference yet fail to achieve a corresponding gain in answer accuracy, indicating that the bottleneck lies in option-discriminative evidence rather than topical relevance alone. We propose PACE (Progressive Acquisition of Critical Evidence), a factor-guided framework for long-video evidence acquisition. PACE proceeds in two stages: it first indexes clip-level descriptions guided by question-derived factors without observing the candidate answers; it then uses the candidate answers to derive contrastive cues and queries the index for verification. On MMR-V with the open-source Qwen3-VL backbone, PACE achieves 42.6% accuracy, outperforming direct inference and prior agentic baselines including Deep Video Discovery (DVD). On the same diagnostic subset, PACE recovers 66.9% of the annotated cues, providing empirical evidence that its gains are associated with improved evidence recovery rather than stronger answer-side priors alone. Consistent gains over DVD on LVBench, Video-MME, EgoSchema, and LongVideoBench suggest that option-aware evidence acquisition transfers beyond MMR-V. Code is available at https://github.com/HKUST-KnowComp/PACE.
Figures & tables
Figure 1: Illustration of the limitation in previous works versus our approach. Left: Previous methods suffer from information loss due to task-agnostic captioning. Right: Our method generates focused, contextual captions to support effective evidence discovery.
Figure 2: Overview of PACE. The first stage builds an evidence database from clip-level descriptions guided by question-derived factors, without observing the candidate answers. The second stage derives contrastive cues from the candidate answers and queries the database to verify them.
Tasks
Video Categories
Model
Overall
Implicit
Explicit
Art
Life
TV
Film
Ani.
Phi.
Proprietary models
GPT-4o-mini
34.8
38.0
26.3
29.5
25.4
29.6
33.0
48.7
18.6
Claude-3.5-Sonnet
43.3
45.0
38.9
33.8
31.1
41.3
41.3
55.8
44.4
GPT-4o
44.0
46.6
37.6
38.1
37.3
34.9
41.0
61.6
32.6
GPT-4.1
46.6
49.1
40.3
43.2
35.6
43.9
46.5
57.1
34.9
Table 1: Evaluation results (%) on MMR-V. Bold values indicate the best performance among Baseline and Agentic Framework. The backbone for VideoTree, DVD and PACE is Qwen3-VL-30B-A3B-Thinking.
Figure 3: Needle recall and accuracy on the 100-question diagnostic subset of MMR-V. Avg. Recall is the overall needle recall rate, Avg. Recall (Correct) is the recall rate restricted to correctly answered questions, and Accuracy is the answer accuracy on the subset.
Method
LVBench
LongVideoBench
Video-MME
EgoSchema
Overall
Val (Long)
Long (w/o sub)
Val
Qwen3-VL-30B-A3B-Thinking
59.2
46.1
59.4
66.0
Deep Video Discovery
62.3
54.6
57.2
67.9
PACE
62.6
55.1
58.9
68.1
Table 2: Controlled transfer results on general long-video benchmarks using the Qwen3-VL-30B backbone.
Backbone Model
DVD
PACE
Delta
Qwen2.5-VL-7B
34.2
37.6
+3.4
Qwen3-VL-8B
34.0
36.1
+2.1
Qwen3-VL-30B-A3B-Thinking
39.5
42.6
+3.1
Table 3: Performance comparison (%) between DVD and PACE frameworks on the MMR-V benchmark using different backbone configurations. Delta shows the absolute improvement of PACE over DVD. For Qwen2.5-VL-7B, the reasoning agent is Qwen3-8B.
Model/Agent
Accuracy
PACE
42.6
- w/o question-conditioned indexing
41.8~_{{\color[rgb]{0.5,0.5,0.5}-0.8\downarrow}}
- w/o option-aware verification
41.5~_{{\color[rgb]{0.5,0.5,0.5}-1.1\downarrow}}
Table 4: Ablation study of PACE on MMR-V. We replace question-conditioned captions with standard captions or remove option-aware verification.
Figure 4: Impact of the number of coarse query-conditioned factors on MMR-V accuracy.
Figure 5: Distribution of inference trajectory lengths of DVD and our proposed PACE framework.
Tool
w/ Tool (%)
w/o Tool (%)
Drop
Samples
EvidenceRetrieve
38.4
30.7
− 7.7
65
ClipSearch
46.4
41.0
− 5.4
56
Table 5: Conditional ablation study evaluating the performance on the specific subsets of samples where low-usage tools were originally invoked. The notable performance drops indicate the necessity of these tools when explicitly required.
Stage
Avg. Prompt Token
Avg. Completion Token
Captioning (per clip)
17,584.6
1,109.5
Reasoning (per query)
68,164.0
10,262.0
Table 6: Average prompt and completion token counts during the captioning and reasoning stages.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Method
LVBench
LongVideoBench
Video-MME
EgoSchema
Overall
Val (Long)
Long (w/o sub)
Val
Proprietary foundation models
Gemini-2.0-Flash
48.3
45.7
63.0
71.2
GPT-4o
48.9
60.9
65.3
70.4
OpenAI o3
57.1
60.6
64.7
63.2
Seed-1.8
73.0
77.4
87.8
–
Appendix
Table 7: Full comparison on general long-video understanding benchmarks.
Clip Duration ( T )
Accuracy (%)
5s
37.2
10s
42.6
20s
38.1
Appendix
Table 9: Ablation study on the video clip duration ( T ) used during contextual environment construction on the MMR-V benchmark.
Department of Earth System Science, Tsinghua University · 5The Hong Kong University of Science and Technology (Guangzhou) · School of Artificial Intelligence, Sun Yat-Sen University +2