Autonomous agents are shifting information seeking from passive retrieval toward proactive, open-ended research, yet this progress has largely bypassed video, one of the web's richest sources of evidence. We formalize Agentic Video Browsing, in which an agent must query video search engines, decide which returned videos to watch, and answer questions grounded in their visual content. To study this setting reproducibly, we introduce Video-BrowseComp, a benchmark and infrastructure suite with 660 human-annotated questions at three difficulty levels, paired online and offline evaluation tracks, and a fixed video search stack of roughly 12,000 videos spanning over 2,000 hours for evaluation and training-data construction. Our analysis reveals that the main bottleneck lies before video understanding: agents often fail to find the answer-bearing video at all. This bottleneck stems from two coupled challenges. Search engines index textual metadata such as titles and tags, whereas questions often target implicit visual or cross-source evidence absent from these surface forms; and exhaustively inspecting the many returned videos is prohibitively expensive. We propose Video-Browser, an evidence-surfacing agent that improves both sides of search. A Translation Memory helps the Planner phrase queries by analogy to past question-to-query transformations that successfully surfaced their target videos, and a Watcher performs sparse visual screening over returned candidates, retaining only videos that plausibly contain answer-bearing evidence for later dense verification. On the offline track, Video-Browser raises overall accuracy from 11.6% to 27.3% with Qwen3-VL-8B and from 39.8% to 45.1% with Gemini-3.1-Pro over the corresponding ReAct search agents, showing that better query formulation and post-search candidate selection improve end-to-end browsing performance.
Figures & tables
Figure 1: Statistics and example from Video-BrowseComp. Left: category distribution of evaluation set. Right: an example showing the question, seed video frames, answer, and evidence chain.
Search Method
Query form
R@1
R@5
R@10
R@20
R@50
R@100
BM25
raw question
3.1
6.0
7.2
9.9
11.8
15.0
BM25
+ Qwen3-VL-8B
6.9
13.5
15.5
20.4
25.7
29.6
BM25
+ GPT-5.4
10.7
20.7
23.9
28.0
35.7
42.4
BM25
+ Gemini-3.1-Pro
15.0
24.5
29.9
33.5
39.6
44.9
BGE-M3
raw question
9.3
18.8
23.2
29.0
39.2
49.0
BGE-M3
+ Qwen3-VL-8B
8.6
16.1
19.2
25.7
32.6
39.6
Table 1: Search recall across three backbones and four query forms. Bold marks best performance.
VLM
L1
L2
L3
All
Qwen3-VL-8B
64.4
59.8
57.4
61.6
GPT-5.4
80.5
70.7
77.0
76.0
Gemini-3.1-Pro
87.3
80.4
73.8
82.7
Table 2: Reasoning accuracy when ground-truth videos are provided.
Figure 2: Overview of Video-Browser. The top panel constructs Translation Memory by selecting search-validated queries from training examples, while the bottom panel shows the online browsing loop, where the Planner generates searchable queries, the Watcher selects promising videos, and the Analyst performs dense visual verification.
Model
OA
Recall
Difficulty
CE
Genre Accuracy (%)
(%)
(%)
L1
L2
L3
(%)
Doc
Edu
Film
Game
Music
Sport
TV
Vid
ReAct Search Agent
Qwen3-VL-8B
11.6
23.1
14.6
8.2
11.5
69.7
16.2
7.8
12.0
1.8
21.5
9.0
11.3
13.1
GPT-5.4
27.8
41.8
34.6
20.7
26.2
21.7
35.1
21.6
30.0
8.8
43.1
25.4
32.3
26.2
Gemini-3.1-Pro
39.8
43.6
46.3
35.9
29.5
21.3
48.6
47.1
36.0
26.3
61.5
25.4
40.3
36.1
Video Agent
Table 3: Offline-track performance comparison across agent baselines. Recall denotes cumulative recall over candidates retrieved during the browsing loop. We report overall, difficulty-level, and genre-wise accuracy, with calibration error. Bold marks the best result.
Table 6
Method
Top-1 Hit
Cond. Top-1
Cond. MRR@10
L1 Top-1
L2 Top-1
L3 Top-1
Search Engine
6.0
–
–
7.3
2.2
13.1
LLM Reranker
12.0
57.4
73.5
11.2
7.1
29.5
Watcher
15.6 (+3.6)
75.3 (+17.9)
83.6 (+10.1)
15.6 (+4.4)
9.8 (+2.7)
32.8 (+3.3)
Table 5: Post-search reranking on BM25 Top-10 results using Qwen3-VL-8B. Top-1 Hit is measured over all questions; Cond. Top-1 and Cond. MRR@10 are computed only when the seed video is present in the BM25 Top-10 pool. Bold marks the best result.
Setting
OA
CE
Recall
Watched
Baseline
22.2
66.9
42.9
34.2
+ Memory
25.1
64.3
46.2
34.4
+ Watcher
27.3
56.8
46.2
36.2
w/o Transcript
19.8
65.1
45.3
34.9
w/o Visual
9.8
71.4
46.0
34.9
Table 6: Component and modality ablations.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Benchmark samples.
Genre
Eval Offline (450)
Eval Online (210)
Train (1000)
Eval All
Train All
L1
L2
L3
L1
L2
L3
L1
L2
Documentary
18
9
10
2
8
0
53
45
47
98
Education & Knowledge
19
26
6
23
2
0
68
74
76
142
Film
21
17
12
46
0
15
67
56
111
123
Games & Esports
21
22
14
13
9
0
60
46
79
106
Music & Variety
32
28
5
5
5
0
35
53
75
88
Appendix
Table 7: Per-genre and per-difficulty composition of Video-BrowseComp. The eval set is split into offline (450) and online (210) tracks, the training set (1,000) is auto-generated.
Length bucket
# videos
% of pool
< 1 minute
5,413
43.5%
1–5 minutes
3,195
25.7%
5–10 minutes
1,138
9.1%
10–30 minutes
1,486
11.9%
30–60 minutes
682
5.5%
> 1 hour
541
4.3%
Appendix
Table 8: Video duration distribution over the 12,455 videos in the offline pool.
Stage
Candidates
Filter applied
1. Decoy search (4 rewriters × 450 q.)
19,895
—
2. Prefilter (dedup, drop pool, drop GT)
17,168
remove existing pool / any-question GT
3. Layer-1 text verify (Qwen3-VL-8B)
8,883
keep No with High / Mid relevance
4. Layer-2 visual verify (707 shortlisted)
549
keep No with High / Mid relevance
5. Final selection (top-3 merged into pool)
522
per-question cap and selection ranking
Appendix
Table 9: Hard-negative pipeline funnel. Candidate counts denote the number surviving each filtering stage. Layer-2 evaluates 707 candidates shortlisted from the 8,883 Layer-1 survivors, of which 549 pass visual verification.
Model
OA
Difficulty
CE
Genre Accuracy (%)
(%)
L1
L2
L3
(%)
Doc
Edu
Film
Game
Music
Sport
TV
Vid
Close Source Search Agent
Gemini-2.5-Flash*
20.95
32.80
4.84
0.00
35.98
0.00
36.00
29.51
4.55
10.00
11.11
36.84
16.67
Gemini-2.5-Pro*
23.81
37.60
4.84
0.00
31.45
20.00
32.00
31.15
9.09
10.00
8.89
57.89
16.67
GPT-5.1*
15.24
21.60
6.45
4.35
30.20
0.00
8.00
21.31
18.18
10.00
8.89
21.05
22.22
o4-mini-deep-research
22.86
30.40
12.90
8.70
42.55
10.00
28.00
29.51
27.27
10.00
17.78
21.05
16.67
Appendix
Table 10: Comprehensive results with a heatmap on Genre Accuracy. Bold indicates best performance. * means model has web search ability.
Backbone
Phase
Mean per question
Tokens / Q (k)
Tokens / unit (k)
% of total
Qwen3-VL-8B-Instruct
Browse
18.8 candidates browsed
38.3
2.0
38%
Verify
3.0 videos verified
62.6
20.9
62%
Total
—
100.9
—
100%
Gemini-3.1-Pro
Browse
18.8 candidates browsed
73.1
3.9
15%
Verify
3.0 videos verified
427.6
142.5
85%
Total
—
500.7
—
100%
Appendix
Table 11: Per-question token decomposition for Video-Browser, averaged over 450 offline-track questions. “Browse” aggregates Planner, Searcher, Stage-1 visual selector, and Checker calls across the Tmax=3 loops; “Verify” is the single Analyst call at termination. “Tokens / unit” is per browsed candidate or per verified video. The asymmetry is the intended sparse-vs-dense split of the pipeline.
Existing deep research agents are designed primarily for text- and image-based web sources, while video reasoning systems typically assume that relevant videos are provided in advance. We study open-web video research, where an agent must autonomously discover relevant videos, navigate their temporal content, and ground answers in visual evidence. Training such agents at scale is challenging as live video interaction is slow and unreliable, whereas fixed local simulation can induce retrieval-specific shortcuts that fail to transfer to the open web. We introduce VideoResearchAgent, a scalable training framework to address these challenges. First, we introduce controllable task synthesis pipeline to synthesize multi-hop research tasks from timestamped visual evidence while filtering text-only shortcuts. Second, we build a field-aligned local video simulator that preserves deployment-facing search and watch interactions while accelerating video search by a factor of 34.5-64.6. Third, we introduce Retrieval-Domain-Randomized GRPO (RDR-GRPO), which diversifies candidate rankings, distractors, metadata, and result structure during training to reduce overfitting to simulated retrieval. On Video-BrowseComp, the VideoResearchAgent trained using Qwen3.5-4B achieves 40.48% accuracy, comparable to Gemini-3-Flash-Preview, while reducing cumulative API-token consumption by 74.9% relative to the untrained model. Together, these results establish an accurate and efficient training recipe for open-web video research.
Yuhang Zhou, Fei Li, Yuxi Wu +2
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · Singapore Management University
Comprehensive video understanding is crucial for advancing artificial intelligence toward the intricate dynamics of the physical world. While recent advances in Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in video understanding, existing benchmarks remain confined to simple scene-level queries or global summaries that require only single-step inference. Real-world video understanding involves more challenging tasks that require multi-hop multimodal reasoning, and there is a critical absence of video benchmarks equipped to rigorously evaluate these agentic capabilities. To bridge this gap, we introduce AgentVidBench, a multi-hop video question answering benchmark focused on evaluating the spatial, temporal, and causal reasoning capabilities of MLLM agents. Beyond standard question-answer pairs, AgentVidBench provides step-by-step solution traces to support trajectory evaluation that assesses whether agents explicitly acquire the evidence needed to justify their answers. Experiments with 12 proprietary and open-source MLLMs show that single-turn performance remains limited on AgentVidBench, while integrating these models into state-of-the-art agentic workflows generally improves performance with respect to both accuracy and trajectory scores. We further present a simple yet effective agentic strategy that serves as a competitive baseline on AgentVidBench, establishing our benchmark as a holistic testbed for future research on agentic video understanding. Code and datasets are available at https://github.com/krafton-ai/agentvidbench and https://huggingface.co/datasets/agentvidbench/agentvidbench.
Video understanding is moving beyond closed-context perception toward open-world evidence exploration, a paradigm formalized as Video Deep Research (VDR). However, existing multimodal search agents primarily target static images, and the current VDR benchmark relies on text-centric retrieval that discards crucial visual information. To address these limitations, we propose VideoSearcher, a closed-loop agentic framework that empowers Vision-Language Models with multi-tool reasoning for VDR. VideoSearcher unifies temporal localization, spatial focusing, and multimodal search within a single reasoning trajectory, enabling agents to progressively ground visual clues, retrieve relevant evidence, and synthesize answers. To optimize knowledge-intensive reasoning trajectories, we propose Bi-branch Sequence Policy Optimization (BiSPO), a reinforcement learning algorithm that decouples tool-invocation optimization from answer-accuracy optimization. This design provides stable learning signals for both evidence-grounded reasoning and purposeful tool use. Furthermore, we construct VideoSearch-QA, the first benchmark designed to evaluate open-world video information grounding and multimodal search-based reasoning. Extensive experiments demonstrate that VideoSearcher significantly outperforms prior open-source agentic baselines across various search-oriented and multimodal understanding benchmarks.
Zhenkun Gao, Yicheng Bao, Jinlong Peng +13
East China Normal University · Tencent Youtu Lab · HFIPS, Chinese Academy of Sciences +3