Organizations: Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · Singapore Management University
Existing deep research agents are designed primarily for text- and image-based web sources, while video reasoning systems typically assume that relevant videos are provided in advance. We study open-web video research, where an agent must autonomously discover relevant videos, navigate their temporal content, and ground answers in visual evidence. Training such agents at scale is challenging as live video interaction is slow and unreliable, whereas fixed local simulation can induce retrieval-specific shortcuts that fail to transfer to the open web. We introduce VideoResearchAgent, a scalable training framework to address these challenges. First, we introduce controllable task synthesis pipeline to synthesize multi-hop research tasks from timestamped visual evidence while filtering text-only shortcuts. Second, we build a field-aligned local video simulator that preserves deployment-facing search and watch interactions while accelerating video search by a factor of 34.5-64.6. Third, we introduce Retrieval-Domain-Randomized GRPO (RDR-GRPO), which diversifies candidate rankings, distractors, metadata, and result structure during training to reduce overfitting to simulated retrieval. On Video-BrowseComp, the VideoResearchAgent trained using Qwen3.5-4B achieves 40.48% accuracy, comparable to Gemini-3-Flash-Preview, while reducing cumulative API-token consumption by 74.9% relative to the untrained model. Together, these results establish an accurate and efficient training recipe for open-web video research.
Figures & tables
Figure 1: Overview of VideoResearchAgent. A VLM iteratively searches, watches, aggregates, and cross-verifies video evidence.
System
Evidence
Video Research
Data & Learning
Web Text
Visual
Web Video Search
Temporal Navigation
Cross-Video Evidence
Task Synthesis
Open Train Data
Policy Training
Text-based deep research
WebSailor-V2 ( Li et al., 2025b )
✓
✗
✗
✗
✗
✓
✗
✓
Tongyi DeepResearch ( Team et al., 2025b )
✓
✗
✗
✗
✗
✓
✗
✓
DR Tulu ( Shao et al., 2025 )
✓
✗
✗
✗
✗
✗
✓
✓
MiroThinker ( Team et al., 2025a )
✓
✗
✗
✗
✗
✓
✗
✓
Table 1: Comparison of representative deep-research and video agents. Task synthesis denotes a dedicated pipeline for constructing training tasks or tool-use trajectories. Under Open Train Data, ✓ indicates a publicly downloadable training set, △ an announced but not yet accessible release, and ✗ no such release. VideoResearchAgent is the only compared system that combines open-web video search, temporal navigation, cross-video evidence, task synthesis, and policy training.
Figure 2: Pipeline for synthesizing video-grounded research tasks. Diverse video queries yield visual entities and multi-hop evidence chains, which are turned into questions with concise answers. Task-level verification reduces text-only shortcuts and increases reliance on video-grounded reasoning.
Model
Difficulty
Genre Accuracy
OA
L1
L2
L3
Doc
Edu
Film
Game
Music
Sport
TV
Vid
Tool-Free Models
Qwen3-VL-8B-Think
12.00
0.00
0.00
10.00
0.00
13.11
0.00
20.00
0.00
5.26
16.67
7.14
Qwen3-VL-235B-Ins
22.40
0.00
0.00
0.00
12.00
19.67
13.64
0.00
2.22
21.05
27.78
13.33
GLM-4.6V
16.80
3.23
0.00
0.00
12.00
11.48
13.64
10.00
2.22
26.32
16.67
10.95
GPT-4o
28.00
3.23
0.00
10.00
16.00
26.23
13.64
0.00
4.44
31.58
27.78
17.62
Table 2: Video-BrowseComp results across difficulty levels and video genres. We report difficulty-level accuracy, genre accuracy (%), and overall accuracy (OA). Tool-Free and Search Models results are reported from Liang et al. (2026) ; (S) denotes integrated web-search access. For the “VideoResearchAgent with Small-size Models” group, the highest value in each column is shown in bold with green shading, and the second highest is underlined with amber shading.
Operation
Live Backend / SR
Local Simulator / SR
Approx. Gap
Video search
12.93 s / 73.3%
0.20–0.375 s / 100%
34.5 – 64.6×
Watch video (cold)
41.36 s / 80.0%
∼ 7.85 s / 100%
5.3×
Watch video (fully cached)
0.235 s / 100%
∼ 0.35 s / 100%
Comparable
Table 3: Mean successful-call latency of the live backend and local simulator. SR means success rate.
Figure 4: Training and test performance for fixed-retrieval GRPO and RDR-GRPO. The left panel shows raw on-policy training scores at each step. The center and right panels show accuracy on test-set data for fixed-retrieval GRPO and RDR-GRPO, with dashed lines and shaded areas indicating linear fits and 95% confidence intervals.
Figure 5: Behavioral statistics for Qwen3.5-4B at the Base, SFT, and SFT+RDR-GRPO stages. Panel (a) reports the composition of tool calls and annotates the total calls per query. Panels (b)–(d) report mean agent turns, cumulative API token usage, and final serialized context token length. Error bars are 95% task-bootstrap confidence intervals.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Base policy
Qwen3.5-4B
Tuning scope
Full-parameter SFT; language model trainable; vision encoder and aligner frozen
Maximum length 196,608; padding-free batches; packing disabled; over-length examples deleted
Appendix
Table 4: Supervised fine-tuning configuration for Qwen3.5-4B.
Tool
Core arguments
Policy-relevant observation
web_search
query
Ranked web titles, URLs, and snippets for entity discovery.
search_youtube
query, top- k
Ranked video titles, URLs, descriptions, durations, and thumbnails.
watch_video
URL, mode, temporal range
Transcript context (if available) and either sparse overview frames or dense frames from a selected window.
visual_grounding
URL, object prompt, timestamps
Localized objects, confidence values, and annotated frames.
Appendix
Table 5: Policy-facing tools. watch_video and visual_grounding return image tokens in addition to compact textual metadata.
Figure A1: Composition of the 3,534 synthesized video research tasks.
Tier
Sampling probability
Purpose
easy
55%
Preserves low-perturbation cases and a stable training signal.
medium
35%
Applies moderate rank, distractor, metadata, and structural perturbations.
hard
10%
Applies stronger perturbations and occasional gold omission or empty results.
Appendix
Table 6: Difficulty tiers and their sampling probabilities.
Perturbation type
easy
medium
hard
Rank intervention
gold 100% keep at top
gold 50% keep, 35% sink to middle, 13% sink to bottom, 2% drop
gold 25% keep, 30% sink to middle, 33% sink to bottom, 12% drop
Hard negative injection
0
2–4
4–8
Title jitter probability
0%
40%
80%
Snippet jitter probability
0%
40%
80%
Empty result probability
0%
1%
3%
Display count variation
none
− 1 to +2
− 2 to +3
Appendix
Table 7: RDR perturbation parameters for each difficulty tier.
Base
SFT
RDR-GRPO
Accuracy (%)
30.0
36.7
40.5
Trajectories hitting the 50-turn cap (%)
60.0
15.7
2.9
Trajectories with a degenerate loop (%)
50.5
0.0
0.0
Tool calls per task
35.8
17.7
13.3
Distinct tool calls per task
17.9
17.2
13.1
Exact-repeat tool calls (%)
50.1
2.8
2.0
Appendix
Table 8: Interaction statistics of the three Qwen3.5-4B stages on the 210 tasks. Abstention: the final answer states that the answer cannot be determined.
50-turn cap (%)
After empty search (%)
Policy
Acc. (%)
All
w/ loop
Calls/task
Reform.
Minor edit
Repeat
Untrained Qwen3.5, increasing scale
Qwen3.5-4B
30.0
60.0
49.5
35.8
38
24
35
Qwen3.5-9B
32.9
35.7
24.3
26.0
54
9
34
Qwen3.5-27B
34.8
23.6
12.1
21.6
68
10
18
Qwen3.5-Plus
43.3
26.2
0.0
25.0
92
2
0
Appendix
Table 9: Interaction control of the tested model policies. 50-turn cap: trajectories that hit the cap, and those among them that contain a degenerate loop. After an empty search result: the next tool call after a search that returns no results; in the remaining 2–8% of cases the policy switches to a video tool or answers.
Figure A2: An illustrative reasoning trajectory of VideoResearchAgent . The trajectory details the agent’s internal thought process, tool invocations, and observations, ultimately culminating in the correct final answer. Text highlighted in red denotes pivotal steps along the agent’s reasoning path, illustrating how it progressively gathers evidence to reach the final answer.
We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding coupled with open-web exploration. Preliminary evaluations reveal two critical bottlenecks in current models: (1) modality bias, where agents bypass visual tools in favor of textual search, and (2) parametric knowledge leakage, where models rely on internal memory rather than genuine tool-augmented execution. To address these challenges, we propose Video-DR, featuring a decoupled perception-exploration pipeline with stage-wise tool unlocking that compels exhaustive cross-frame visual grounding prior to web retrieval. Our framework adopts a two-stage training recipe: supervised fine-tuning followed by Group Relative Policy Optimization (GRPO), enabling autonomous exploration that breaks the imitation-learning ceiling. Furthermore, we curate Video-DR-Bench, a human-AI collaborative benchmark comprising 200 complex, multi-hop VQA instances. Empirical results demonstrate that our Video-DeepResearch-35B-A3B establishes a new state-of-the-art of 64.0% average accuracy, surpassing proprietary Claude-4.5-Sonnet (59.0%) by 5.0 points and significantly outperforming GPT-5 (52.5%) and Gemini 2.5 Pro (57.5%). The 30B-A3B variant achieves 59.3%, competitive with Claude-4.5-Sonnet and demonstrating the effectiveness of our training paradigm even at compact scale. Code: https://github.com/Osilly/Vision-DeepResearch.
Open-world video understanding often requires a model to locate sparse visual evidence and acquire external knowledge that is absent from the video and its parametric memory. While Thinking-with-Videos enables active temporal perception and Deep Research supports multi-step information seeking, the two capabilities are typically developed in isolation. We introduce VideoRover, a unified Video Deep Research framework that iteratively coordinates video cropping, multimodal search, and webpage browsing. Given a video-question pair, VideoRover uses each tool result to select the next action, so localized video clips guide external retrieval and retrieved evidence triggers further video inspection and verification. To develop this capability, we construct an automated data curation pipeline, producing 26K verified SFT trajectories and 3K challenging RL instances. We also introduce VideoRover-Bench, a benchmark stratified by video duration and research difficulty. Experiments on VideoDR and VideoRover-Bench show that our VideoRover-8B-RL achieves performance comparable to proprietary models in the direct-answer setting without tool use while outperforming larger open-source models equipped with the same tool suite. Ablation studies and training dynamics further validate the complementary roles of active video grounding, external retrieval, and long-horizon reinforcement learning.
Wenqi Liu, Shijie Ma, Yunxiao Wang +21
Shandong University · Institute of Automation, Chinese Academy of Sciences · Beihang University +4
Video understanding is moving beyond closed-context perception toward open-world evidence exploration, a paradigm formalized as Video Deep Research (VDR). However, existing multimodal search agents primarily target static images, and the current VDR benchmark relies on text-centric retrieval that discards crucial visual information. To address these limitations, we propose VideoSearcher, a closed-loop agentic framework that empowers Vision-Language Models with multi-tool reasoning for VDR. VideoSearcher unifies temporal localization, spatial focusing, and multimodal search within a single reasoning trajectory, enabling agents to progressively ground visual clues, retrieve relevant evidence, and synthesize answers. To optimize knowledge-intensive reasoning trajectories, we propose Bi-branch Sequence Policy Optimization (BiSPO), a reinforcement learning algorithm that decouples tool-invocation optimization from answer-accuracy optimization. This design provides stable learning signals for both evidence-grounded reasoning and purposeful tool use. Furthermore, we construct VideoSearch-QA, the first benchmark designed to evaluate open-world video information grounding and multimodal search-based reasoning. Extensive experiments demonstrate that VideoSearcher significantly outperforms prior open-source agentic baselines across various search-oriented and multimodal understanding benchmarks.
Zhenkun Gao, Yicheng Bao, Jinlong Peng +13
East China Normal University · Tencent Youtu Lab · HFIPS, Chinese Academy of Sciences +3