Thinking Beyond Videos: Unifying Video Reasoning and Deep Research for Open-World Video Agents
Authors: Wenqi Liu, Shijie Ma, Yunxiao Wang, Meng Liu, Qile Su, Han Liu, Bohan Hou, Zeyu Wang, +16 more
Organizations: Shandong University · Institute of Automation, Chinese Academy of Sciences · Beihang University · City University of Hong Kong · Nanyang Technological University · Kuaishou Technology · Southern University of Science and Technology
Open-world video understanding often requires a model to locate sparse visual evidence and acquire external knowledge that is absent from the video and its parametric memory. While Thinking-with-Videos enables active temporal perception and Deep Research supports multi-step information seeking, the two capabilities are typically developed in isolation. We introduce VideoRover, a unified Video Deep Research framework that iteratively coordinates video cropping, multimodal search, and webpage browsing. Given a video-question pair, VideoRover uses each tool result to select the next action, so localized video clips guide external retrieval and retrieved evidence triggers further video inspection and verification. To develop this capability, we construct an automated data curation pipeline, producing 26K verified SFT trajectories and 3K challenging RL instances. We also introduce VideoRover-Bench, a benchmark stratified by video duration and research difficulty. Experiments on VideoDR and VideoRover-Bench show that our VideoRover-8B-RL achieves performance comparable to proprietary models in the direct-answer setting without tool use while outperforming larger open-source models equipped with the same tool suite. Ablation studies and training dynamics further validate the complementary roles of active video grounding, external retrieval, and long-horizon reinforcement learning.
Figures & tables
Figure 1: Comparison of Thinking-with-Videos, Deep Research, and VideoRover . VideoRover iteratively coordinates video cropping, multimodal search, and webpage browsing, using each result to guide the next action.
Figure 2: Pipeline for constructing Video Deep Research questions.
Figure 3: Pipeline for synthesizing Video Deep Research trajectories.
Figure 4: Construction of VideoRover-Bench .
Model
VideoDR
VideoRover-Bench
Avg.
Acc.
S-Easy
S-Medium
S-Hard
L-Easy
L-Medium
L-Hard
Proprietary Models (Direct Answer)
GPT-5
57.00
70.00
64.00
36.00
64.00
50.00
44.00
55.00
GPT-5.2
31.00
56.00
52.00
24.00
60.00
30.00
24.00
39.57
GPT-5.4
51.00
56.00
54.00
26.00
24.00
24.00
16.00
35.85
Gemini-2.5-Flash
47.00
54.00
30.00
30.00
54.00
30.00
28.00
39.00
Table 1: Main results on VideoDR and VideoRover-Bench . All values are accuracy (%). S and L denote short and long videos, respectively. Avg. is the unweighted mean over VideoDR and the six VideoRover-Bench subsets. VideoRover variants are shaded in blue. Within the open-source agentic group, the best and second-best results are shown in bold and underlined, respectively.
Model
VideoDR
VideoRover-Bench
Avg.
Acc.
S-Easy
S-Medium
S-Hard
L-Easy
L-Medium
L-Hard
VideoRover -8B-RL
56.00
80.00
64.00
40.00
70.00
52.00
42.00
57.71
− crop_video
45.00
74.00
52.00
34.00
64.00
42.00
36.00
49.57
− image_search
43.00
76.00
54.00
34.00
56.00
44.00
36.00
49.00
− text_search
24.00
40.00
26.00
20.00
30.00
16.00
10.00
23.71
− visit
41.00
76.00
44.00
30.00
68.00
46.00
38.00
49.00
Table 2: Ablation results on VideoDR and VideoRover-Bench . All values are accuracy (%). Each row removes one tool from VideoRover -8B-RL, except web , which jointly removes all external retrieval tools. S and L indicate short and long videos, respectively.
Figure 5: RL training dynamics of VideoRover . Light curves show raw measurements, and dark curves show smoothed trends.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Qualitative case on identifying the year of Rafael Márquez’s third World Cup goal. VideoRover crops the interview and match segments, combines image search with text search, and visits a retrieved webpage to verify the final answer.
Figure 7: Qualitative case on tracing the ancestry of the gens associated with an ancient Roman coin. After localizing the coin in the video, VideoRover revises unsuccessful text-search attempts with image search and webpage browsing, ultimately identifying Calpus as the relevant son of Numa Pompilius.
Video understanding is moving beyond closed-context perception toward open-world evidence exploration, a paradigm formalized as Video Deep Research (VDR). However, existing multimodal search agents primarily target static images, and the current VDR benchmark relies on text-centric retrieval that discards crucial visual information. To address these limitations, we propose VideoSearcher, a closed-loop agentic framework that empowers Vision-Language Models with multi-tool reasoning for VDR. VideoSearcher unifies temporal localization, spatial focusing, and multimodal search within a single reasoning trajectory, enabling agents to progressively ground visual clues, retrieve relevant evidence, and synthesize answers. To optimize knowledge-intensive reasoning trajectories, we propose Bi-branch Sequence Policy Optimization (BiSPO), a reinforcement learning algorithm that decouples tool-invocation optimization from answer-accuracy optimization. This design provides stable learning signals for both evidence-grounded reasoning and purposeful tool use. Furthermore, we construct VideoSearch-QA, the first benchmark designed to evaluate open-world video information grounding and multimodal search-based reasoning. Extensive experiments demonstrate that VideoSearcher significantly outperforms prior open-source agentic baselines across various search-oriented and multimodal understanding benchmarks.
Zhenkun Gao, Yicheng Bao, Jinlong Peng +13
East China Normal University · Tencent Youtu Lab · HFIPS, Chinese Academy of Sciences +3
Existing deep research agents are designed primarily for text- and image-based web sources, while video reasoning systems typically assume that relevant videos are provided in advance. We study open-web video research, where an agent must autonomously discover relevant videos, navigate their temporal content, and ground answers in visual evidence. Training such agents at scale is challenging as live video interaction is slow and unreliable, whereas fixed local simulation can induce retrieval-specific shortcuts that fail to transfer to the open web. We introduce VideoResearchAgent, a scalable training framework to address these challenges. First, we introduce controllable task synthesis pipeline to synthesize multi-hop research tasks from timestamped visual evidence while filtering text-only shortcuts. Second, we build a field-aligned local video simulator that preserves deployment-facing search and watch interactions while accelerating video search by a factor of 34.5-64.6. Third, we introduce Retrieval-Domain-Randomized GRPO (RDR-GRPO), which diversifies candidate rankings, distractors, metadata, and result structure during training to reduce overfitting to simulated retrieval. On Video-BrowseComp, the VideoResearchAgent trained using Qwen3.5-4B achieves 40.48% accuracy, comparable to Gemini-3-Flash-Preview, while reducing cumulative API-token consumption by 74.9% relative to the untrained model. Together, these results establish an accurate and efficient training recipe for open-web video research.
Yuhang Zhou, Fei Li, Yuxi Wu +2
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · Singapore Management University
We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding coupled with open-web exploration. Preliminary evaluations reveal two critical bottlenecks in current models: (1) modality bias, where agents bypass visual tools in favor of textual search, and (2) parametric knowledge leakage, where models rely on internal memory rather than genuine tool-augmented execution. To address these challenges, we propose Video-DR, featuring a decoupled perception-exploration pipeline with stage-wise tool unlocking that compels exhaustive cross-frame visual grounding prior to web retrieval. Our framework adopts a two-stage training recipe: supervised fine-tuning followed by Group Relative Policy Optimization (GRPO), enabling autonomous exploration that breaks the imitation-learning ceiling. Furthermore, we curate Video-DR-Bench, a human-AI collaborative benchmark comprising 200 complex, multi-hop VQA instances. Empirical results demonstrate that our Video-DeepResearch-35B-A3B establishes a new state-of-the-art of 64.0% average accuracy, surpassing proprietary Claude-4.5-Sonnet (59.0%) by 5.0 points and significantly outperforming GPT-5 (52.5%) and Gemini 2.5 Pro (57.5%). The 30B-A3B variant achieves 59.3%, competitive with Claude-4.5-Sonnet and demonstrating the effectiveness of our training paradigm even at compact scale. Code: https://github.com/Osilly/Vision-DeepResearch.