This paper addresses the critical and underexplored challenge of long video understanding with low computational budgets. We propose LongVideo-R1, an active, reasoning-equipped multimodal large language model (MLLM) agent designed for efficient video context navigation, avoiding the redundancy of exhaustive search. At the core of LongVideo-R1 lies a reasoning module that leverages high-level visual cues to infer the most informative video clip for subsequent processing. During inference, the agent initiates traversal from top-level visual summaries and iteratively refines its focus, immediately halting the exploration process upon acquiring sufficient knowledge to answer the query. To facilitate training, we first extract hierarchical video captions from CGBench, a video corpus with grounding annotations, and guide GPT-5 to generate 33K high-quality chain-of-thought-with-tool trajectories. The LongVideo-R1 agent is fine-tuned upon the Qwen-3-8B model through a two-stage paradigm: supervised fine-tuning (SFT) followed by reinforcement learning (RL), where RL employs a specifically designed reward function to maximize selective and efficient clip navigation. Experiments on multiple long video benchmarks validate the effectiveness of name, which enjoys superior tradeoff between QA accuracy and efficiency. All curated data and source code are provided in the supplementary material and will be made publicly available. Code and data are available at: https://github.com/qiujihao19/LongVideo-R1
Figures & tables
Figure 2: An illustration of generating CoTwT trajectories from clue-grounded video QA data. Drawing style was inspired by Ego-R1 [ 45 ]
Table 2
Method
KIR
TG
LVBench
V-MME/L
SFT 10K
46.4
39.1
39.1
57.7
SFT full
48.5
50.9
41.6
59.2
Ours 10K w/ RL
54.0
55.0
47.4
60.2
Ours full w/ RL
56.4
56.4
50.0
64.4
Table 4: QA accuracy (%) with respect to SFT data size.
Method
KIR
TG
LVBench
V-MME/L
SFT
48.5
50.9
41.6
59.2
w/o rloc
49.1
53.2
45.8
61.4
Ours
56.4
56.4
50.0
64.4
Table 5: QA accuracy (%) with or without the location reward.
Model
KIR
TG
LVBench
V-MME/L
Time
Qwen2.5-VL-3B
48.8
53.2
44.5
56.0
50.5
Qwen2.5-VL-7B
51.5
52.7
46.0
57.8
57.4
Qwen2.5-VL-32B
58.4
54.5
49.8
63.9
135.0
Qwen2.5-VL-72B
56.4
56.4
50.0
64.4
175.7
Table 6: QA accuracy (%) with respect to MLLM scales. The last column shows average inference time (on LVBench, in seconds).
Max Rounds
KIR
TG
LVBench
V-MME/L
Time (s)
10
49.1
52.3
43.0
57.1
103.6
20
49.5
58.2
47.3
63.2
170.8
25
54.6
54.1
48.5
63.6
172.8
30
56.4
56.4
50.0
64.4
175.7
Table 7: QA accuracy (%) with respect to maximum rounds of tool use (mainly the video_cap() function).
Figure 3: LongVideo-R1 can navigate in ultra-long videos efficiently. We show an example in a long-form TV drama, A Lifelong Journey .
Figure 4: An example of how LongVideo-R1 smartly navigates to the critical segment and answers the question.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
SFT
RL
Model init
Qwen3-8B
LongVideo-R1-SFT
LLM sequence length
32768
32768
Learning rate
1×10−5
1×10−6
Learning rate schedule
cosine decay
constant
Training epochs
3
2
Global batch size
32
12
Appendix
Table 8: Training hyper-parameters.
Table 9: System prompt for data generation.
Table 10: Video Caption Model Prompt.
Figure 5: More example on ultra-long videos.
Figure 6: More qualitative examples.
Figure 7: More qualitative examples.
Figure 8: LongVideo-R1 may sometimes be disturbed by similar information, but people can guide the model back on track with just a few hints.
Figure 9: LongVideo-R1 may sometimes be disturbed by similar information, but people can guide the model back on track with just a few hints.
Multimodal Large Language Models (MLLMs) have achieved remarkable progress on short video understanding yet remain limited on long videos due to the limited visual context window. Prevailing approaches rely on uniform frame sampling or recent coarse-to-fine agentic zooming, both of which struggle to localize sparse, decisive evidence in sufficiently long videos. We formulate long video understanding as a Sequential Evidence Acquisition (SEA) problem, in which an agent reads the video turn by turn along the temporal axis, deciding at each turn how fast to watch, what evidence to retain, when to revisit uncertain segments, and when to stop and answer. Inspired by this view, we propose VideoScout, a multi-turn reasoning agent that instantiates the SEA paradigm through adaptive reasoning pacing. Specifically, by dynamically controlling the viewing pace, VideoScout enables efficient traversal of long videos within a bounded visual context window, allowing the agent to access more video content while balancing content analysis depth with reading efficiency. To train VideoScout, we construct VideoScout-66K, a set of over 66K high-quality exploration turns from 10K answer-verified trajectories, and adopt a two-stage pipeline: cold-start supervised fine-tuning teaches the agent per-turn output format, while the Decoupled Clip and Dynamic sAmpling Policy Optimization (DAPO) algorithm performs trajectory-level reinforcement learning with a composite reward that jointly considers answer accuracy, output format compliance, and the temporal alignment between the agent's viewing progress and the teacher's answer timing measured by intersection-over-union (IoU). Extensive experiments on long video understanding and reasoning benchmarks demonstrate that our 7B model achieves strong performance compared with existing trained 7B agentic models.
Long-video understanding remains challenging for Multimodal Large Language Models (MLLMs) due to limited context length. Uniform sampling may miss crucial moments, while agent-based frame video understanding methods often evaluate frames independently, overlooking the temporal organization of videos. Ideally, evidence selection should mimic how humans answer questions about long videos: first locating the relevant segment from the global context, then zooming into local objects and details. We propose Temporal Tree of Thought T^3, a training-free framework for adaptive coarse-to-fine long-video understanding. T^3 constructs a question-agnostic hierarchical temporal tree via recursive temporally constrained clustering, where each node represents a contiguous segment with an informative key frame. During inference, T^3 performs an answer-retrieve-explore loop: it reasons over coarse representative frames, generates a search statement when evidence is insufficient, and expands relevant branches for finer-grained evidence. This process adaptively shifts the search target from temporal regions to specific objects and visual details to help video understanding. Experiments on VideoMME, LongVideoBench, and LVBench show that T^3 improves Qwen2.5-VL-7B by 0.5%, 4.6%, and 4.4%, respectively, under the same frame budget, demonstrating the effectiveness of structured temporal reasoning.
We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process. VSeek utilizes a natural language-driven search to identify relevant context within long videos and is post-trained with reinforcement learning (RL) to jointly formulate targeted search queries and reason over retrieved clips for LVQA. While RL post-training has revolutionized reasoning in symbolic domains such as mathematics and code, its application to long-video understanding remains hindered by a lack of verified rewards. To ensure that the retrieved context is relevant, we propose a novel neuro-symbolic approach that bridges open-ended natural language with discrete visual verification. Specifically, complex user queries are compiled into formal temporal logic specifications for systematically decomposing natural language questions into a definitive checklist of required atomic visual primitives, such as key objects and activities, along with their temporal ordering. These systematically derived grounding events provide the critical feedback signal for RL post-training, enabling dense, verifiable rewards based on the successful retrieval of these specific visual elements rather than relying entirely on outcome-only answer accuracy. By explicitly optimizing for this verifiable evidence-seeking behavior, VSeek improves Pass@1 scores by up to 8% and Pass@4 scores by 15% on long-video understanding benchmarks compared to base models. We open-source our code at https://utaustin-swarmlab.github.io/VSeek.
Harsh Goel, S P Sharan, Sahil Shah +4
The University of Texas at Austin, Austin, TX, USA