SAVE: Less is More in Agentic Video Reasoning
Organizations: Technical University of Munich · ETH Zurich · MCML · Ulm University
Abstract
Agentic video-reasoning frameworks iteratively explore a video, choosing what to observe based on the query and evidence gathered so far. Existing methods are typically evaluated with different backbones, preprocessing pipelines, and computational budgets, making it difficult to determine which components actually drive performance, how much they cost, and whether agentic inference is preferable to native video reasoning in the first place. We conduct a controlled evaluation of a diverse set of recent agentic video-reasoning systems across two strong contemporary backbones, with explicit token and monetary-cost accounting. Across Gemini 3 Flash and Gemma 4, most existing agentic frameworks fail to consistently improve over native video inference and often increase computational cost. We investigate two primary questions: (1) How much agentic framework complexity is necessary for effective video reasoning? (2) When, if at all, is agentic video reasoning preferable to just using the backbone directly? To answer the first question, we introduce a deliberately minimal Active Exploration Baseline and show that it remains competitive with substantially more elaborate agents. We then propose targeted extensions to this baseline, yielding SAVE (Simple Agentic Video Explorer). SAVE outperforms all evaluated prior agentic methods in five of six backbone-benchmark settings. Turning to the second question, our results indicate that the advantage of agentic exploration grows when dense processing would otherwise represent answer-relevant events too sparsely, particularly for long videos and temporally concentrated evidence. Overall, our results suggest that agentic video reasoning should favor simple frameworks over increasingly elaborate pipelines and focus primarily on long-form video, where selective evidence acquisition offers the clearest advantage over native inference.
Figures & tables
| Gemini 3 Flash ( Perf. / Cost ) | Gemma 4 ( Perf. / Tokens ) | |||||
| Model | Minerva | Ego. | VSI | Minerva | Ego. | VSI |
| Native | 64.8 / $80.5 | 75.4 / $5.75 | 59.2 / $10.5 | 51.6 / 15.9K | 72.4 / 16.0K | 49.1 / 15.9K |
| VideoTree | 50.4 / $93.4 | 73.1 / $31.1 | 45.5 / $20.2 | 51.6 / 36.4K | 71.1 / 34.4K | 42.8 / 24.6K |
| VideoAgent | 38.5 / $34.7 | 64.2 / $4.76 | 40.6 / $10.6 | 42.3 / 22.3K | 67.7 / 5.1K | 44.5 / 26.2K |
| VideoLucy | 56.5 / $154.2 | 72.7 / $27.4 | 48.2 / $35.4 | 58.4 / 72.0K | 73.2 / 103.6K | 43.8 / 63.4K |
| VideoCuriousAgent | 51.7 / $235.8 | 72.3 / $51.8 | 45.6 / $61.2 | 36.4 / 26.4K | 67.7 / 9.9K | 41.4 / 24.1K |
| Dataset | Model | Acc./Ave. (%) | Input Tokens | Output Tokens | Total Tokens |
|---|---|---|---|---|---|
| Minerva | Active Exploration Baseline | 52.8 | 29.4K | 4.2K | 33.6K |
| SAVE w/o HQ and AD | 62.7 | 36.2K | 2.5K | 38.7K | |
| SAVE w/o AD | 63.3 | 40.5K | 2.8K | 43.3K | |
| SAVE | 66.4 | 38.3K | 1.5K | 39.7K | |
| VSIBench (tiny) | Active Exp. Baseline | 47.2 | 25.8K | 3.5K | 29.3K |
| SAVE w/o HQ and AD | 51.3 | 33.2K | 3.0K | 36.2K |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Performance (%) | Input Tokens | Output Tokens | Total Tokens | Inference Cost |
|---|---|---|---|---|---|
| Gemini 3 Flash | |||||
| Minerva | |||||
| Gemini 3 Flash Native Google DeepMind (2025) | 64.8 | 61.1K | 9.5K | 70.6K | $80.54 |
| VideoTree Wang et al. (2025c) | 50.4 | 99.8K | 6.2K | 106.0K | $93.39 |
| VideoAgent Wang et al. (2024) | 38.5 | 6.7K | 7.3K | 14.1K | $34.72 |
| VideoLucy Zuo et al. (2025) | 56.5 | 141.1K | 21.6K | 162.7K | $154.13 |
Video Reasoning
Video reasoning studies how AI systems use video evidence to infer events, actions, temporal relations, causes, and future states.
See all 220 papers