cs.CVOct 13, 2026
SaveOpen-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidence
Abstract
Reliable video reasoning requires models to produce correct answers and expose supporting visual evidence. Open-o3-Video enables verifiable reasoning by embedding timestamps and object-level bounding boxes into reasoning traces, but is restricted to frames from an initial sparse sampler. We introduce Open-o3-Video-v2, which formulates video reasoning as a unified agentic evidencegrounded process. The model identifies missing information, acquires additional observations, grounds the moments and objects supporting its reasoning, and writes this evidence into the trajectory before continuing or answering. To support this behavior, we construct STGR-v2-CoT135K, including 52K multi-turn agentic grounded reasoning trajectories, and STGR-v2-RL-13K for reinforcement learning. We further propose Clue-Seek GRPO (CS-GRPO), which tracks how tool calls change accumulated evidence, rewards clue coverage and step-wise gains, and penalizes excessive, repetitive, or uninformative retrieval. On V-STaR, Open-o3-Video-v2-4B achieves 40.4% mAM and 56.3% mLGM, outperforming its Qwen3-VL-4B backbone by 8.5% and 12.6%, respectively. It also delivers consistent gains across general video understanding, long-video reasoning, and temporal grounding benchmarks. These results show that active evidence acquisition and verifiable object-level grounding are complementary for reliable video reasoning. The code and models are released at https://msalab-pku.github.io/projects/Open-o3-Video-v2.
Figures & tables
Figure 1: From single-pass spatio-temporal grounded reasoning to agentic evidence seeking and grounded reasoning. Left: Open-o3-Video-v1 reasons over fixed sampled frames and grounds visible evidence, yielding verifiable spatio-temporal support on short videos. Center: Open-o3-Video-v2 performs adaptive temporal zoom-in through iterative crop_video calls, following a seek–ground–reason process to find evidence beyond the initial frames for long video understanding. Right: Comparison of five models on V-STaR and six general video understanding and reasoning benchmarks.
| Method | Adaptive temp. res. | Temp. ev. | Obj. ev. | Native agent |
| Open-o3-Video [ 38 ] | ✗ | ✓ | ✓ | ✗ |
| VideoZoomer [ 15 ] | ✓ | ✗ | ✗ | ✓ |
| Video-o3 [ 67 ] | ✓ | ✗ | ✗ | ✓ |
| VideoTemp-o3 [ 36 ] | ✓ | ✗ | ✗ | ✓ |
| VideoSeeker [ 76 ] | ✓ | ✗ | ✗ | ✓ |
| Open-o3-Video-v2 | ✓ | ✓ | ✓ | ✓ |
Table 1: Capability comparison with closely related video reasoning methods. “Adaptive temp. res.” denotes adaptive temporal resampling, “Temp. ev.” temporal evidence explicitly cited in reasoning, “Obj. ev.” object-level evidence explicitly cited in reasoning, and “Native agent” multi-turn interaction with interleaved observations.
Figure 2: Overview of the data construction pipelines and STGR-v2 data statistics. (a) Open-o3-Video constructs single-turn spatio-temporal grounded reasoning data through initial annotation, bounding-box filtering, and self-consistency checking. (b) Open-o3-Video-v2 extends this pipeline to agentic multi-turn reasoning through strict data filtering, spatio-temporal evidence annotation, CoT (chain-of-thought) rewriting with tool observations, and quality checking. (c) STGR-v2-CoT-135K contains 135K SFT samples across seven task groups, including 59K spatio-temporal grounded reasoning samples, while STGR-v2-RL-13K contains 13K RL samples selected through strict filtering.
Figure 3: Overview of the reinforcement learning designs in Open-o3-Video and Open-o3-Video-v2. (a) Open-o3-Video produces a single-turn spatio-temporal grounded reasoning trace and optimizes it with format, answer-accuracy, and grounded thinking rewards; the thinking reward jointly evaluates temporal proximity and spatial overlap with the target evidence. (b) Open-o3-Video-v2 extends this formulation to multi-turn agentic reasoning with CS-GRPO. The model iteratively calls crop_video to acquire denser observations, while CS-GRPO tracks how the accumulated evidence evolves across calls and assigns credit according to whether each interaction contributes useful new evidence.
| Model | What | When (Temporal IoU) | Where (Visual IoU) | Overall | |||
| Acc | Chain1 | Chain2 | Chain1 | Chain2 | mAM | mLGM | |
| GPT-4o [ 29 ] | 60.8 | 16.7 | 12.8 | 6.5 | 3.0 | 26.8 | 38.2 |
| Gemini-2-Flash [ 13 ] | 53.0 | 24.5 | 23.8 | 4.6 | 2.2 | 26.9 | 35.6 |
| Gemini-3-Pro [ 13 ] | 59.1 | 24.2 | 23.8 | 7.2 | 4.8 | 29.9 | 41.2 |
| Video-LLaMA3-7B [ 69 ] | 41.9 | 23.0 | 23.1 | 0.9 | 0.2 | 21.7 | 27.0 |
| LLaVA-Video-7B [ 75 ] | 49.5 | 10.5 | 12.2 | 1.9 | 1.3 | 20.8 | 27.3 |
Table 2: Performance on V-STaR (%). Chain1 denotes what–when–where , whereas Chain2 denotes what–where–when . mAM and mLGM jointly summarize answer accuracy and spatio-temporal alignment. Bold and underlined numbers denote the best and second-best results, respectively.
| Model | General Video Understanding and Reasoning | Temporal Grounding | |||||||
| Video- Holmes | LV Bench | Video- MMMU | WorldSense | Video MME | Video MME-v2 | Charades- TimeLens | TVG Bench | ||
| Acc | Acc | Acc | Overall | Recog. | Acc | Acc | mIoU | mIoU | |
| Qwen2.5-VL-7B [ 3 ] | 34.7 | 45.3 | 51.2 | 36.1 | 33.7 | 62.4 | 25.3 | 39.3 | 16.3 |
| Qwen3-VL-4B [ 2 ] | 37.3 | 46.0 | 54.4 | 41.7 | 42.0 | 65.3 | 26.2 | 45.6 | 27.1 |
| Qwen3-VL-8B [ 2 ] | 44.2 | 47.3 | 57.3 | 45.7 | 44.0 | 67.4 | 27.9 | 48.3 | 29.4 |
| VideoRFT-7B [ 53 ] | 40.3 | 40.4 | 51.1 | 38.2 | 36.6 | 59.8 | 22.4 | 17.0 | 14.3 |
Table 3: Performance on general video understanding and temporal grounding benchmarks (%). For WorldSense, we report overall and recognition accuracy. The last two columns report mIoU. All other columns report accuracy. Bold and underlined numbers denote the best and second-best results, respectively.
Figure 4: Qualitative comparison with Video-o3. Left: Video-o3 searches an incorrect segment, while Open-o3-Video-v2 inspects the relevant 1170–1340 s interval and grounds the pigs at 1310 s, providing direct evidence for answer D. Right: although the question explicitly specifies 07:48–07:50, Video-o3 queries 478–480 s and incorrectly predicts bowing. Open-o3-Video-v2 inspects the correct 468–470 s interval, grounds the three girls at 469 s, and identifies their group handshake. In both cases, the acquired observation is converted into explicit spatio-temporal evidence and used directly in the subsequent reasoning process.
| Model | V-STaR | VideoMME | |
| mAM | mLGM | Acc | |
| Qwen3-VL-4B (baseline) | 31.9 | 43.7 | 65.3 |
| Open-o3-Video-v2-SFT-4B (SFT) | 38.2 | 53.4 | 63.7 |
| Open-o3-Video-v2-4B (SFT+RL) | 40.4 | 56.3 | 66.6 |
Table 4: Stage-wise ablation of Open-o3-Video-v2-4B (%). V-STaR is evaluated with mAM and mLGM, while VideoMME is evaluated with accuracy.
| Reasoning mechanism | Video-Holmes | LVBench | V-STaR mLGM |
| Qwen3-VL-4B (base) [ 2 ] | 37.3 | 46.0 | 43.7 |
| Single-turn, no evidence | 45.0 | 48.8 | 47.8 |
| Multi-turn, no evidence | 47.1 | 51.9 | 52.3 |
| Multi-turn with grounded evidence (ours) | 48.0 | 53.2 | 56.3 |
Table 5: Comparison of reasoning mechanisms. All variants use the same backbone, source training data, and evaluation setting. Only the reasoning mechanism is changed.
| Method | Video-Holmes | LVBench | VideoMME |
| GRPO | 46.7 | 52.0 | 65.1 |
| CS-GRPO w/o gain bonus | 47.0 | 52.2 | 66.2 |
| CS-GRPO w/o penalties | 46.9 | 50.4 | 65.5 |
| CS-GRPO (ours) | 48.0 | 53.2 | 66.6 |
Table 6: Comparison of GRPO and CS-GRPO variants across three video understanding benchmarks (%).
| Setting | SFT | CS-GRPO |
| Training framework | ms-swift | verl AgentLoop |
| Model initialization | Qwen3-VL-4B-Instruct | SFT checkpoint |
| Training corpus | STGR-v2-CoT-135K | STGR-v2-RL-13K |
| Training epochs | 1 | 1 |
| Global batch size | 16 | 32 |
| Learning rate |
Table 7: Training and optimization configurations of Open-o3-Video-v2-4B.
| Stage | Initial Observation | Coarse Crop | Medium Crop | Fine Crop | Interaction Limit |
| SFT | 1 FPS, frames | 1 FPS, frames | 2 FPS, frames | 4 FPS, frames | Given by annotated trajectory |
| CS-GRPO | 1 FPS, uniform frames | frames | frames | frames | assistant turns |
| Inference | 1 FPS, frames | Model-selected crop_video observations | assistant turns | ||
Table 8: Video observation and interaction settings of Open-o3-Video-v2. SFT frame caps are applied to each observation. For CS-GRPO, crop strategies control the maximum number of uniformly sampled frames within the requested interval rather than fixed sampling rates.
| Setting | WorldSense | VideoMMMU |
| Single response ( ) | 37.5 | 52.3 |
| Majority voting ( ) | 37.3 | 53.1 |
| Confidence-aware voting ( ) | 38.5 | 54.1 |
Table 13: Test-time scaling of Open-o3-Video on WorldSense and VideoMMMU (%). Confidence-aware voting uses grounded evidence to weight eight sampled responses.
| Model | QA | S | T | S&T | QA&S | QA&T | QA&S&T |
| Qwen3-VL-4B | 59.6 | 52.7 | 25.9 | 17.8 | 40.1 | 15.6 | 11.7 |
| Open-o3-Video-v2-4B | 62.8 | 68.0 | 41.5 | 32.8 | 50.1 | 26.4 | 21.7 |
Table 14: Consistency between answer correctness and spatio-temporal localization on V-STaR Chain1 (%). Spatial correctness uses vIoU , and temporal correctness uses tIoU .
Figure 5: Effect of initial input-frame count and maximum interaction budget on LVBench and Video-Holmes. Results are evaluated on randomly sampled 30% subsets of each benchmark using the same model and decoding configuration.
| Reward level | Coefficient | Low / default / high | Video-Holmes | V-STaR mLGM |
| Final reward | 47.6 / 48.0 / 48.0 | 55.8 / 56.3 / 56.2 | ||
| 48.1 / 48.0 / 48.2 | 56.1 / 56.3 / 55.7 | |||
| 48.4 / 48.0 / 47.6 | 55.9 / 56.3 / 56.6 | |||
| 47.3 / 48.0 / 47.9 | 56.0 / 56.3 / 56.3 | |||
| Tool reward | 47.6 / 48.0 / 48.1 | 56.0 / 56.3 / 56.6 | ||
| 47.3 / 48.0 / 47.8 | 55.4 / 56.3 / 57.1 |
Table 15: Sensitivity of CS-GRPO to reward weights. Each coefficient is varied independently while all others remain at their default values. Results follow the low/default/high order of the tested weights.
| Setting | VideoMME | Video-Holmes | LVBench |
| Uniform sampling (120 frames, no tool) | 65.4 | 47.4 | 45.5 |
| Length-matched random intervals | 64.6 | 47.1 | 46.2 |
| Open-o3-Video-v2 | 66.6 | 48.0 | 53.2 |
Table 16: Analysis of adaptive evidence acquisition under comparable visual observation budgets (%). The uniform baseline uses the same model with tool use disabled and uniformly samples 120 frames. For random retrieval, each tool-returned clip is replaced by a length-matched random interval from the same video while preserving the number of tool calls and returned frames.
| Model | Accuracy (%) |
| Qwen3-VL-4B | 37.6 |
| Video-o3 | 38.3 |
| Open-o3-Video-v2 | 48.3 |
Table 17: Accuracy on a 149-question long-horizon subset of LVBench, covering long videos with sparse evidence and questions involving event chains, ordinal localization, or repeated-instance tracking.
Figure 6: Qualitative comparisons on fine-grained visual reasoning. Top: For action-change perception, Open-o3-Video-v2 inspects the relevant 134–143 s interval and grounds the man’s hand, sea urchin, and fish at 138.5 s. The denser observation reveals the fish’s blue body and yellow fins after the interaction, enabling the correct answer. Bottom: For fine-grained event understanding, Open-o3-Video-v2 grounds the metal ring cutter and pineapple slice and further inspects the 33–46 s interval using denser temporal sampling. The resulting observation reveals the repeated removal of the fibrous core and provides the missing evidence for answer B. In both cases, targeted temporal observation together with explicit object-level grounding exposes the decisive visual evidence and yields a human-verifiable spatio-temporal reasoning trace.
Figure 7: Qualitative comparisons on fine-grained visual counting. Top: For counting food pictures on the menu at 17:49, Open-o3-Video-v2 inspects the relevant 1065–1075 s interval and grounds the menu at 1069.5 s, providing a clearer view of the individual food images and enabling the correct count of 11. In contrast, Video-o3 searches an incorrect temporal interval and produces an incorrect count. Bottom: For counting the people in the car, both methods inspect the relevant 10–12 s interval, but Open-o3-Video-v2 further grounds the car and its occupants, allowing it to distinguish three individuals rather than two. In both cases, explicit instance-level grounding complements temporal evidence acquisition by resolving the individual visual entities required for accurate counting.
Figure 8: Representative failure cases of Open-o3-Video-v2. Top: Insufficient identity knowledge leads to an incorrect answer even after a relevant face is retrieved. Middle: During a camera pan, inaccurate instance discrimination and association across changing views lead to an incorrect count of the women appearing offstage. Bottom: The model anchors its reasoning to the wrong scoring event and misidentifies the player who ends the game. These examples expose remaining limitations in world knowledge, fine-grained spatio-temporal instance reasoning, and temporal event selection, respectively.
More in this topic
Video Reasoning
Video reasoning studies how AI systems use video evidence to infer events, actions, temporal relations, causes, and future states.
See all 200 papers