Agentic video reasoning systems gather and assess video evidence across stages, selecting relevant spans as visual evidence for answering questions. New candidates and observations prompt reassessment of the current span, but revisions can correct localization errors or discard useful evidence already present in the adopted span. Systems therefore need to connect evidence organization across stages with temporal state control, allowing accumulated information to jointly inform revision assessment while explicit adoption decisions govern how assessment outcomes affect downstream visual evidence. We introduce VideoSlate, a stateful commit-control framework built around shared evidence. A typed evidence blackboard links candidate spans, provenance, construction, and verifier observations, enabling an incumbent-relative certifier to assess revision support from the candidate, current span, and related proposals. Based on certification, the controller authorizes a revision or retains the current span, separating evidence accumulation from state updates and recording each decision and its evidence on the blackboard. Extensive evaluations across video grounding and grounded question-answering benchmarks show that VideoSlate can improve temporal localization through selective revision.
Figures & tables
Figure 1: (a) A challenger selected by VideoMind’s verifier can overwrite a stronger incumbent; deferral preserves it. (b) Selective commits retain beneficial revisions while suppressing harmful writes. (c) The certifier jointly assesses relative support, source support, and proposal context from the evidence blackboard. The controller uses the best candidate’s replacement score to authorize replacement or retain the current result.
Figure 2: VideoSlate’s selective temporal state revision. Ground & Propose obtains raw spans and their verifier support before constructing additional candidates. Record & Certify links candidates to their sources and scores replacement using stored source features. Decide & Commit selects the strongest challenger, using separate interval observations to resolve score ties when available. Only acceptance changes the committed span; both retention actions preserve it with distinct traces. The blackboard records proposals, evidence, and decisions throughout the step. Score bars are schematic.
Table 1: Grounded VideoQA on ReXTime ( Chen et al., 2024a ) . Acc@IoU requires a correct answer and temporal IoU ≥0.5 . Bold marks the best results; underlining marks the second best.
Method
Size
long-acc.
mIoU
rec.@IoU
acc.@IoU
Closed-source systems
GPT-4o ( OpenAI, 2024 )
–
45.2
5.62
8.30
4.38
Gemini-1.5-Pro ( Reid et al., 2024 )
–
37.2
3.95
5.81
2.53
Claude-3.5-Sonnet ( Anthropic, 2024 )
–
40.5
3.99
5.67
2.79
Open-source systems
Video-LLaVA ( Lin et al., 2023a )
7B
16.2
1.13
1.96
0.59
Table 2: Grounded VideoQA on the 3,000-question CG-Bench Mini evaluation ( Chen et al., 2025 ) . long-acc. is answer accuracy; rec.@IoU and acc.@IoU average localization recall and answer-correct hits over IoU thresholds 0.1–0.5.
Table 3: Localization and grounded-QA performance on the NExT-GQA official test ( Xiao et al., 2024 ) . IoP divides temporal intersection by predicted duration; Acc@GQA requires a correct answer and IoP ≥0.5 .
Table 5: Module-aligned controls on the ReXTime replay. Feature and candidate interventions retain other coefficients and thresholds. QA uses VideoMind-7B for controls and Qwen3-VL-8B for the full policy; joint metrics compare systems.
Table 8
Figure 3: Two outcomes of selective temporal revision. Blue and orange outlines mark the incumbent and challenger; coral reference spans and IoU values are evaluation-only overlays. Source-support difference is one certifier input; the replacement score determines whether the acceptance threshold is met. Top: accepting a challenger yields an [email protected] rescue. Bottom: deferral preserves the incumbent and avoids a harmful expansion.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Blackboard record
Stored information
Write and update rule
Incumbent bt
Currently selected span
Initialized from grounder output; replaced only on controller-authorized acceptance
Candidate records Rt
Boundaries, ranks, provenance, and parent links
Appended by proposal construction; do not change bt
Evidence records Et
Observations linked to candidates
Appended by evidence collection; read during certification and selection
Decision traces Tt
Selected challenger, score, thresholds, action, and fallback reason
Recorded after each decision, including retention of bt
Appendix
Table S1: Blackboard records and update rules in VideoSlate.
Operator
Construction rule
Direct proposal
Preserve the boundaries of a .
Intersection
Use bt∩a if its duration is positive.
Minimum-duration window
Center at m(a) with width max(w(a),min(64,T)) .
Expanded window
Center at m(a) with width max(1.25w(a),min(64,T)) .
Capped incumbent–proposal union
Form the smallest interval enclosing bt and a ; if longer than min(96,T) , use that width at its midpoint.
Capped verifier-cluster union
Form the smallest interval enclosing the support cluster; if longer than min(96,T) , use that width at the support-weighted cluster center.
Appendix
Table S2: Candidate construction operators in the primary configuration. All times are in seconds; windows are restricted to [0,T] .
Bucket
Condition
Label
Hard regression
IoU(b,g)≥0.5 and IoU(ci,g)<0.3
0
Strong positive
Δi≥0.20
1
Weak positive
0.05≤Δi<0.20
1
Unlabeled tie
∣Δi∣<0.05
None
Weak negative
−0.20<Δi≤−0.05
0
Strong negative
Δi≤−0.20
0
Appendix
Table S3: Train-side pair-label construction. “None” denotes an unlabeled tie used for analysis but not binary fitting.
Index
Symbol
Feature
1
sb
Incumbent source support
2
ri
Stored proposal-rank coordinate
3
si
Candidate source support
4
simax
Source-cluster maximum
5
sˉi
Source-cluster mean
6
ui
Capped verifier-cluster-union indicator
Appendix
Table S4: Ordered input features of the primary certifier. Source-support coordinates and interval observations are distinct.
Table S5: Charades-STA commit controls on shared frozen candidates. Random commit matches the duration rule’s selected challengers and commit count; it averages seeds 3030–3032, with event counts rounded to integers. Other commit counts differ. R/S/H denote rescue, strict hurt, and hard hurt.
Table S6: Decision-rule controls on ReXTime with shared frozen candidates and aligned deterministic ties. Scalar gates use interval observations; the primary policy uses the pairwise replacement score.
Table S7: Frozen feature-contribution interventions on the ReXTime evaluation. Remaining coefficients and thresholds stay fixed; removing the replacement score returns the incumbent. These are score interventions without refitting.
Table S8: Candidate-family interventions with fixed coefficients and thresholds. Provenance-based families may overlap. A/D/K count accept/defer/keep actions; Strict and Hard count the corresponding hurts.
Table S9: Original budget grid on the full ReXTime replay. Random (mean) averages evaluation results over three random seeds (3030–3032); event counts are rounded to the nearest integer. All orderings coincide at 100%. Localization metrics are percentages; Rescue, Strict, and Hard are event counts.
Table S10: Learned minus comparator at 50% budget: effect [95% descriptive paired video-bootstrap interval]. Localization differences are percentage points. The random comparator uses one fixed ordering (seed 3030).
Reliable video reasoning requires models to produce correct answers and expose supporting visual evidence. Open-o3-Video enables verifiable reasoning by embedding timestamps and object-level bounding boxes into reasoning traces, but is restricted to frames from an initial sparse sampler. We introduce Open-o3-Video-v2, which formulates video reasoning as a unified agentic evidencegrounded process. The model identifies missing information, acquires additional observations, grounds the moments and objects supporting its reasoning, and writes this evidence into the trajectory before continuing or answering. To support this behavior, we construct STGR-v2-CoT135K, including 52K multi-turn agentic grounded reasoning trajectories, and STGR-v2-RL-13K for reinforcement learning. We further propose Clue-Seek GRPO (CS-GRPO), which tracks how tool calls change accumulated evidence, rewards clue coverage and step-wise gains, and penalizes excessive, repetitive, or uninformative retrieval. On V-STaR, Open-o3-Video-v2-4B achieves 40.4% mAM and 56.3% mLGM, outperforming its Qwen3-VL-4B backbone by 8.5% and 12.6%, respectively. It also delivers consistent gains across general video understanding, long-video reasoning, and temporal grounding benchmarks. These results show that active evidence acquisition and verifiable object-level grounding are complementary for reliable video reasoning. The code and models are released at https://msalab-pku.github.io/projects/Open-o3-Video-v2.
Video MLLMs often struggle with fine-grained spatio-temporal reasoning, sometimes generating correct answers based on irrelevant frames or objects. Although outputting spatio-temporal evidence during reasoning is a promising direction, existing RL frameworks typically rely on geometry-only (IoU) rewards, which can be sensitive to boundary perturbations and overlook semantic alignment. To address this, we propose Semantic Evidence Reward (SER), which reformulates spatio-temporal evidence grounding as a constrained verification task. Instead of computing pixel-level overlap, SER uses a referee VLM as a local checker to evaluate model-generated evidence claims across two dimensions: relevance and localization quality, combined with a temporal penalty. This design reduces the reliance on dense box annotations and enables training directly on standard video QA data. On the V-STAR benchmark, SER achieves 49.6% mLGM, improving by 3.0 points over the strong evidence-grounded baseline Open-o3-Video, demonstrating its potential in enhancing both answer accuracy and evidence grounding.
Sheng Xia, Zhengqin Lai, Tianxiang Jiang +4
Nanjing University · Shanghai Innovation Institute · Harbin Institute of Technology, Shenzhen +2
The VRR-QA challenge evaluates visual relational reasoning in videos, where answers often depend on implicit spatial relations, event boundaries, target identity, and dialogue context rather than a single salient frame. We present a test-time reasoning pipeline built around a strong GPT-5.5 video QA solver and a set of question-aware evidence ledgers. The initial solver answers each question from a uniform video representation, while routed ledgers are prompted to make the required targets, count units, reference frames, and temporal or spatial scope explicit for counting, spatial, endpoint, viewpoint, and dialogue reasoning. External tools such as open-vocabulary detection, depth cues, pair crops, ASR, and scene-graph ledgers are used only as evidence sources. A conservative gate keeps the current answer unless independent evidence uniquely supports a different option. The final evidence-gated pipeline achieves 92.95% overall accuracy and 93.79% macro accuracy on the challenge test split.
Yilin Ou, Mengshi Qi, Huadong Ma
State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications, China