VideoSlate: Stateful Commit Control for Temporally Grounded Agentic Video Reasoning

Shuo Zheng; Yuanjie Dang; Peng Chen; Jianjun Xiang; Ronghua Liang; Mohammed Bennamoun; Qiuhong Ke

cs.CVOct 12, 2026

VideoSlate: Stateful Commit Control for Temporally Grounded Agentic Video Reasoning

Authors: Shuo Zheng, Yuanjie Dang, Peng Chen, Jianjun Xiang, Ronghua Liang, Mohammed Bennamoun, Qiuhong Ke

Organizations: Zhejiang University of Technology · University of Western Australia · Monash University

Abstract

Agentic video reasoning systems gather and assess video evidence across stages, selecting relevant spans as visual evidence for answering questions. New candidates and observations prompt reassessment of the current span, but revisions can correct localization errors or discard useful evidence already present in the adopted span. Systems therefore need to connect evidence organization across stages with temporal state control, allowing accumulated information to jointly inform revision assessment while explicit adoption decisions govern how assessment outcomes affect downstream visual evidence. We introduce VideoSlate, a stateful commit-control framework built around shared evidence. A typed evidence blackboard links candidate spans, provenance, construction, and verifier observations, enabling an incumbent-relative certifier to assess revision support from the candidate, current span, and related proposals. Based on certification, the controller authorizes a revision or retains the current span, separating evidence accumulation from state updates and recording each decision and its evidence on the blackboard. Extensive evaluations across video grounding and grounded question-answering benchmarks show that VideoSlate can improve temporal localization through selective revision.

Figures & tables

Figure 1: (a) A challenger selected by VideoMind’s verifier can overwrite a stronger incumbent; deferral preserves it. (b) Selective commits retain beneficial revisions while suppressing harmful writes. (c) The certifier jointly assesses relative support, source support, and proposal context from the evidence blackboard. The controller uses the best candidate’s replacement score to authorize replacement or retain the current result.

Figure 2: VideoSlate’s selective temporal state revision. Ground & Propose obtains raw spans and their verifier support before constructing additional candidates. Record & Certify links candidates to their sources and scores replacement using stored source features. Decide & Commit selects the strongest challenger, using separate interval observations to resolve score ties when available. Only acceptance changes the committed span; both retention actions preserve it with distinct traces. The blackboard records proposals, evidence, and decisions throughout the step. Score bars are schematic.

Method	Size	[email protected]	[email protected]	mIoU	Acc	Acc@IoU
VTimeLLM ( Huang et al., 2024a )	7B	28.84	17.41	20.14	36.16	–
TimeChat ( Ren et al., 2024 )	7B	14.42	7.61	11.65	40.04	–
LITA ( Huang et al., 2024b )	13B	29.49	16.29	21.49	34.44	–
VTimeLLM (FT) ( Huang et al., 2024a )	7B	43.69	26.13	29.92	57.58	17.13
TimeChat (FT) ( Ren et al., 2024 )	7B	40.13	21.42	26.29	49.46	10.92
TOGA ( Gupta et al., 2025 )	7B	29.91	19.79	25.53	–	–

Table 1: Grounded VideoQA on ReXTime ( Chen et al., 2024a ) . Acc@IoU requires a correct answer and temporal IoU

\geq 0.5

. Bold marks the best results; underlining marks the second best.

Closed-source systems
Method	Size	long-acc.	mIoU	rec.@IoU	acc.@IoU
GPT-4o ( OpenAI, 2024 )	–	45.2	5.62	8.30	4.38
Gemini-1.5-Pro ( Reid et al., 2024 )	–	37.2	3.95	5.81	2.53
Claude-3.5-Sonnet ( Anthropic, 2024 )	–	40.5	3.99	5.67	2.79
Open-source systems
Video-LLaVA ( Lin et al., 2023a )	7B	16.2	1.13	1.96	0.59

Table 2: Grounded VideoQA on the 3,000-question CG-Bench Mini evaluation ( Chen et al., 2025 ) . long-acc. is answer accuracy; rec.@IoU and acc.@IoU average localization recall and answer-correct hits over IoU thresholds 0.1–0.5.

Method	Size	IoU			IoP			Acc@GQA
	Size	[email protected]	[email protected]	mIoU	[email protected]	[email protected]	mIoP	Acc@GQA
FrozenBiLM NG+ ( Yang et al., 2022 )	890M	13.50	6.10	9.60	28.50	23.70	24.20	17.50
SeViLA ( Yu et al., 2023 )	4B	29.20	13.80	21.70	34.70	22.90	29.50	16.60
LangRepo ( Kahatapitiya et al., 2024 )	$8{\times}$ 7B	–	12.20	18.50	–	28.70	31.30	17.10
VideoStreaming ( Qian et al., 2024b )	8.3B	–	13.30	19.30	–	31.00	32.20	17.80
LLoVi ( Zhang et al., 2024a )	1.8T	–	15.30	20.00	–	36.90	37.30	24.30

Table 3: Localization and grounded-QA performance on the NExT-GQA official test ( Xiao et al., 2024 ) . IoP divides temporal intersection by predicted duration; Acc@GQA requires a correct answer and IoP

\geq 0.5

Method	Size	Charades-STA				ActivityNet-Captions
Method	Size	[email protected]	[email protected]	[email protected]	mIoU	[email protected]	[email protected]	[email protected]	mIoU
VTimeLLM ( Huang et al., 2024a )	7B	51.00	27.50	11.40	31.20	44.00	27.80	14.30	30.40
TimeChat ( Ren et al., 2024 )	7B	51.50	32.20	13.40	–	–	–	–	–
Momentor ( Qian et al., 2024a )	7B	42.60	26.60	11.60	28.50	42.90	23.00	12.40	29.30
ChatVTG ( Qu et al., 2024 )	7B	52.70	33.00	15.90	34.90	40.70	22.50	9.40	27.20
E.T. Chat ( Liu et al., 2024d )	4B	65.70	45.90	20.00	42.30	24.10	12.80	6.10	18.90

Table 4: Video temporal grounding on the Charades-STA official test and ActivityNet-Captions.

Ground & Propose
Stage and intervention	[email protected]	[email protected]	mIoU	Acc	Acc@IoU
Cluster-union proposals only	41.04	25.41	28.32	74.27	19.98
Record & Certify
w/o cluster-support terms	42.13	26.60	29.46	74.48	21.17
w/o union-indicator term	41.37	25.84	28.61	74.48	20.63
Decide & Commit

Table 5: Module-aligned controls on the ReXTime replay. Feature and candidate interventions retain other coefficients and thresholds. QA uses VideoMind-7B for controls and Qwen3-VL-8B for the full policy; joint metrics compare systems.

Table 8

Figure 3: Two outcomes of selective temporal revision. Blue and orange outlines mark the incumbent and challenger; coral reference spans and IoU values are evaluation-only overlays. Source-support difference is one certifier input; the replacement score determines whether the acceptance threshold is met. Top: accepting a challenger yields an R@0.5 rescue. Bottom: deferral preserves the incumbent and avoids a harmful expansion. — Figure 3: Two outcomes of selective temporal revision. Blue and orange outlines mark the incumbent and challenger; coral reference spans and IoU values are evaluation-only overlays. Source-support difference is one certifier input; the replacement score determines whether the acceptance threshold is met. Top: accepting a challenger yields an [email protected] rescue. Bottom: deferral preserves the incumbent and avoids a harmful expansion.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Blackboard record	Stored information	Write and update rule
Incumbent $b_{t}$	Currently selected span	Initialized from grounder output; replaced only on controller-authorized acceptance
Candidate records $\mathcal{R}_{t}$	Boundaries, ranks, provenance, and parent links	Appended by proposal construction; do not change $b_{t}$
Evidence records $\mathcal{E}_{t}$	Observations linked to candidates	Appended by evidence collection; read during certification and selection
Decision traces $\mathcal{T}_{t}$	Selected challenger, score, thresholds, action, and fallback reason	Recorded after each decision, including retention of $b_{t}$

Appendix

Table S1: Blackboard records and update rules in VideoSlate.

Operator	Construction rule
Direct proposal	Preserve the boundaries of $a$ .
Intersection	Use $b_{t}\cap a$ if its duration is positive.
Minimum-duration window	Center at $m(a)$ with width $\max(w(a),\min(64,T))$ .
Expanded window	Center at $m(a)$ with width $\max(1.25w(a),\min(64,T))$ .
Capped incumbent–proposal union	Form the smallest interval enclosing $b_{t}$ and $a$ ; if longer than $\min(96,T)$ , use that width at its midpoint.
Capped verifier-cluster union	Form the smallest interval enclosing the support cluster; if longer than $\min(96,T)$ , use that width at the support-weighted cluster center.

Appendix

Table S2: Candidate construction operators in the primary configuration. All times are in seconds; windows are restricted to

[0,T]

Bucket	Condition	Label
Hard regression	$\operatorname{IoU}(b,g)\geq 0.5$ and $\operatorname{IoU}(c_{i},g)<0.3$	0
Strong positive	$\Delta_{i}\geq 0.20$	1
Weak positive	$0.05\leq\Delta_{i}<0.20$	1
Unlabeled tie	$\|\Delta_{i}\|<0.05$	None
Weak negative	$-0.20<\Delta_{i}\leq-0.05$	0
Strong negative	$\Delta_{i}\leq-0.20$	0

Appendix

Table S3: Train-side pair-label construction. “None” denotes an unlabeled tie used for analysis but not binary fitting.

Index	Symbol	Feature
1	$s_{b}$	Incumbent source support
2	$r_{i}$	Stored proposal-rank coordinate
3	$s_{i}$	Candidate source support
4	$s_{i}^{\max}$	Source-cluster maximum
5	$\bar{s}_{i}$	Source-cluster mean
6	$u_{i}$	Capped verifier-cluster-union indicator

Appendix

Table S4: Ordered input features of the primary certifier. Source-support coordinates and interval observations are distinct.

Commit rule	[email protected]	[email protected]	[email protected]	mIoU	Commits	R/S/H
Retain incumbent	73.71	58.76	29.92	49.72	0	0/0/0
Proposal probability	73.82	58.92	30.16	49.83	382	21/15/0
Proposal ranking	74.35	60.27	32.42	50.78	1,494	105/49/0
Verifier maximum	75.54	60.05	31.48	50.94	2,347	158/110/5
Random commit (mean)	74.96	60.04	32.24	50.79	2,176	143/96/1
Unconditional commit	75.67	60.78	33.58	51.57	3,720	242/167/2

Appendix

Table S5: Charades-STA commit controls on shared frozen candidates. Random commit matches the duration rule’s selected challengers and commit count; it averages seeds 3030–3032, with event counts rounded to integers. Other commit counts differ. R/S/H denote rescue, strict hurt, and hard hurt.

Output policy	Commit rule	[email protected]	[email protected]	mIoU
Protected incumbent	Never replace	40.28	25.30	27.98
Absolute score margin 0.10	$v(c)\!\geq\!0.35,\ v(c)-v(b)\!\geq\!0.10$	42.45	27.04	30.00
Absolute score margin 0.05	$v(c)\!\geq\!0.35,\ v(c)-v(b)\!\geq\!0.05$	42.78	27.90	31.36
Absolute verifier top-1	Unconditional $\arg\max_{c}v(c)$	46.91	28.88	33.55
Primary frozen policy	Pairwise score + fixed threshold	58.85	38.98	41.18

Appendix

Table S6: Decision-rule controls on ReXTime with shared frozen candidates and aligned deterministic ties. Scalar gates use interval observations; the primary policy uses the pairwise replacement score.

Configuration	Replacement score	Consensus	Provenance	[email protected]	[email protected]	mIoU
No replacement score	–	–	–	40.28	25.30	27.98
No cluster terms	$\checkmark$	–	$\checkmark$	42.13	26.60	29.46
No union-indicator term	$\checkmark$	$\checkmark$	–	41.37	25.84	28.61
Primary frozen policy	$\checkmark$	$\checkmark$	$\checkmark$	58.85	38.98	41.18

Appendix

Table S7: Frozen feature-contribution interventions on the ReXTime evaluation. Remaining coefficients and thresholds stay fixed; removing the replacement score returns the incumbent. These are score interventions without refitting.

Candidate configuration	[email protected]	[email protected]	mIoU	A/D/K	Rescue	Strict	Hard
Primary frozen policy	58.85	38.98	41.18	881/40/0	134	8	0
Remove direct	41.48	25.52	28.51	211/710/0	14	12	1
Direct only	59.50	39.41	41.36	798/123/0	132	2	0
Remove merge candidates	59.50	39.41	41.40	821/100/0	132	2	0
Only merge candidates	41.04	25.41	28.32	171/666/84	11	10	1

Appendix

Table S8: Candidate-family interventions with fixed coefficients and thresholds. Provenance-based families may overlap. A/D/K count accept/defer/keep actions; Strict and Hard count the corresponding hurts.

Budget	Gate	[email protected]	[email protected]	mIoU	Rescue	Strict	Hard
25%	Candidate confidence	43.65	27.47	30.16	22	2	0
25%	Verifier margin	45.93	29.32	32.24	38	1	0
25%	Learned certifier	46.58	29.21	32.42	37	1	0
25%	Random (mean)	44.63	28.52	31.14	32	3	0
50%	Candidate confidence	48.21	30.40	33.34	53	6	0
50%	Verifier margin	50.81	32.14	35.44	65	2	0

Appendix

Table S9: Original budget grid on the full ReXTime replay. Random (mean) averages evaluation results over three random seeds (3030–3032); event counts are rounded to the nearest integer. All orderings coincide at 100%. Localization metrics are percentages; Rescue, Strict, and Hard are event counts.

Comparator	$\Delta$ [email protected]	$\Delta$ [email protected]	$\Delta$ mIoU
Random (single run)	$+2.71\,[+0.78,+4.50]$	$+2.71\,[+0.76,+4.35]$	$+2.15\,[+1.30,+2.93]$
Candidate confidence	$+3.47\,[+1.09,+5.54]$	$+3.80\,[+1.53,+5.71]$	$+3.03\,[+1.93,+4.00]$
Verifier margin	$+0.87\,[-0.54,+1.99]$	$+2.06\,[+0.67,+3.09]$	$+0.93\,[+0.20,+1.49]$

Appendix

Table S10: Learned minus comparator at 50% budget: effect [95% descriptive paired video-bootstrap interval]. Localization differences are percentage points. The random comparator uses one fixed ordering (seed 3030).

Temporal Video Grounding

Temporal video grounding locates the moments in a video that match a natural-language query, enabling precise search and evidence-based understanding.

See all 169 papers

Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidence
Oct 13, 2026Jiahao Meng, Zhongyuan Bao, Yue Tan +7Video Reasoning Video Understanding
SER: Learning to Ground Video Reasoning with Semantic Evidence Rewards
Jun 23, 2026Sheng Xia, Zhengqin Lai, Tianxiang Jiang +4Temporal Video Grounding Video Reasoning
Nanjing University · Shanghai Innovation Institute · Harbin Institute of Technology, Shenzhen +2
Question-Aware Evidence Ledgers for Video Relational Reasoning
Jun 1, 2026Yilin Ou, Mengshi Qi, Huadong MaVisual Spatial Reasoning Relational Reasoning
State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications, China

Oct 13, 2026J/K move · Enter open · S save

Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidence

Jiahao Meng, Zhongyuan Bao, Yue Tan, Haiquan Wen+6

Video Reasoning Video Understanding Temporal Video Grounding Group Relative Policy Optimization

Open paper AskPDF arXiv

Reliable video reasoning requires models to produce correct answers and expose supporting visual evidence. Open-o3-Video enables verifiable reasoning by embedding timestamps and object-level bounding boxes into reasoning traces, but is restricted to frames from an initial sparse sampler. We introduce Open-o3-Video-v2, which formulates video reasoning as a unified agentic evidencegrounded process. The model identifies missing information, acquires additional observations, grounds the moments and objects supporting its reasoning, and writes this evidence into the trajectory before continuing or answering. To support this behavior, we construct STGR-v2-CoT135K, including 52K multi-turn agentic grounded reasoning trajectories, and STGR-v2-RL-13K for reinforcement learning. We further propose Clue-Seek GRPO (CS-GRPO), which tracks how tool calls change accumulated evidence, rewards clue coverage and step-wise gains, and penalizes excessive, repetitive, or uninformative retrieval. On V-STaR, Open-o3-Video-v2-4B achieves 40.4% mAM and 56.3% mLGM, outperforming its Qwen3-VL-4B backbone by 8.5% and 12.6%, respectively. It also delivers consistent gains across general video understanding, long-video reasoning, and temporal grounding benchmarks. These results show that active evidence acquisition and verifiable object-level grounding are complementary for reliable video reasoning. The code and models are released at https://msalab-pku.github.io/projects/Open-o3-Video-v2.

Abstract

Temporal Video Grounding

Explore similar work

Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidence

SER: Learning to Ground Video Reasoning with Semantic Evidence Rewards

Question-Aware Evidence Ledgers for Video Relational Reasoning