Tempo-Zero: Recursive Self-Play in Executable Worlds for Video Reasoning
Authors: Jingmin Zhu, Yanyan Zou, Xiuyu Li, James Bailey, Dinh Phung, Tao Yu, Yunyun Hou, Jun Liu, Jun Zhou, Sirui Wang, Nan Duan, Jiachen Liu, Qiuhong Ke
Organizations: Monash University · Institute of Automation, Chinese Academy of Sciences · Communication University of China · Lancaster University · Griffith University
Video reasoning requires understanding how events unfold over time, yet videos with accurate temporal annotations are expensive to collect. Self-play offers an alternative by letting a model generate and solve its own tasks without manual annotation. However, existing video self-play methods train on fixed sets of recorded videos, so they cannot alter the underlying event dynamics and rely on noisy pseudo-labels derived from the model's own outputs, such as majority voting, that risk reinforcing shared model errors. To overcome these limitations, we introduce Tempo-Zero, a self-improvement framework that closes a loop of executable environments, execution-verified self-play, and environment evolution, where executing a program renders a video with the ground-truth option and interval of every question. A single policy plays both Questioner and Solver, proposing questions and answering them with timestamped evidence lines, and is trained on execution-grounded rewards instead of model consensus. The Solver's performance steers a curriculum over tasks and difficulty levels, and the policy selects rule changes that enrich the event dynamics of later rounds. Despite being trained solely on simple programs, without supervised fine-tuning or external video data which prior methods typically require, Tempo-Zero outperforms the base models and recent self-play baselines on average across twelve real-world video benchmarks at different model scales, with particularly strong gains on temporal grounding tasks. Project page: https://alchemist-jo.github.io/tempo-zero/
Figures & tables
Figure 1: Limitations of video self-play and program-generated video. (a) On a shell-game video, majority voting over eight sampled answers selects a wrong label over the two correct ones, while execution gives the correct label. (b) Under a fixed rule, all Solver rollouts are correct; changing the cushion rule and re-executing the program yields a new path with exact labels.
Figure 2: Overview of Tempo-Zero. Programs in library P execute and compile into video v , criteria C , and canonical question qˉ . One policy proposes questions as the Questioner and answers them as the Solver; bank B stores measured questions for replay, and execution-grounded rewards rQ and rS drive GRPO updates.
Figure 3: Environment compilation. Ground truth is recomputed on the sampled frames, and criteria are screened for temporal order. Colors identify entities, blue brackets mark ground-truth intervals, and the dark option is the ground-truth option.
Figure 4: Execution-verified self-play. The Questioner’s proposal is bound to criterion cq and passes gate g(q) . The Solver’s rollouts give pass rate p , which sets rlearn for the Questioner, and only an evidence line that states the criterion’s event earns interval credit.
Figure 5: Environment evolution. Pass rates guide allocation ωf,a (Eq. 6 ) and set level ℓ , and the policy selects a rule change, which the executor validates before deployment.
Table 1: Temporal grounding on real-world benchmarks. Metrics in %; Avg. averages the mIoU of the first three benchmarks and [email protected]. Bold marks the best per column within each backbone, and underline marks the second best at 4B and 8B, where all methods are evaluated; arrows give the change over the base model, the only comparison at 30B-A3B and Gemma.
Method
Long-Video Understanding
Video Reasoning
Avg.
LongVideo Bench
MLVU
Video-MME Long
LSDBench
LongVideo Reason
MMVU
VideoMath QA
VideoMMMU
Base model
57.70
62.60
51.60
54.68
69.90
64.32
19.05
57.56
54.68
VisPlay
59.20
63.50
53.20
54.83
69.60
63.36
18.33
57.44
54.93
V-Zero
59.50
63.50
51.90
54.68
68.20
64.48
20.95
58.33
55.19
Video-Zero
59.10
63.40
53.60
56.52
70.30
64.96
21.67
61.56
56.39
4B
Tempo-Zero
61.00 ↑ 3.30
67.36 ↑ 4.76
53.30 ↑ 1.70
56.06 ↑ 1.38
72.40 ↑ 2.50
66.08 ↑ 1.76
22.38 ↑ 3.33
63.11 ↑ 5.55
57.71 ↑ 3.03
Table 2: Long-video understanding and video reasoning. Official accuracy in %; marks as in Table 1 .
Variant
Temporal Grounding
Video QA g & Reasoning
Overall
Questioner
w/o novelty reward
36.53
56.88
50.10
w/o learnability reward
35.25
57.27
49.93
w/o Questioner
31.86
55.63
47.71
Solver
w/o evidence score
35.34
57.33
50.00
Table 3: Ablations on Qwen3-VL-4B. Temporal Grounding and Video QA & Reasoning are the Avg. columns of Tables 1 and 2 , and Overall averages all twelve benchmarks; marks as in Table 1 .
Figure 6: Training dynamics. (A) Held-out accuracy. (B) Evidence (filled) and grounded (hollow) scores; B is the base model. (C) Proposals passing the gate. (D) Families whose rules change for each round.
Figure 7: Execution labels. (A) Majority vote versus execution labels on 7,420 Solver groups (%). (B) Rollouts credited only on wrong-event lines, and credit kept by rgrd (%).
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Setting
Backbone
Qwen3-VL-4B/8B/30B-A3B-Instruct and Gemma-4-E4B-it; LoRA with rank 64 and α=128 on the q/k/v/o projections of the language model; the vision tower and the A3B experts carry no adapter
32 instances per step; playback scale λr=1.0,1.25,1.5,1.75,2.0 for rounds r=1,…,5
Questioner
4 proposals per instance; one to three located evidence lines
Solver
M=8 rollouts per question; one to eight evidence lines; criteria are those of the events the target rests on, or the target itself when it rests on none; at most 72 groups per step (32 canonical, up to 32 preselected proposals, 8 replayed)
Question bank
Capacity 640, shared across rounds; up to 8 replayed questions per step; at most one preselected proposal per instance; one prior example in the Questioner prompt
Appendix
Table 5: Training configuration. Settings shared by the 4B, 8B, 30B-A3B, and Gemma configurations.
Table 7: Twelve environment families. Each family supports three difficulty levels; the columns summarize its observable dynamics and the reasoning targets it supports.
Figure 8: Executable environments. Rendered frame pairs from Dominoes, Filling glasses, and Traffic lights; the labels inside the frames are part of the rendered video.
Figure 9: Training environments. (A) Key events and duration by level. (B) Static-question screen classes in %; only invariant criteria are removed.
Sampled frames
Events
Recomputation
Static-question screen
Level
Instances
Count
fps
between
Criteria
Disagree
Unverified
Ordered
Invariant
Undet.
0
2,599
58.1
1.69
2.82
25,338
23
25
67.93
10.76
21.31
1
1,179
63.8
1.00
10.02
11,623
44
44
74.33
7.29
18.38
2
1,296
64.0
0.62
28.56
12,564
141
126
76.21
7.02
16.77
Overall
5,074
60.9
1.26
16.60
49,525
208
195
71.51
9.01
19.49
Appendix
Table 8: Verification checks on the training environments. Instances per level, mean sampled frames and effective frame rate, share in % of recorded events that fall between two sampled frames, criteria compiled and discarded by the recomputation, and shares in % of the static-question screen classes among the criteria that pass the recomputation. Invariant criteria are removed, and ordered and undetermined criteria are kept.
Figure 10: A rule change in Billiards. Frames at 0.0, 4.0, 7.9, 11.9, 15.9, and 19.8 s of the same instance before and after the rule change; the overlays are part of the rendered frames.
Majority voting
Interval credit
Backbone
Groups
Wrong
Outvoted
Unanimous
Tie
Wrong line
Kept
4B
3,703
56.82
23.17
21.33
7.24
48.78
38.01
8B
3,717
48.21
29.32
8.34
10.36
52.21
36.44
Pooled
7,420
52.51
26.25
14.82
8.80
50.50
37.18
Appendix
Table 9: Majority voting and interval credit. Left: Solver groups, with shares in % of groups whose majority label is wrong, whose majority label is wrong although at least one rollout chooses the correct option, whose eight rollouts agree on a wrong option, and whose vote ties. Right: Solver rollouts, with the share in % of rollouts credited by the content-blind overlap whose credit falls only on lines with a non-matching statement, and the share in % of the content-blind credit that rgrd keeps.
Backbone
Metric
B
1
2
3
4
5
4B
Accuracy
22.9
27.9
32.7
32.4
33.2
31.1
Evidence
24.3
40.4
42.1
41.7
43.7
47.0
Grounded
3.1
7.6
8.9
7.1
10.0
10.8
8B
Accuracy
25.9
31.3
36.1
40.5
37.5
42.8
Evidence
27.2
38.3
40.3
40.9
40.7
42.6
Grounded
3.4
7.0
10.0
11.3
10.8
11.6
Appendix
Table 10: Held-out instances. Accuracy, evidence score, and grounded score in % on the 128 held-out instances with eight answers each, for the base model (B) and after each of the five rounds; bold marks the best round.
Changed
Kept
Backbone
2
3
4
5
Total
Selected
Failed check
Usable alternatives
4B
10
12
7
3
32
12
4
129 / 129
8B
9
7
3
5
24
23
1
141 / 141
Total
19
19
10
8
56
35
5
270 / 270
Appendix
Table 11: Rule decisions. Families whose rules change for each round, decisions that keep the rules by the policy’s selection or after a failed check, and menu alternatives with usable previews. Each backbone makes 12 decisions per round.
Variant
Change from the full method
Component tested
w/o novelty reward
Remove the novelty reward rnov from rQ
Diversity of proposed questions
w/o learnability reward
Remove the learnability reward rlearn from rQ
Targeting of questions at the Solver’s capability
w/o Questioner
Train the Solver only on canonical questions, without Questioner proposals
Self-proposed questions
w/o evidence score
Remove the evidence score revid from rS
Statement-level F1 of evidence lines
w/o grounded score
Remove the grounded score rgrd from rS
Interval credit on matched lines
Answer reward only
Keep only the answer reward rans in rS
Both evidence terms
Appendix
Table 12: Ablation variants. Each variant changes one component of the full method.
Self-evolution offers a promising path for improving reasoning models without relying on intensive human annotation. However, extending this paradigm to video understanding remains underexplored and challenging: videos are long, dynamic, and redundant, while the evidence needed for reasoning is often sparse and temporally localized. Naively generating difficult question-answer pairs from full videos can therefore produce supervision that appears challenging but is weakly grounded, relying on static cues or language priors rather than temporal evidence. In this work, we argue that the key bottleneck of video self-evolution is not difficulty alone, but grounding. We propose Video-Zero, an annotation-free Questioner--Solver co-evolution framework that centers self-evolution on temporally localized evidence. The Questioner discovers informative evidence segments and generates evidence-grounded questions, while the Solver learns to answer and align its predictions with the supporting evidence. This closes an iterative loop of evidence discovery, grounded supervision, and evidence-aligned learning. Across 13 benchmarks spanning temporal grounding, long-video understanding, and video reasoning, Video-Zero consistently improves multiple video VLM backbones, demonstrating the effectiveness and transferability of evidence-centered self-evolution.
Ruixu Zhang, Deyi Ji, Lanyun Zhu +4
1Tsinghua University · 2Tencent · 3Tongji University +1
Recent Video Large Language Models (Video-LLMs) have demonstrated strong capabilities in video reasoning through reinforcement learning (RL). However, existing RL pipelines rely heavily on human-annotated tasks and solutions, making them costly to scale and fundamentally constrained by human expertise. Self-evolving frameworks have recently emerged as a promising alternative through autonomous Questioner-Solver self-play. Unfortunately, these approaches are primarily designed for static modalities such as text and images, fundamentally failing to capture the temporal dynamics that are central to video reasoning. In this work, we propose EvoVid, a temporal-centric self-evolving framework that enables Video-LLMs to improve directly from raw, unannotated videos. Specifically, we introduce two complementary temporal-centric rewards: a temporal-aware Questioner reward that encourages temporally dependent question generation through temporal perturbation sensitivity, and a temporal-grounded Solver reward that provides automatic temporal supervision via inherent video segment localization. Extensive experiments across four base models and six benchmarks demonstrate consistent improvements over both base models and existing self-evolving baselines, achieving competitive performance with supervised methods. These results highlight temporal-centric self-evolution as an effective and scalable paradigm for video understanding and reasoning.
Shiqi Huang, Ziyue Wang, Zhongrong Zuo +3
School of Electrical and Electronic Engineering, Nanyang Technological University · ByteDance
Video reasoning requires models to identify and verify temporally localized evidence within long video sequences. Recent Video Large Language Models (Video-LLMs) have shown promising reasoning abilities when aligned with reinforcement learning, yet existing approaches typically rely on outcome-based rewards that supervise only the final prediction. Such supervision provides limited guidance on how models should discover the relevant temporal evidence during intermediate reasoning. In this work, we propose TimeThink, a reinforcement learning framework that explicitly guides temporal evidence discovery in Video-LLMs. Our key idea is to treat temporal clue steps as the fundamental optimization primitive of video reasoning, where each reasoning step references a candidate time interval in the video. We introduce a step-wise temporal process reward that provides localized credit assignment for these clues and a joint process--outcome optimization objective that balances reasoning fidelity with task correctness. To enable scalable training, we construct TimeThink-RFT-20K, a dataset with automatically derived temporal evidence segments. Extensive experiments across video reasoning, temporal grounding, and general video understanding benchmarks show that TimeThink consistently improves both temporal localization and reasoning performance, achieving state-of-the-art results among open-source video RL models.
Handong Li, Longteng Guo, Zikang Liu +8
School of Artificial Intelligence, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.