The transition from image to video understanding requires vision-language models (VLMs) to shift from recognizing static patterns to reasoning over temporal dynamics such as motion trajectories, speed changes, and state transitions. Yet current post-training methods fall short due to two critical limitations: (1) existing datasets often lack temporal-centricity, where answers can be inferred from isolated keyframes rather than requiring holistic temporal integration; and (2) training data generated by proprietary models contains systematic errors in fundamental temporal perception, such as confusing motion directions or misjudging speeds. We introduce SynRL, a post-training framework that teaches models temporal primitives, the fundamental building blocks of temporal understanding including direction, speed, and state tracking. Our key insight is that these abstract primitives, learned from programmatically generated synthetic videos, transfer effectively to real-world scenarios. We decompose temporal understanding into short-term perceptual primitives (speed, direction) and long-term cognitive primitives, constructing 7.7K CoT and 7K RL samples with ground-truth frame-level annotations through code-based video generation. Despite training on simple geometric shapes, SynRL achieves substantial improvements across 15 benchmarks spanning temporal grounding, complex reasoning, and general video understanding. Remarkably, our 7.7K synthetic CoT samples outperform Video-R1 with 165K real-world samples. We attribute this to fundamental temporal skills, such as tracking frame by frame changes and comparing velocity, that transfer effectively from abstract synthetic patterns to complex real-world scenarios. This establishes a new paradigm for video post-training: video temporal learning through carefully designed synthetic data provides a more cost efficient scaling path.
Figures & tables
Figure 1: Systematic failures in proprietary models’ temporal perception. ( a ) Gemini-2.5-Pro incorrectly describes a simple geometric shape’s motion trajectory. ( b ) When used to annotate real-world videos, such flawed descriptions propagate errors into training data. ( Erroneous phrases in red. )
Figure 2: Overview of SynRL framework. ( Left ) Programmatic Generation Pipeline: Object properties and motion dynamics are specified in Python code to generate videos with accurate frame-level metadata. Videos are rendered at 30 FPS, and QA pairs are instantiated from hand-crafted templates conditioned on the same metadata, yielding temporally grounded synthetic video-QA triples. ( Middle ) Chain-of-Thought Augmentation: Given a synthetic video, its metadata, and its QA pair, a multimodal LLM generates step-by-step reasoning chains. A Judger verifies their consistency with the event timeline, filtering out incorrect outputs. The verified reasoning chains are then passed to a Polisher, which refines them into more natural and fluent CoT annotations while preserving factual correctness. ( Right ) Training Strategy: The target model is first trained via supervised fine-tuning on the polished CoT data to learn explicit temporal reasoning patterns. It is then optimized with group relative policy optimization (GRPO) on synthetic video-QA samples with verifiable rewards, so that reinforcement learning further improves reasoning quality under strictly correct supervisory signals.
Figure 3: (a) Training Examples: We generate diverse synthetic videos spanning 8 major categories with 18 subcategories, covering short-term perceptual primitives (speed perception, motion tracking, direction identification) and long-term cognitive primitives (grid-based object tracking, symbol manipulation, code execution, mathematical operations, container management). Each video is procedurally generated with verifiable ground-truth answers. (b) Constructing CoT using Meta Information: Using a grid-based object tracking game as an example, we provide the video with its code-derived metadata (initial state, swap events, timestamps) to a VLM. The model generates step-by-step reasoning chains that track each object through every swap operation, explicitly referencing frame timestamps (e.g., “at 00:00”, “at 00:02”) and logging state transitions. This metadata-conditioned generation ensures temporal grounding and verifiable correctness, as the reasoning must align with the documented event sequence.
Complex Reasoning
Comprehensive Video Understanding
Models
TOMATO
Video-TT
MVBench
VideoMME
TemporalBench
Binary Acc.
Multiple Binary Acc.
LLaVA-OneVision-7B
25.5
-
58.6
63.3
61.9
21.2
NVILA-8B
-
-
68.1
64.2
-
-
Apollo-7B
-
-
-
61.3
-
-
VideoLLaMA-2-7B
10.1
-
57.3
54.9
-
-
Table 1: Performance comparison on diverse video benchmarks.
Benchmark
Qwen3-VL-4B
Qwen3-VL-4B + SynRL
Qwen3-VL-8B
Qwen3-VL-8B + SynRL
General Video QA
Aotbench
52.7
54.4 ↑ 1.7
54.8
57.7 ↑ 2.9
vinoground
40.8
43.2 ↑ 2.4
43.4
47.6 ↑ 4.2
VideoMMMU
54.1
54.5 ↑ 0.4
60.0
61.0 ↑ 1.0
Complex Reasoning
vcrbench
30.1
31.5 ↑ 1.4
34.2
35.6 ↑ 1.4
Table 2: Performance comparison on additional video benchmarks.
Table 3: Performance comparison on NExTGQA and RexTime benchmarks. It can be observed that SynRL outperforms existing SFT-based methods trained with large-scale data.
Table 4: Performance comparison on Charades dataset
Figure 4: mIoP Performance comparison on NExTGQA and RexTime benchmarks.
Training Strategy
Performance
Base
SFT
RL
vinoground
TOMATO
MVBench
TVBench
✓
✗
✗
40.8
32.1
65.4
51.9
✓
✗
✓
41.2 ↑ 0.4
32.8 ↑ 0.7
65.8 ↑ 0.4
52.1 ↑ 0.2
✓
✓
✗
42.6 ↑ 1.8
34.2 ↑ 2.1
66.1 ↑ 0.7
52.8 ↑ 0.9
✓
✓
✓
43.4 ↑ 2.6
36.9 ↑ 4.8
66.8 ↑ 1.4
53.7 ↑ 1.8
Table 5: Ablation study on the necessity of cold-start SFT before RL training. Models without cold-start fail to effectively learn from our challenging synthetic videos.
Benchmark
Base Model
Video-R1 CoT-165K
SynRL-7.7K (Ours)
vinoground
40.8
36.6 ↓ 4.2
42.6 ↑ 1.8
TOMATO
32.1
28.2 ↓ 3.9
34.2 ↑ 2.1
MVBench
65.4
64.7 ↓ 0.7
66.1 ↑ 0.7
TVBench
51.9
51.2 ↓ 0.7
52.8 ↑ 0.9
Charades
41.9
41.4 ↓ 0.5
43.4 ↑ 1.5
TB. BA.
67.4
63.7 ↓ 3.7
67.9 ↑ 0.5
Table 6: Quality versus quantity in cold-start data. Our 7.7K synthetic CoT samples outperform Video-R1’s 165K model-generated data across all benchmarks, demonstrating that correctness matters more than scale. TB. denotes TemporalBench.
Figure 5: Real-world motion understanding examples. The base model fails on direction recognition and speed perception, while the SynRL-trained model answers both correctly, indicating that temporal skills acquired from synthetic videos can transfer to real-world video understanding.
Model
Video-MME
AoTBench
Minerva
TOMATO
Base Model
60.9
52.7
32.5
32.1
+ SynRL (1st round)
62.0 ↑ 1.1
54.4 ↑ 1.7
32.8 ↑ 0.3
36.7 ↑ 4.6
+ + SynRL (2nd round)
62.4 ↑ 0.4
54.9 ↑ 0.5
33.5 ↑ 0.7
37.0 ↑ 0.3
Table 7: Performance improvements through iterative RL training with increasingly challenging synthetic videos.
Figure 6: Scaling analysis
Model
Complex Reas.
Comp. VU
Gen. VideoQA
Qwen3-VL-4B (Base)
35.5
64.6
49.2
+ VideoR1 Only
32.4 − 3.1
62.7 − 1.9
47.6 − 1.6
+ Synthetic Only
39.2 +3.7
65.6 +1.0
49.9 +0.7
+ Mixed
38.7 +3.2
65.8 +1.2
50.7 +1.5
Table 8: Comparison of synthetic vs. real-world training data across three evaluation categories. VideoR1 represents real-world data with model-generated annotations.
Training Data
TOM.
MV.-action
MV.-move
VE.-short
VE.-long
TB-action
TB-motion
VSI
Base Model
32.1
66.3
70.1
71.7
51.1
63.1
64.2
35.0
+ Spatial
33.8 +1.7
67.1 +0.8
74.3 +4.2
72.3 +0.6
51.8 +0.7
64.2 +1.1
65.1 +0.9
38.4 +3.4
+ Temporal
35.9 +3.8
68.7 +2.4
71.2 +1.1
72.1 +0.4
53.6 +2.5
67.4 +4.3
65.8 +1.6
35.9 +0.9
+ Short-term
33.2 +1.1
67.8 +1.5
72.5 +2.4
73.1 +1.4
51.5 +0.4
65.6 +2.5
67.2 +3.0
36.7 +1.7
+ Long-term
36.2 +4.1
67.2 +0.9
70.8 +0.7
71.9 +0.2
54.1 +3.0
64.8 +1.7
64.5 +0.3
36.1 +1.1
+ Full SynRL
36.7 +4.6
69.1 +2.8
73.8 +3.7
73.4 +1.7
54.5 +3.4
67.1 +4.0
67.8 +3.6
38.2 +3.2
Table 9: Per-category ablation study showing capability-specific transfer patterns of each synthetic primitive type. TOM. = TOMATO; MV. = MVBench; VE. = VideoMME; TB = TemporalBench; VSI = VSIBench. Best results in each column are in bold .
Model
Pass@1
Pass@4
Complex
Comp. VU
Gen. VQA
VideoMME
Complex
Comp. VU
Gen. VQA
Base Model
35.5
64.6
49.2
58.2
37.8
66.3
51.5
+ SynRL
38.7 +3.2
65.8 +1.2
50.7 +1.5
60.5 +2.3
42.6 +4.8
68.4 +3.1
53.4 +1.9
Table 10: Pass@ k performance comparison ( k∈{1,4} ). SynRL shows a larger gap between Pass@1 and Pass@4, indicating a higher reasoning potential ceiling.
Model
Complex Reas.
Comp. VU
Gen. VideoQA
Qwen3-VL-4B (Base)
35.5
64.6
49.2
+ CoT + RL (7K)
38.7 +3.2
65.8 +1.2
50.7 +1.5
+ Only RL (7K)
37.1 +1.6
65.2 +0.6
50.0 +0.8
+ Only RL (15K)
39.0 +3.5
65.6 +1.0
50.9 +1.7
Table 11: Ablation study on Chain-of-Thought (CoT) supervision. CoT warm-up accelerates RL learning but models still benefit from synthetic videos without CoT supervision when given more data.
Video reasoning requires models to identify and verify temporally localized evidence within long video sequences. Recent Video Large Language Models (Video-LLMs) have shown promising reasoning abilities when aligned with reinforcement learning, yet existing approaches typically rely on outcome-based rewards that supervise only the final prediction. Such supervision provides limited guidance on how models should discover the relevant temporal evidence during intermediate reasoning. In this work, we propose TimeThink, a reinforcement learning framework that explicitly guides temporal evidence discovery in Video-LLMs. Our key idea is to treat temporal clue steps as the fundamental optimization primitive of video reasoning, where each reasoning step references a candidate time interval in the video. We introduce a step-wise temporal process reward that provides localized credit assignment for these clues and a joint process--outcome optimization objective that balances reasoning fidelity with task correctness. To enable scalable training, we construct TimeThink-RFT-20K, a dataset with automatically derived temporal evidence segments. Extensive experiments across video reasoning, temporal grounding, and general video understanding benchmarks show that TimeThink consistently improves both temporal localization and reasoning performance, achieving state-of-the-art results among open-source video RL models.
Handong Li, Longteng Guo, Zikang Liu +8
School of Artificial Intelligence, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.
Spatio-temporal reasoning is a core capability for Multimodal Large Language Models (MLLMs) operating in the real world. As such, evaluating it precisely has become an essential challenge. However, existing spatio-temporal reasoning benchmark datasets primarily rely on static image sets or passively curated video data, which limits the evaluation of fine-grained reasoning capabilities. In this paper, we introduce VGenST-Bench, a video benchmark that employs generative models to actively synthesize highly controlled and diverse evaluation scenarios. To construct VGenST-Bench, we propose a multi-agent pipeline incorporating a human quality control stage, ensuring the quality of all generated videos and QA pairs. We establish a comprehensive 3x2x2 video taxonomy, encompassing Spatial Scale, Perspective, and Scene Dynamics to span diverse scenarios. Furthermore, we design a hierarchical task suite that decouples low-level visual perception from high-level spatio-temporal reasoning. By shifting the paradigm from passive curation to active synthesis, VGenST-Bench enables fine-grained diagnosis of spatio-temporal understanding in MLLMs.
Jinho Park, Youbin Kim, Hogun Park +1
Department of Artificial Intelligence, Sungkyunkwan University · Department of Artificial Intelligence, Yonsei University
Video reasoning requires models to locate and track question-relevant evidence across frames. While reinforcement learning (RL) with verifiable rewards improves accuracy, it still struggles to achieve reliable spatio-temporal grounding during the reasoning process. Moreover, improving grounding typically relies on scaled training data or inference-time perception tools, which increases annotation cost or computational cost. To address this challenge, we propose VisonCoach, an input-adaptive RL framework that improves spatio-temporal grounding through visual prompting as training-time guidance. During RL training, visual prompts are selectively applied to challenging inputs to amplify question-relevant evidence and suppress distractors. The model then internalizes these improvements through self-distillation, enabling grounded reasoning directly on raw videos without visual prompting at inference. VisonCoach consists of two components: (1) Visual Prompt Selector, which predicts appropriate prompt types conditioned on the video and question, and (2) Spatio-Temporal Reasoner, optimized with RL under visual prompt guidance and object-aware grounding rewards that enforce object identity consistency and multi-region bounding-box overlap. Extensive experiments demonstrate that VisonCoach achieves state-of-the-art performance under comparable settings, across diverse video reasoning, video understanding, and temporal grounding benchmarks (V-STAR, VideoMME, World-Sense, VideoMMMU, PerceptionTest, and Charades-STA), while maintaining a single efficient inference pathway without external tools. Our results show that visual prompting during training improves grounded video reasoning, while self-distillation enables the model to internalize this ability without requiring prompts at inference time.