Wan-R1: Verifiable-Reinforcement Learning for Video Reasoning

Ming Liu; Yunbei Zhang; Shilong Liu; Liwen Wang; Wensheng Zhang

cs.CVMar 29, 2026

Wan-R1: Verifiable-Reinforcement Learning for Video Reasoning

Authors: Ming Liu, Yunbei Zhang, Shilong Liu, Liwen Wang, Wensheng Zhang

Organizations: Iowa State University, Ames, IA 50011, USA · Tulane University, New Orleans, LA 70118, USA · Princeton University, Princeton, NJ 08544, USA

Abstract

Video generation models produce visually coherent content but struggle with tasks requiring spatial reasoning and multi-step planning. Reinforcement learning (RL) offers a path to improve generalization, but its effectiveness in video reasoning hinges on reward design -- a challenge that has received little systematic study. We investigate this problem by adapting Group Relative Policy Optimization (GRPO) to flow-based video models and training them on maze-solving and robotic navigation tasks. We first show that multimodal reward models fail catastrophically in this setting. To address this, we design verifiable reward functions grounded in objective task metrics. For structured game environments, we introduce a multi-component trajectory reward. For robotic navigation, we propose an embedding-level verifiable reward. Our experiments show that RL fine-tuning with verifiable rewards improves generalization. For example, on complex 3D mazes, our model improves exact match accuracy by 29.1% over the SFT baseline, and on trap-avoidance tasks by 51.4%. Our systematic reward analysis reveals that verifiable rewards are critical for stable training, while multimodal reward models could lead to degenerate solutions. These findings establish verifiable reward design as a key enabler for robust video reasoning. Code will be publicly available.

Figures & tables

Figure 1: VLLM fails to provide reliable reward, verified rewards can guarantee correct supervision.

Figure 2: Flow-GRPO with verified reward.

Method		EM ( $\uparrow$ )					SR ( $\uparrow$ )					PR ( $\uparrow$ )					SD ( $\downarrow$ )
Method		Base	Irreg	Trap	3D	Soko	Base	Irreg	Trap	3D	Soko	Base	Irreg	Trap	3D	Soko	Base	Irreg	Trap	3D	Soko
General Video Model	Closed-Source
	Veo-3.1-fast	0.0	0.0	0.0	0.0	2.8	40.3	36.1	38.9	48.6	43.1	20.2	24.8	28.2	13.4	21.7	195.3	111.5	80.7	33.5	112.3
	Veo-3.1-pro	0.0	4.2	1.4	0.0	0.0	47.2	36.1	59.7	50.0	37.5	24.6	33.9	39.1	18.0	21.4	140.7	94.5	85.4	40.1	141.8
	Sora-2	1.4	5.6	0.0	0.0	4.2	75.0	72.2	83.0	37.5	43.1	45.1	45.7	46.6	19.3	27.4	302.9	187.0	145.1	92.4	138.7
	kling-v1	0.0	0.0	0.0	0.0	0.0	2.8	0.0	1.4	27.8	12.5	6.3	8.8	10.4	11.7	9.0	25.2	–	–	69.7	356.1

Table 1: VR-Bench comprises five tasks: Base (Regular Maze), Irreg (Irregular Maze), Trap (TrapField), 3D (3D Maze), and Soko (Sokoban). Last row shows the performance gain compared with Wan2.2-TI2V base model.

Task

Base

0.0

8.3

4.2

13.2

3.8

2.7

154.8

–

0.0 ( +0.0 )

4.2 ( +4.2 )

0.0 ( +0.0 )

83.3 ( +75.0 )

41.7 ( +33.4 )

58.3 ( +54.1 )

40.1 ( +26.9 )

26.1 ( +22.3 )

6.0 ( +3.3 )

28.2 ( -126.6 )

10.1 ( – )

– ( – )

87.5 ( +87.5 )

66.7 ( +66.7 )

29.2 ( +29.2 )

100.0 ( +91.7 )

87.5 ( +79.2 )

41.7 ( +37.5 )

97.0 ( +83.8 )

85.4 ( +81.6 )

57.2 ( +54.5 )

1.4 ( -153.4 )

2.5 ( – )

14.6 ( – )

Irrg

0.0

29.2

4.2

13.4

8.1

5.8

48.3

–

39.3

83.3 ( +83.3 )

66.7 ( +66.7 )

54.2 ( +54.2 )

95.8 ( +66.6 )

87.5 ( +83.3 )

62.5 ( +58.3 )

88.0 ( +74.6 )

74.8 ( +66.7 )

68.4 ( +62.6 )

3.5 ( -44.8 )

7.8 ( – )

3.1 ( -36.2 )

Table 2: Difficulty generalization performance of Wan-R1 on VR-Bench. Each task block presents a comparison between the baseline model (Wan2.2-TI2V-5B) and Wan-SFT (trained exclusively on Easy-level data) and Wan-R1 (trained on easy tasks with flow-GRPO) evaluated across three difficulty levels (E/M/H = Easy/Medium/Hard).

Figure 3: Target-Bench results. Wan-R1 achieves the highest overall score, with lower displacement errors (ADE, FDE) and miss rate, and higher soft endpoint (SE) and approach consistency (AC) compared to both the base model and Wan-SFT.

Table 3: Ablation studies on KL regularization and denoising reduction.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Group size	$G=8$
LoRA rank	$r=32$
Number of frames	193
Training Epoch	$T_{\text{train}}=1$
Inference denoising steps	$S_{\text{infer}}=50$
KL coefficient	$\beta_{KL}=0.04$

Appendix

Table 4: Training Hyperparameters

Task

Model

Base

Irreg

Soko

Base

Irreg

Soko

Base

Irreg

Soko

Base

Irreg

Soko

Wan2.2-TI2V-5B Baseline

0.0

6.9

12.5

31.9

11.1

6.6

9.1

12.8

9.2

388.7

66.1

5.4

176.6

Regular Maze

Wan-SFT

33.3 (+33.3)

5.6 (+5.6)

0.0 (+0.0)

76.4 (+69.5)

8.3 (-4.2)

69.4 (+37.5)

30.6 (+19.5)

60.6 (+54.0)

22.7 (+13.6)

13.7 (+0.9)

19.0 (+9.8)

10.3 (-378.4)

51.7 (-14.4)

12.7 (+7.3)

49.3 (-127.3)

Wan-R1

61.1 (+61.1)

4.2 (+4.2)

0.0 (+0.0)

75.0 (+68.1)

13.7 (+1.2)

63.9 (+32.0)

15.3 (+4.2)

78.6 (+72.0)

23.9 (+14.8)

13.1 (+0.3)

20.6 (+11.4)

5.2 (-383.5)

27.1 (-39.0)

10.9 (+5.5)

9.0 (-167.6)

Irregular Maze

Wan-SFT

0.0 (+0.0)

56.9 (+56.9)

0.0 (+0.0)

11.1 (+4.2)

69.4 (+56.9)

79.2 (+47.3)

12.5 (+1.4)

16.6 (+10.0)

71.6 (+62.5)

16.8 (+4.0)

15.5 (+6.3)

35.8 (-352.9)

2.4 (-63.7)

9.0 (+3.6)

40.7 (-135.9)

Wan-R1

0.0 (+0.0)

47.9 (+47.9)

0.0 (+0.0)

9.7 (+2.8)

69.7 (+57.2)

83.3 (+51.4)

16.7 (+5.6)

11.6 (+5.0)

68.4 (+59.3)

15.1 (+2.3)

16.9 (+7.7)

0.0 (-388.7)

8.3 (-57.8)

7.5 (+2.1)

9.3 (-167.3)

Appendix

Table 5: Performance comparison between the baseline model (Wan2.2-TI2V-5B), task-specific Wan-SFT models, and our method across four game types: Base, Irreg, 3D, and Soko. Each cell displays the absolute performance alongside the relative change from the baseline across four metrics (EM, SR, PR, SD).

Task		EM			SR			PR			SD
		Raw	Sk2	Sk3	Raw	Sk2	Sk3	Raw	Sk2	Sk3	Raw	Sk2	Sk3
Base	Base	0.0	0.0	0.0	6.9	4.2	2.8	6.6	4.6	9.4	388.7	11.7	14.9
	SFT	33.3	1.4	23.6	76.4	38.9	68.1	60.6	12.3	51.0	10.3	19.0	9.0
	R1	61.1	31.9	31.9	75.0	70.8	70.8	78.6	52.5	52.5	5.2	27.1	27.1
	$\Delta$	+61.1	+31.9	+31.9	+68.1	+66.6	+68.0	+72.0	+47.9	+43.1	-383.5	+15.4	+12.2
Irreg	Base	0.0	0.0	0.0	12.5	4.2	2.1	9.1	12.0	47.8	66.1	39.5	42.0

Appendix

Table 6: Texture generalization performance across different visual skins. For each task, we report the baseline (Base), Wan-SFT , and Wan-R1 across three texture conditions (Raw, Sk2=Skin2, Sk3=Skin3), with

\Delta

indicating Wan-R1 improvement over baseline. See texture sample in Figure 7 .

Method		MF ( $\uparrow$ )
Method		Base	Irreg	Trap	3D	Soko
General Video Model	Closed-Source
	Veo-3.1-fast	43.2	86.3	22.5	69.1	63.5
	Veo-3.1-pro	80.5	89.3	82.2	73.4	95.8
	Sora-2	96.5	97.2	97.2	95.4	95.2
	kling-v1	55.5	73.8	54.4	84.9	72.9

Appendix

Table 7: MF denotes Maze Fidelity.

Table 8: Ablation studies on cold start (left) and reward models (right).

Figure 4: Test-time scaling performance of Wan-R1 on Irregular Maze tasks. Performance is evaluated across varying numbers of samples (

K\in\{1,4,8,12,16\}

) and three difficulty levels (Easy, Medium, Hard). RL improves base capability (K=1) while maintaining beneficial scaling with increased sampling.

Figure 5: Given the video output along with the evaluation prompt describing maze-solving rules, Qwen2.5-VL-7B-Instruct produces a step-by-step analysis and assigns a reward score. Although the generated video exhibits visible glitches and degraded visual quality, the model assigns a perfect score of 1.0. This illustrates the core vulnerability of VLM-based reward models: rather than detecting actual visual artifacts, the model hallucinates quality based on high-level semantic cues—a visible agent, a recognizable maze structure, and apparent goal-directed motion—and incorrectly concludes that the video contains “no glitches, noise, or artifacts.”

Figure 6: Trajectory tracking across five maze types (This figure is from [ 45 ] ). Green: ground-truth trajectory; Blue: tracked trajectory from generated video. Columns show temporally ordered frames illustrating agent motion over time.

Figure 7: Overview of VR-Bench (This figure is from VR-Bench [ 45 ] ) (A) Maze Types: Five maze types—Regular, Irregular, 3D, Trapfield, and Sokoban—spanning 2D/3D settings and diverse task structures. (B) Reasoning Paradigm: Chain-of-frame reasoning [ 40 ] requiring frame-by-frame inferences for sequential visual reasoning. (C) Benchmark Performance: Leading VLMs evaluated on four metrics across all maze types. (D) Additional Analysis: Evaluations on difficulty, texture, and maze-type generalization, plus test-time scaling.

Figure 8: Overview of Target-Bench (This figure is from Target-Bench [ 37 ] ). Target-Bench introduces a quadruped-robot dataset and benchmark for assessing world models in mapless navigation toward text-specified goals with implicit semantic content. Given a single camera frame and a natural language description of the desired target state, world models must synthesize a future video depicting the trajectory to that goal. A world decoder then recovers the planned path from the generated video, which is evaluated against ground-truth maneuvers collected from a human-operated quadruped.

Explore similar work

Video Models Can Reason with Verifiable Rewards
May 14, 2026Tinghui Zhu, Sheng Zhang, James Y. Huang +5Video Diffusion Models RL for Video Generation
University of California, Davis · Microsoft Research · University of Southern California +1
Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning
Apr 6, 2026Songyuan Yang, Weijiang Yu, Jilin Ma +5Video Reasoning VLM Inference
National University of Defense Technology · Sun Yat-sen University
VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Prompting
Mar 15, 2026Daeun Lee, Shoubin Yu, Yue Zhang +1Video Reasoning Temporal Video Grounding
University of North Carolina, Chapel Hill, USA

May 14, 2026J/K move · Enter open · S save

Video Models Can Reason with Verifiable Rewards

Tinghui Zhu, Sheng Zhang, James Y. Huang, Selena Song+4

University of California, Davis · Microsoft Research · University of Southern California+1

Video Diffusion Models RL for Video Generation Video Reasoning Reinforcement Learning with Verifiable Rewards RL for Diffusion Models

Open paper AskPDF arXiv

Video diffusion models have made rapid progress in perceptual realism and temporal coherence, but they remain primarily optimized for plausible generation rather than verifiable reasoning. This limitation is especially pronounced in tasks where generated videos must satisfy explicit spatial, temporal, or logical constraints. Inspired by the role of reinforcement learning with verifiable rewards (RLVR) in reasoning-oriented language models, we introduce VideoRLVR, a practical recipe for optimizing video diffusion models with rule-based feedback. VideoRLVR formulates video reasoning as the generation of verifiable visual trajectories and consists of an SDE-GRPO optimization backbone, dense decomposed rewards, and an Early-Step Focus strategy for efficient training. The Early-Step Focus strategy restricts policy optimization to the early denoising phase, reducing training latency by about 40% while preserving performance. We evaluate VideoRLVR on Maze, FlowFree, and Sokoban, three procedurally generated domains with objective success criteria. Across these tasks, VideoRLVR consistently improves over supervised fine-tuning baselines, with dense decomposed rewards proving especially important in low-success-rate settings. Our RL-optimized model also outperforms the evaluated proprietary and open-source video generation models on these verifiable reasoning benchmarks and out-of-domain benchmarks. These results suggest that verifiable RL can move video models beyond perceptual imitation toward more reliable rule-consistent visual reasoning.

Abstract

Explore similar work

Video Models Can Reason with Verifiable Rewards

Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning

VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Prompting