SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agents

Yu Yang; Yue Liao; Jianbiao Mei; Baisen Wang; Xuemeng Yang; Licheng Wen; Jiangning Zhang; Xiangtai Li

cs.CVMar 9, 2026

SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agents

Authors: Yu Yang, Yue Liao, Jianbiao Mei, Baisen Wang, Xuemeng Yang, Licheng Wen, Jiangning Zhang, Xiangtai Li,

Organizations: Zhejiang University · KnowledgeXLab at Shanghai AI Lab · National University of Singapore · Chinese Academy of Sciences ·

Abstract

Long-horizon action-conditioned video generation aims to synthesize temporally coherent videos that follow complex action instructions over extended horizons, requiring procedural ordering, persistent action execution, and scene consistency beyond conventional TI2V's short-term fidelity. Existing single-shot video generation models typically operate in an open-loop manner, leading to incomplete action execution, hallucinated motions, and temporal drift. To address this, we propose SPIRAL, a closed-loop framework that performs sequential planning and iterative reflection for action-conditioned long-horizon video generation. Specifically, SPIRAL instantiates a think-act-reflect process: a PlanAgent decomposes high-level goals into sub-actions, which condition a VideoGenerator to synthesize each segment alongside a memory context, while a CriticAgent evaluates intermediate video segments to provide corrective feedback for iterative refinement. This closed-loop design further supports self-evolution by utilizing PlanAgent-proposed actions and CriticAgent-derived rewards for GRPO-based post-training to enhance the video generator's long-horizon consistency. Moreover, we introduce ActVideoGen-Dataset for task-specific training, and establish ActVideoGen-Bench as a dedicated evaluation suite for measuring action quality and temporal coherence. Experiments across multiple TI2V backbones alongside the self-evolving strategy show consistent gains on ActVideoGen-Bench and VBench, demonstrating the effectiveness of SPIRAL.

Figures & tables

Figure 2: S P I R A L Overview. (a) Closed-Loop Framework : The PlanAgent decomposes abstract goals into atomic plans for action-conditioned video generation, while the CriticAgent evaluates action-video alignments and triggers dual-level (inner/outer) feedback for refinement (Fig. 11 ). (b) Self-Evolving via GRPO : Guided by the PlanAgent, the VideoGenerator produces video rollouts, using CriticAgent rewards for policy optimization (Fig. 12 ).

Figure 3: Overview of ActVideoGen-Dataset. (a) A structured data example comprising goal, CoT, and step-wise video-action-critic tuples. (b-f) Detailed dataset distributions across diversity, temporal scale, and action complexity.

Table 6

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	SFT	DPO	Description
Optimization & Training Strategy
Base model	Qwen3-VL-8B-Thinking		Backbone vision-language model
Learning rate	$1.0\times 10^{-4}$	$5.0\times 10^{-6}$	Peak learning rate for AdamW
LR scheduler	Cosine	Cosine	Decay schedule type
Warmup ratio	0.1	0.1	Ratio of warmup steps
Max epochs	3.0	3.0	Number of training epochs

Appendix

Table 7: Hyperparameter settings for PlanAgent training . Parameters are differentiated by two training phases: Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO).

Parameter	SFT	RM	Description
Optimization & Training Strategy
Base model	Qwen3-VL-8B-Instruct		Backbone vision-language model
Learning rate	$1.0\times 10^{-4}$	$5.0\times 10^{-6}$	Peak learning rate for AdamW
LR scheduler	Cosine	Cosine	Decay schedule type
Warmup ratio	0.1	0.1	Ratio of warmup steps
Max epochs	3.0	1.0	Number of training epochs

Appendix

Table 8: Hyperparameter settings for CriticAgent training . Parameters are differentiated by the two training phases: Supervised Fine-Tuning (SFT) and Pairwise Reward Modeling (RM).

Figure 11: Closed-Loop Think-Act-Reflect Pipeline. PlanAgent decomposes the goal and reference context into atomic action plans, VideoGenerator executes each step into video segments, and CriticAgent verifies action-video alignment. Successful steps are stored in ContextMemory, while critic feedback triggers local refinement or global replanning for closed-loop inference-time correction .

Figure 12: Training-Time Self-Evolving via GRPO. PlanAgent samples action steps, VideoGenerator produces group rollouts, and CriticAgent assigns rewards for GRPO-based policy optimization. A curriculum gradually increases task complexity from atomic actions to long-horizon procedures, enabling VideoGenerator to internalize planning and verification signals for progressive-evolution .

Figure 13: Detailed annotation example from ActVideoGen-Dataset. A procedural task is represented by a global goal and CoT planning rationale, then decomposed into temporally grounded action-video pairs. Each step contains an action instruction, an object-tool tuple, and pre-/post-conditions, providing structured supervision for long-horizon action planning and video generation.

Figure 14: Case studies of our multi-agent evaluator. The evaluator grounds step-wise judgments in action decompositions and visual evidence, providing multi-dimensional assessments of action quality. Successful executions satisfy post-conditions, while failed actions reveal implausible motions.

Table 13

Setting	Model	Inference Cost		Action Quality
Setting	Model	Avg. Time (s)	Memory (GB)	Action Completeness	Action Smoothness	Object Interaction	Physical Fidelity	Overall Quality
Single-Shot Baseline	LongLive-2B [ 14 ]	8	28	0.273	0.646	0.776	0.808	0.626
PlanAgent (Ours)	Qwen3-VL-8B [ 24 ]	4	19	-	-	-	-	-
VideoGenerator (Per-step)	LongLive-2B [ 14 ]	8	28	-	-	-	-	-
CriticAgent (Per-step)	Qwen3-VL-8B [ 24 ]	7	22	-	-	-	-	-
SPIRAL (T=3, Retry=1)	Plan+Gen+Critic	49	69	0.721 +44.8%	0.902 +25.6%	0.860 +8.4%	0.924 +11.6%	0.852 +22.6%

Appendix

Table 11: Inference Cost and Generation Quality Analysis. We report average inference time, memory usage, and action quality. Experiments are conducted on the same NVIDIA A100 GPUs.

Figure 15: End-to-end pipeline of SPIRAL. Given a user goal, PlanAgent first decomposes the task into step-wise actions with explicit pre-/post-conditions. At each step, VideoGenerator synthesizes a video segment, while CriticAgent evaluates action-video alignment and returns judgment with feedback. Verified segments are sequentially accumulated to compose the final long-horizon video.

Figure 16: Closed-loop feedback refinement. For the soccer procedure, CriticAgent detects that the Step 2 first generation bypasses the defender but loses ball control, violating the planned post-condition. SPIRAL uses this feedback to locally refine the action instruction, regenerate a successful crossover while preserving possession, and continue to final shooting step without propagating error.

Figure 17: Procedural action video generation results. We visualize two procedural generation modes: third-person human kinematics , where actions control full-body motion (e.g., sports behaviors); and egocentric behavior generation , where actions are executed from a first-person perspective.

Figure 18: Ultra-long procedural generation results. We visualize two eight-step kitchen procedures: tomato preparation and storage, and making tomato and cucumber salad. SPIRAL maintains coherent scene context, object states, and action ordering across extended multi-step manipulation.

Figure 19: Single-shot open-loop vs. closed-loop generation on long-horizon procedural tasks. The open-loop baseline often fails to complete the full procedure, exhibiting missing actions, incorrect ordering, and implausible transitions. In contrast, SPIRAL leverages explicit planning and verification, generating complete and long-chain action sequences with coherent ordering and procedural fidelity.

Figure 20: Single-shot open-loop vs. closed-loop generation on long-horizon procedural tasks. The open-loop baseline often fails to complete the full procedure, exhibiting missing actions, incorrect ordering, and implausible transitions. In contrast, SPIRAL leverages explicit planning and verification, generating complete and long-chain action sequences with coherent ordering and procedural fidelity.

Figure 21: Effect of GRPO-based self-evolving. Compared with the model without GRPO, GRPO-based self-evolving improves action completion, temporal coherence, and physical plausibility across long-horizon procedural generation by leveraging critic-derived reward signals during post-training.

Action-Conditioned Video Generation

Action-conditioned video generation creates videos whose motion and scene changes are guided by controls, actions, or interaction signals.

See all 112 papers

Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generation
Apr 20, 2026Yanjun Guo, Zhengqiang Zhang, Pengfei Wang +3Camera-Conditioned Video Generation Long-Horizon Video Generation
The Hong Kong Polytechnic University · OPPO Research Institute
LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation
Sep 29, 2026Byoungwoo Park, Jaemoo Choi, Juho Lee +1Video Diffusion Models Long-Horizon Video Generation
KAIST · Georgia Tech
TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL
Jun 26, 2026Jing Wang, Xiangxin Zhou, Jiajun Liang +5Autoregressive Diffusion RL for Video Generation
Shenzhen Campus of Sun Yat-Sen University · Tencent Hunyuan · Tsinghua University +2

Apr 20, 2026J/K move · Enter open · S save

Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generation

Yanjun Guo, Zhengqiang Zhang, Pengfei Wang, Xinyue Liang+2

The Hong Kong Polytechnic University · OPPO Research Institute

Camera-Conditioned Video Generation Long-Horizon Video Generation Memory-Augmented Video Generation Temporal Consistency in Video Generation

Open paper AskPDF arXiv

Spatially consistent long-horizon video generation aims to maintain temporal and spatial consistency along predefined camera trajectories. Existing methods mostly entangle memory modeling with video generation, leading to inconsistent content during scene revisits and diminished generative capacity when exploring novel regions, even when trained on extensive annotated data. To address these limitations, we propose a decoupled framework that separates memory conditioning from generation. Our approach significantly reduces training costs while simultaneously enhancing spatial consistency and preserving the generative capacity for novel scene exploration. Specifically, we employ a lightweight, independent memory branch to learn precise spatial consistency from historical observations. We first introduce a hybrid memory representation to capture complementary temporal and spatial cues from generated frames, then leverage a per-frame cross-attention mechanism to ensure each frame is conditioned exclusively on the most spatially relevant historical information, which is injected into the generative model to ensure spatial consistency. When generating new scenes, a camera-aware gating mechanism is proposed to mediate the interaction between memory and generation modules, enabling memory conditioning only when meaningful historical references exist. Compared with the existing method, our method is highly data-efficient, yet the experiments demonstrate that our approach achieves state-of-the-art performance in terms of both visual quality and spatial consistency. Project page can be found at https://iguoyanjun.github.io/memorize-when-needed-project-page/.

Abstract

Action-Conditioned Video Generation

Explore similar work

Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generation

LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation

TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL