Recent video foundation models generate high-quality and limited-length clips, but orchestrating them into minute-level, story-coherent films remains an open problem. Existing video agents highly rely on natural-language planning or predefined workflows, which make their production hard to program, customize, and extend. Therefore, we introduce VOL-Agent, a programmable video agent built on a Video Orchestration Language (VOL). With static compilation and verification, followed by runtime dynamic loading, VOL-Agent decouples structural reliability from execution flexibility. Through the skill-driven executor, VOL-Agent dynamically resolves tools and skills from an open-world repository and supports user-defined skills in natural language. Moreover, VOL-Agent is capable of crafting new director skills from learned knowledge and improves itself through feedback to accumulate filmmaking knowledge. To overcome the disconnect between different segments, we apply a structural memory mechanism to record and recall world and entity states, sustaining entity and scene consistency across different shots. Critic feedback supports local revision, while reusable filmmaking knowledge and experiences can be retained for subsequent tasks. Experiments on three benchmarks show that VOL-Agent generates minute-scale videos with strong overall quality and limited cost.
Figures & tables
Figure 1: Top: the cost and performance of different video agents; Bottom: the comparison of Predefined workflows versus skill-driven video orchestration in long video generation.
Figure 2: The overall framework of the proposed VOL-Agent. The default workflow of generating long video in VOL-Agent is CODING , BUILD , and RUN . In CODING , the raw script is translated into a program that defines entities, scenes, and shots. In BUILD , the program is validated for unique identifiers, acyclic dependencies, and valid references, then elaborated into execution steps specifying details. In RUN , an orchestrator constructs workflow plans. Each plan specifies the tools and associated skills used to execute the generation steps and produce the final clip.
Figure 3: The multi-modal memory mechanism in VOL-Agent. The memory mechanism employs Entity Memory , Scene Memory and Shot Memory . These memories are built from the reference images and optional clay renderings. Reference images provide the visual information of entities or scenes to maintain consistent ID or scene preservation across multiple video generations, and clay rendering ensures structural consistency so that scene layout or entities’ positions do not shift due to camera cuts.
Methods
Entity
Scene
Transition
Semantic
Cinematic
Narrative
Action&
Average
Consistency
Consistency
Consistency
Alignment
Aesthetics
Coherence
Staging
Autoregressive base model
MiniMax-H3 [ 26 ]
3.539
3.573
3.051
3.206
3.468
3.210
2.917
3.280
Seedance2.0-fast [ 33 ]
3.914
3.998
3.301
3.762
3.633
3.739
3.599
3.706
Agent-based methods
MovieAgent [ 44 ]
3.337
3.208
2.871
3.435
3.294
3.081
3.093
3.188
Table 1: The comparison of VOL-Agent with other video agent methods on our benchmark (higher is better).
Benchmark
Methods
Entity
Scene
Transition
Semantic
Cinematic
Narrative
Action&
Average
Consistency
Consistency
Consistency
Alignment
Aesthetics
Coherence
Staging
ViMaxBench
MiniMax-H3 [ 26 ]
3.969
3.947
3.120
3.721
3.620
3.627
3.522
3.647
MovieAgent [ 44 ]
3.557
3.725
2.794
3.587
3.511
3.408
3.263
3.406
ViMax [ 16 ]
4.011
3.993
3.056
3.791
3.570
3.572
3.496
3.641
VideoMemory [ 51 ]
3.858
3.893
3.155
3.686
3.676
3.510
3.422
3.600
CineCrew [ 3 ]
4.302
4.279
3.070
3.909
3.788
3.778
3.718
3.835
Table 2: The comparison of VOL-Agent with other video agent methods on two public benchmarks. Note that we follow CineCrew to sample 20 narratives from MovieBench.
Methods
Entity
Scene
Transition
Semantic
Cinematic
Narrative
Action&
Average
Consistency
Consistency
Consistency
Alignment
Aesthetics
Coherence
Staging
Entity&Scene memory
4.208
4.108
3.252
4.078
4.128
4.036
3.988
3.971
+ Shot memory
4.281
4.207
3.671
4.181
4.207
4.182
4.010
4.105
w/o Critic
3.915
3.972
3.554
4.096
4.082
4.089
3.992
3.957
w/o Experience
3.998
4.074
3.582
4.103
4.063
4.109
4.027
3.993
w/o VOL
3.823
3.909
3.517
4.087
4.079
4.101
3.807
3.903
Table 3: The ablation studies of VOL-Agent on the proposed benchmark.
Figure 4: Qualitative comparison across different methods.
Methods
EC
SC
TC
SA
CA
NC
AS
Avg.
Baseline
4.214
4.143
3.462
4.214
4.000
4.071
3.786
3.984
Best-of- N
N=2
4.267
4.178
3.662
4.273
3.857
3.929
3.792
3.994
N=4
4.071
4.214
3.615
4.214
3.929
4.001
3.929
3.996
N=6
4.211
4.141
3.690
4.355
3.998
4.069
3.570
4.005
Refine- N
Table 4: The results with Best-of- N selection and critic-guided refinement N times on the subset of our benchmark.
Comparison
Win ( ↑ )
Tie
Lose ( ↓ )
Round-2 vs. No Reference
97.9%
2.1%
0.0%
Round-2 vs. Ref. Video as Input
80.2%
14.6%
5.2%
Round-2 vs. Round-1
24.0%
57.3%
18.7%
Table 5: Reference-style alignment of VOL-Agent using different round self-crafting director skills, as ranked by Gemini-3.8-Flash.
Figure 5: Human studies on different video agents against the proposed VOL-Agent.
Figure 6: Additional visual comparisons and director-style examples. The upper rows compare video agents; the lower rows show VOL-Agent outputs conditioned on filmmaking styles, with reference films shown below.
School of Information and Communication Engineering, Communication University of China, Beijing, China · State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China