SkillPE: Creativity-Oriented Cinematic Skill Evolution for Text-to-Video Prompt Engineering

Yanwei Huang; Mingxuan Zhu; Shujie Li; Shiyuan Liu; Yuanxing Zhang; Arpit Narechania

cs.CVSep 28, 2026

SkillPE: Creativity-Oriented Cinematic Skill Evolution for Text-to-Video Prompt Engineering

Authors: Yanwei Huang, Mingxuan Zhu, Shujie Li, Shiyuan Liu, Yuanxing Zhang, Arpit Narechania

Organizations: HKUST · KlingAI · Shanghai Jiaotong University · The University of Hong Kong

Abstract

Achieving high-quality, cinematic results in text-to-video generation remains challenging for non-experts, whose prompts often lack professional narrative and creative design. We propose SkillPE, a prompt engineering (PE) framework that evolves reusable cinematic skills from expert-authored seeds. SkillPE represents shot logic, composition, lighting, sound design, and other filmmaking cues in a fine-grained format, and retrieves movie references categorized as resonators (good matches), dissonants (weak matches), and divergents (creatively useful near-misses). The first two refine when and how a skill should be applied, while divergents inspire alternative cinematic realizations at different degrees of modification while preserving the user intent. Candidate skills are assessed through generated videos along prompt fidelity, cinematic quality, narrative appeal, and creativity to construct the final skill libraries. Experiments on StoryEval and VBench show improvements of up to 1.40 points over the strongest external baseline and 0.51 points over seed skills on 7-point four-dimensional evaluation, while remaining competitive on benchmark-native metrics. Overall, SkillPE offers a practical approach to balancing fidelity and creativity in cinematic text-to-video generation. Code is available at https://github.com/Ais0n/SkillPE .

Figures & tables

Figure 1: Examples for comparing baselines with SkillPE and showing the skills’ effect.

Figure 2: An overview of SkillPE. Starting from expert-authored seed skills, SkillPE first enriches the nuanced cinematography details through an extended shot format. The skills then go through adaptive discovery and evolution, where reference movies are classified into resonators, dissonants, and divergents by similarity. The first two are distilled for usage guidance while the divergents are used for exploring skill variants of varying degrees of deviation. The consequent skills are evaluated and compiled into libraries through a pipeline of video generation and holistic assessment.

		StoryEval						VBench
Backbone	Baseline	Official	PF	CQ	NA	CR	Overall	Official	PF	CQ	NA	CR	Overall
MiniMax-H3	Raw Prompt	0.63	5.10	5.57	4.70	2.04	4.35	0.80	5.43	5.35	3.47	2.40	4.16
	Gemini 3.1 Pro	0.80	6.04	6.28	5.39	2.87	5.15	0.85	6.66	5.82	2.81	2.02	4.33
	VPO	0.70	5.58	5.69	5.04	2.13	4.61	0.84	6.51	5.57	2.91	1.99	4.24
	Prompt-A-Video	0.59	4.93	5.98	4.70	3.15	4.69	0.82	5.98	6.01	3.28	3.33	4.65
	RAPO	0.65	5.30	5.56	4.77	2.09	4.43	0.82	6.25	5.54	3.64	2.53	4.49

Table 1: Evaluation Results on StoryEval and VBench. For each backbone, the top scores are bolded and the second highest scores are underlined .

	StoryEval					VBench
	PF	CQ	NA	CR	Overall	PF	CQ	NA	CR	Overall
Expert-authored	6.43	6.43	5.73	4.01	5.65	6.70	6.42	4.79	4.67	5.64
Seed Skill	6.39	6.59	5.72	4.57	5.82	6.60	6.47	5.05	4.87	5.75
R&D only	6.35	6.63	5.82	4.57	5.84	6.63	6.54	5.14	5.23	5.88
Bold only	6.19	6.58	5.55	4.91	5.81	6.46	6.28	4.81	5.30	5.71
Wilder only	6.12	6.47	5.38	5.50	5.87	6.41	6.53	5.42	5.83	6.04

Table 3: Ablation study on the skill formulation and evolution approaches.

	100 words				200 words				400 words				800 words
Metric	DLR	Seed	Top-C	Top-O	DLR	Seed	Top-C	Top-O	DLR	Seed	Top-C	Top-O	DLR	Seed	Top-C	Top-O
PF	6.28	5.94	5.70	5.75	6.44	6.02	5.83	6.00	6.62	6.13	6.06	6.08	6.54	6.41	6.22	6.29
CQ	6.23	6.48	6.39	6.41	6.38	6.58	6.56	6.58	6.39	6.66	6.59	6.58	6.42	6.68	6.69	6.64
NA	5.75	5.54	5.33	5.47	5.87	5.54	5.61	5.68	5.99	5.66	5.71	5.71	5.92	5.83	5.85	5.96
CR	2.82	4.49	4.56	4.60	3.15	4.65	5.04	5.05	3.21	4.95	5.35	5.41	3.24	5.21	5.58	5.63
Overall	5.27	5.61	5.50	5.56	5.46	5.70	5.76	5.83	5.55	5.85	5.93	5.94	5.53	6.03	6.08	6.13

Table 4: Prompt-budget-controlled ablation. DLR = Direct LLM Rewrite; Top-C = SkillPE Top Creativity; Top-O = SkillPE Top Overall. Best result within each budget and metric is bolded.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Library	Seed	R&D	Bold	Wilder	Extreme
Selection backbone: MiniMax-H3
SkillPE Top Creativity	20%	15%	10%	30%	25%
SkillPE Top-2 Creativity	27.5%	20%	10%	25%	17.5%
SkillPE Top Overall	5%	25%	10%	25%	35%
Selection backbone: LTX-2.5
SkillPE Top Creativity	5%	20%	10%	40%	25%

Appendix

Table 5: Skill source distributions in the final libraries, grouped by the video-generation backbone used for skill selection. Top, Top-2, and Overall contain 20, 40, and 20 skills, respectively.

Method / Stage	Model / Checkpoint	Temp.	Key Configuration
Raw Prompt	–	–
Direct LLM Rewrite	Gemini 3.1 Pro	0.25	Single-pass rewriting.
VPO	VPO-5B	0	do_sample=False
Prompt-A-Video	Prompt_A_Video_CV	0	do_sample=False
Mora: planning / PE	Gemini 3.1 Pro	0.25	The video generation agent uses MiniMax-H3 or LTX-2.5 for consistency.
Mora: first frame	SDXL Base 1.0	–	$1024{\times}576$ resolution, 24 inference steps, guidance scale 7.0; the generated image is used as the frame-0 condition.

Appendix

Table 6: Prompt-engineering and routing configurations.

Method	PF	CQ	NA	CR	Overall
Direct LLM Rewrite	$5.98\pm 0.17$	$5.98\pm 0.09$	$5.07\pm 0.31$	$2.41\pm 0.08$	$4.86\pm 0.15$
Seed Skill	$6.49\pm 0.17$	$6.51\pm 0.11$	$5.65\pm 0.09$	$4.00\pm 0.22$	$5.66\pm 0.10$
SkillPE Top Creativity	$6.09\pm 0.10$	$6.52\pm 0.07$	$5.64\pm 0.14$	$4.31\pm 0.21$	$5.64\pm 0.08$
SkillPE Top Overall	$\mathbf{6.51\pm 0.08}$	$6.38\pm 0.04$	$\mathbf{5.71\pm 0.02}$	$\mathbf{4.85\pm 0.10}$	$\mathbf{5.86\pm 0.02}$

Appendix

Table 7: Sensitivity to video-generation seeds on StoryEval with MiniMax-H3. Values are mean

\pm

sample standard deviation across three seed replicates, where each replicate is first averaged over the sampled prompts.

Method	StoryEval	VBench
Raw Prompt	$16.1\pm 4.5$	$7.6\pm 5.9$
Direct LLM Rewrite	$43.2\pm 12.6$	$52.7\pm 19.0$
VPO	$105.8\pm 8.5$	$101.9\pm 6.8$
Prompt-A-Video	$99.1\pm 12.0$	$98.1\pm 11.3$
RAPO	$43.4\pm 11.4$	$40.6\pm 14.2$
Mora	$29.9\pm 15.4$	$28.0\pm 15.3$

Appendix

Table 8: Prompt length (mean

\pm

standard deviation, in whitespace-delimited English words) distribution before backbone-specific prompt adaptation.

Comparison	$\Delta$ 4D Avg. [95% CI]
SkillPE Top $-$ Seed	$+0.359$ [ $0.245$ , $0.464$ ]
SkillPE Top-2 $-$ Seed	$+0.242$ [ $0.114$ , $0.364$ ]
SkillPE Overall $-$ Seed	$+0.329$ [ $0.214$ , $0.447$ ]

Appendix

Table 9: Robustness of the human evaluation under two-way bootstrap resampling over prompts and annotators. Intervals are pointwise 95% percentile confidence intervals.

Figure 4: Examples that reveal the effect of different candidate skills in the adaptive skill discovery.

Explore similar work

WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation
Sep 24, 2026Yubo Zhu, Yawen Shao, Ziyun Dai +27Video Generation
Nanjing University · Wan Team, Alibaba Group · University of Science and Technology of China +2
FilmBench: A Film-Grade Benchmark for Cinematic Video Generation
Jul 27, 2026Shengyi Wang, Niantong Li, Guangzheng Hu +27Cinematic Ideal Long-Video Benchmarks
Alibaba Group · Moku Lab, Hujing Digital Media & Entertainment Group · Beijing Film Academy
VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation
Oct 1, 2026Yu Huang, Jungang Li, Zhiyuan Wang +8Text-To-Video Generation Model Video Generation
Department of Data Science, City University of Hong Kong · The Hong Kong University of Science and Technology (Guangzhou) · Westlake University +3

Sep 24, 2026, cs.CVJ/K move · Enter open · S save

WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation

Yubo Zhu, Yawen Shao, Ziyun Dai, Zixun Fang+26

Nanjing University · Wan Team, Alibaba Group · University of Science and Technology of China+2

Video Generation

Open paper AskPDF arXiv

Video generation begins in text space by authoring a cinematic screenplay, then materializes into pixels. As contemporary video generators scale to 30 seconds and faithfully follow complex conditions, the textual prompt largely directs the production, planning how actions, camera trajectories, lighting, and sound unfold across multi-shot sequences. In this paper, we present WanPE, a 397B-parameter prompt enhancement model trained on 1.05M real-world videos to master director-level cinematic planning. WanPE formulates shot-level cinematic plans via video-grounded reverse construction and employs Semantic-Consistency GRPO (SC-GRPO) to faithfully preserve user requirements across shots and over time. To benchmark this capability, we curate WanPEval, a human-annotated testbed covering durations from 5 to 30 seconds across varying intent granularities, supported by approximately 11K blind pairwise assessments. When powering Wan3.0's video generator, WanPE-397B boosts human preference over raw user prompts by 10.66-18.84 points at 5-15 seconds and by a dramatic 50.86 points in the 30-second arena. Ablation studies show that reverse construction demonstrates clear superiority over forward rewriting, while SC-GRPO robustly preserves semantic fidelity across model scales. Ultimately, WanPE leads all evaluated commercial offerings at 5-15 seconds and remains competitive with Seedance 2.5 at 30 seconds.

Abstract

Explore similar work

WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation

FilmBench: A Film-Grade Benchmark for Cinematic Video Generation

VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation