Producing long-form videos from narration requires interdependent decisions that remain coherent across shots. Existing agentic editors often use multi-stage workflows driven by general models, leaving the quality-cost trade-off of compact, task-adapted policies unclear. We address this gap with WeVisCut, a long-form script-to-video system that connects tool execution with policy learning. Its agent harness maintains shared production state so that tool feedback informs subsequent decisions and repairs. Because finished videos do not expose these decisions, our four-stage data pipeline combines editing inversion with verified replay to turn recovered editing intent into executable supervision. Applied to 3,750 finished videos, this pipeline yields over 130K training examples. We use this supervision to train a single 9B policy that coordinates production and recovers from failures within the same harness. Moreover, we introduce a 100-task benchmark and five metrics assessing editing quality and delivery reliability. Under a shared tool environment, WeVisCut substantially outperforms its base model and achieves comparable performance to Claude Opus 5 while using nearly 33x fewer generated tokens per successful output minute. It also outperforms task-matched native editors on our proposed metrics.
Figures & tables
Figure 2: The WeVisCut agent harness with stateful tool selection. A shared policy chooses legal tools from the evolving production state. Validated returns update artifacts and inform inspection, local repair, and replanning before final stitching and delivery.
Figure 3: Executable data construction through editing inversion, verified replay, observe–reflect–repair, and trajectory materialization. (A) Editing inversion recovers aligned narration and shot evidence from a finished video. (B) Verified replay executes this intent with deployment-aligned production tools. (C) Observe–reflect–repair performs bounded inspection and targeted repair to preserve successful artifacts. (D) Trajectory materialization organizes recorded tool interactions, feedback, and repair links into role-specific training examples and rollout records.
Figure 4: Three training phases of WeVisCut. Phase 1 learns tool use and recovery through unified SFT and runtime-verified RFT. Phase 2 applies Guided-OPD with decreasing teacher intervention. Phase 3 uses RCF-DAPO to compare alternatives within matched roles and decision contexts.
Task-specific performance ↑
PCE ↓
Method
SCF
VNSM
TPR
LHC
ERY
Overall
Calls / min
Tokens / min
General-purpose baselines
Qwen3.5-9B
69.8±1.6
59.2±1.8
79.5±1.7
93.7±0.8
70.4±2.1
74.5
3.1
11,883
Qwen3.6-27B
72.6±1.4
59.7±1.6
80.1±1.5
95.1±0.7
72.1±1.9
75.9
3.9
6,742
Qwen3.6-35B-A3B
71.9±1.9
56.7±2.0
83.4±1.8
97.3±0.6
79.4±2.3
77.7
4.7
27,554
Reference models
Table 1: Policy comparison on 100 held-out tasks in the shared tool environment. Component scores are mean ± standard deviation across five runs over the full test set. Overall is the equal-weight mean of the five component means. Calls and generated tokens include all attempts and are normalized by successful output minutes. Arrows indicate preferred directions, and the best results are bold.
Task-matched performance ↑
Native cost ↓
System
SCF
VNSM
TPR
LHC
ERY
Overall
Time (min) / out min
Cost (USD) / out min
Crayotter
81.6±1.3
67.8±1.4
89.1±1.3
94.2±0.9
91.6±1.4
84.9
8.6
0.97
VideoAgent
78.4±1.5
65.2±1.6
86.7±1.5
93.5±0.6
88.7±1.7
82.5
10.4
0.58
WeVisCut (Ours)
88.4±2.1
74.6±2.3
94.0±2.0
98.8±0.5
96.7±1.8
90.5
2.1
0.09
Table 2: End-to-end comparison using each system’s native workflow on the same 100 tasks. Component scores are mean ± standard deviation across five runs, and Overall is the equal-weight mean of the component means. Native costs include all attempted runs and report wall-clock minutes and estimated USD per successful output minute. The best results are bold.
Phase
Variant
SCF
VNSM
TPR
LHC
ERY
Overall
–
Base
69.8±1.6
59.2±1.8
79.5±1.7
93.7±0.8
70.4±2.1
74.5
1
SFT
73.6±2.0
60.5±2.2
81.2±2.1
94.6±1.0
87.6±2.4
79.5
1
SFT → RFT
79.8±1.8
65.1±2.0
86.9±1.9
95.8±0.9
89.8±2.2
83.5
2
SFT → RFT → Guided-OPD
81.0±1.6
67.4±1.8
87.8±1.7
96.6±0.8
91.5±1.9
84.9
3
WeVisCut (Ours)
88.4±2.1
74.6±2.3
94.0±2.0
98.8±0.5
96.7±1.8
90.5
Table 3: Cumulative training-stage analysis with a fixed Qwen3.5-9B backbone and evaluation environment. Phase 1 includes SFT and RFT, Phase 2 adds Guided-OPD, and Phase 3 adds RCF-DAPO . Component scores are mean ± standard deviation across five runs on the 100 held-out tasks. Overall is the equal-weight mean of the component means. The best results are bold.
Figure 5: Cumulative training progression for (a) SCF, (b) VNSM, and (c) LHC. OPD and RL denote Guided-OPD and RCF-DAPO , respectively. Error bars show standard deviations across five evaluation runs on the 100 held-out tasks. Horizontal lines show Qwen3.5-9B, Qwen3.5-397B-A17B, and Claude Opus 5 under the shared tool environment. Results at these checkpoints on all five metrics are shown in Table 3 .
Method
Theme fidelity
Content richness
Narrative coherence
Editing smoothness
Visual quality
Overall preference
Crayotter [ 38 ]
3.58
3.42
3.35
3.48
3.55
3.48
VideoAgent [ 49 ]
3.46
3.60
3.30
3.42
3.52
3.46
WeVisCut (Ours)
4.45
4.32
4.58
4.52
4.38
4.56
Table 4: Blinded human evaluation on 100 held-out tasks with three raters. Entries are mean 1–5 ratings (higher is better). Overall preference is a human rating, distinct from the Overall Score.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Key configuration
Guided-OPD
Qwen3.5-9B student with a Qwen3.5-397B-A17B-FP8 teacher and Kvocab=50 . Per-call prompt/response caps: 12,288/4,096 tokens. Batch 16, 90 steps, βstart=0.8 decaying over 70 steps, and learning rate 10−6 . At most 50 assistant turns with early termination.
RCF-DAPO
Per-call prompt/response caps: 24,576/12,288 tokens. Train/generation/mini-batch: 48/72/24, matched call group size 8, and separate O/G/C batches. Stratified dynamic sampling with coverage replay. Clip [0.8,1.28] , temperature 0.9, 16-turn deployment horizon, learning rate 10−6 , LoRA rank/alpha 32/64, and no KL penalty.
Appendix
Table 5: Registered post-training configuration.
Figure 6: Auditable VNSM evaluator path. A shot packet binds the narration window, timestamps, video and manifest digests, sampler/prompt versions, judge snapshot, and decoding configuration. The frozen sampler exposes four timestamped frames and, only when motion is necessary, one bounded clip. The held-out Claude Opus 5 evaluator must first record visible evidence, then choose one anchored grade and emit schema-validated JSON. Aggregation excludes unavailable calls rather than converting infrastructure failure into a zero-quality judgment.
System
Paradigm
Fixed narration
Shot plan
Trace / repair
Fixed sources
Matched native track
Crayotter
Artifact-centric multi-agent
✓
✓
✓
✓
✓
VideoAgent
Large agent library
✓
✓
✓
✓
✓
CutClaw
Specialized multi-agent
✗
✓
✓
✓
✗
CineAgents
Design-and-compose agents
✗
✓
✓
✓
✗
DirectorAgent
Script-to-video generation
✗
✓
✗
✗
✗
Appendix
Table 6: Baseline task-contract audit. Systems satisfying every required contract element enter the task-matched native-system comparison.
Figure 7: Visual Input to the Evaluation Harness: Case 1. A contact sheet of temporally sampled and timestamped video frames used for evidence-grounded diagnostic evaluation.
Figure 8: Visual Input to the Evaluation Harness: Case 2.
Editing a long-form video from heterogeneous footage requires more than selecting clips: an agent must preserve narrative intent across material preparation, timeline construction, post-production, and revision while leaving enough evidence to diagnose failures. We present Crayotter, an open-source multimodal multi-agent system for prompt-driven video editing. Crayotter organizes production into three phases: coverage-aware material preparation, artifact-based editing research, and tool-grounded timeline execution. Each phase externalizes inspectable artifacts, including coverage reports, multimodal analyses, editing blueprints, tool calls, and intermediate renders. These artifacts make an editing run traceable and allow failed segments to be diagnosed and selectively revised instead of requiring a full restart. We evaluate Crayotter on 23 editing themes against CapCut-Mate and CutClaw. Under human evaluation, Crayotter achieves an average score of 3.40/5, compared with 2.44 and 1.70 for the two baselines, with consistent gains in theme alignment, narrative coherence, and editing smoothness. We additionally describe a replayable trajectory schema and verifiable reward design that prepare these workflows for future policy optimization. Code, traces, and examples are publicly available at https://github.com/idwts/Crayotter.
Video editing has become essential in digital media creation, yet existing automated systems are restricted to short segment processing and domain-specific tasks. They face two critical limitations: i) inability to handle diverse video comprehension and editing operations, and ii) lack of long-video understanding for coherent narrative creation. We propose VideoAgent, an all-in-one agentic framework addressing these challenges through two key innovations. First, we develop automated video shot creation with shot planning agents for coherent narratives and cross-modal retrieval for aligned visual content. Second, we design a multi-agent orchestration framework integrating over thirty specialized editing agents. Intent parsing filters relevant tools while textual-gradient graph optimization assembles complex editing pipelines. Extensive experiments on our newly-proposed VideoEdit benchmark and public datasets demonstrate VideoAgent's superiority over existing multimodal LLMs and agentic systems. VideoAgent achieves 87-95% orchestration success rates while reducing API costs by 60%. Human evaluation across six video categories shows VideoAgent produces professional-quality content approaching human-level performance, with ratings only 4% below human-created videos. We release our code at https://github.com/HKUDS/VideoAgent.
Hengji Zhou, Lingxuan Huang, Jian Wang +4
1Harbin Institute of Technology, Shenzhen · 2South China University of Technology · 3The University of Hong Kong +1
Agentic long video generation requires planning, tool orchestration, and cross-clip coordination over a long horizon. Most existing video agents either rely on static, human-crafted workflows, which require substantial manual effort and poorly adapt across tasks, or iteratively refine the output of the current task without persistently distilling execution experience into reusable skills for future tasks. We introduce VideoWeaver, an agent harness and benchmark that evaluates and evolves skills for long video generation. Given a single high-level instruction, an agent dynamically composes foundation skills into its own workflow rather than following a predefined pipeline. We construct a benchmark of 16 task categories and 285 cases, with references spanning text, image, audio, video, and their combinations. We further propose an evidence-grounded agent-as-judge that inspects both the execution trace and the final video to diagnose process and output failures. Based on this feedback, our evolution algorithm progressively refines category-level composition and creator skills, allowing recurring experience to guide dynamically constructed workflows for unseen cases. Experiments show that explicit composition skills improve the generation process over foundation skills alone, while skill evolution further improves output quality and generalizes to unseen cases. Incorporating judge feedback yields additional gains, especially on output metrics, and the agent-as-judge aligns well with human, particularly on process metrics. Code is available at https://github.com/JianhuiWei7/VideoWeaver.