WeVisCut: The Whole Editing Room in One Agent From Narration to Long-Form Video

Chenyu Mu; Wenming Weng; Qinrui Wang; Xingsong Ye; Hao Yu; Jun He; Chong Sun; Chen Li

cs.CVOct 28, 2026

WeVisCut: The Whole Editing Room in One Agent From Narration to Long-Form Video

Authors: Chenyu Mu, Wenming Weng, Qinrui Wang, Xingsong Ye, Hao Yu, Jun He, Chong Sun, Chen Li,

Organizations: Xidian University · WeChat Vision, Tencent Inc.

Abstract

Producing long-form videos from narration requires interdependent decisions that remain coherent across shots. Existing agentic editors often use multi-stage workflows driven by general models, leaving the quality-cost trade-off of compact, task-adapted policies unclear. We address this gap with WeVisCut, a long-form script-to-video system that connects tool execution with policy learning. Its agent harness maintains shared production state so that tool feedback informs subsequent decisions and repairs. Because finished videos do not expose these decisions, our four-stage data pipeline combines editing inversion with verified replay to turn recovered editing intent into executable supervision. Applied to 3,750 finished videos, this pipeline yields over 130K training examples. We use this supervision to train a single 9B policy that coordinates production and recovers from failures within the same harness. Moreover, we introduce a 100-task benchmark and five metrics assessing editing quality and delivery reliability. Under a shared tool environment, WeVisCut substantially outperforms its base model and achieves comparable performance to Claude Opus 5 while using nearly 33x fewer generated tokens per successful output minute. It also outperforms task-matched native editors on our proposed metrics.

Figures & tables

Figure 2: The WeVisCut agent harness with stateful tool selection. A shared policy chooses legal tools from the evolving production state. Validated returns update artifacts and inform inspection, local repair, and replanning before final stitching and delivery.

Figure 3: Executable data construction through editing inversion, verified replay, observe–reflect–repair, and trajectory materialization. (A) Editing inversion recovers aligned narration and shot evidence from a finished video. (B) Verified replay executes this intent with deployment-aligned production tools. (C) Observe–reflect–repair performs bounded inspection and targeted repair to preserve successful artifacts. (D) Trajectory materialization organizes recorded tool interactions, feedback, and repair links into role-specific training examples and rollout records.

Figure 4: Three training phases of WeVisCut. Phase 1 learns tool use and recovery through unified SFT and runtime-verified RFT. Phase 2 applies Guided-OPD with decreasing teacher intervention. Phase 3 uses RCF-DAPO to compare alternatives within matched roles and decision contexts.

	Task-specific performance $\uparrow$						PCE $\downarrow$
Method	SCF	VNSM	TPR	LHC	ERY	Overall	Calls / min	Tokens / min
General-purpose baselines
Qwen3.5-9B	$69.8{\pm}1.6$	$59.2{\pm}1.8$	$79.5{\pm}1.7$	$93.7{\pm}0.8$	$70.4{\pm}2.1$	74.5	3.1	11,883
Qwen3.6-27B	$72.6{\pm}1.4$	$59.7{\pm}1.6$	$80.1{\pm}1.5$	$95.1{\pm}0.7$	$72.1{\pm}1.9$	75.9	3.9	6,742
Qwen3.6-35B-A3B	$71.9{\pm}1.9$	$56.7{\pm}2.0$	$83.4{\pm}1.8$	$97.3{\pm}0.6$	$79.4{\pm}2.3$	77.7	4.7	27,554
Reference models

Table 1: Policy comparison on 100 held-out tasks in the shared tool environment. Component scores are mean

\pm

standard deviation across five runs over the full test set. Overall is the equal-weight mean of the five component means. Calls and generated tokens include all attempts and are normalized by successful output minutes. Arrows indicate preferred directions, and the best results are bold.

	Task-matched performance $\uparrow$						Native cost $\downarrow$
System	SCF	VNSM	TPR	LHC	ERY	Overall	Time (min) / out min	Cost (USD) / out min
Crayotter	$81.6{\pm}1.3$	$67.8{\pm}1.4$	$89.1{\pm}1.3$	$94.2{\pm}0.9$	$91.6{\pm}1.4$	84.9	8.6	0.97
VideoAgent	$78.4{\pm}1.5$	$65.2{\pm}1.6$	$86.7{\pm}1.5$	$93.5{\pm}0.6$	$88.7{\pm}1.7$	82.5	10.4	0.58
WeVisCut (Ours)	$\mathbf{88.4}{\pm}2.1$	$\mathbf{74.6}{\pm}2.3$	$\mathbf{94.0}{\pm}2.0$	$\mathbf{98.8}{\pm}0.5$	$\mathbf{96.7}{\pm}1.8$	90.5	2.1	0.09

Table 2: End-to-end comparison using each system’s native workflow on the same 100 tasks. Component scores are mean

\pm

standard deviation across five runs, and Overall is the equal-weight mean of the component means. Native costs include all attempted runs and report wall-clock minutes and estimated USD per successful output minute. The best results are bold.

Phase	Variant	SCF	VNSM	TPR	LHC	ERY	Overall
–	Base	$69.8{\pm}1.6$	$59.2{\pm}1.8$	$79.5{\pm}1.7$	$93.7{\pm}0.8$	$70.4{\pm}2.1$	74.5
1	SFT	$73.6{\pm}2.0$	$60.5{\pm}2.2$	$81.2{\pm}2.1$	$94.6{\pm}1.0$	$87.6{\pm}2.4$	79.5
1	SFT $\rightarrow$ RFT	$79.8{\pm}1.8$	$65.1{\pm}2.0$	$86.9{\pm}1.9$	$95.8{\pm}0.9$	$89.8{\pm}2.2$	83.5
2	SFT $\rightarrow$ RFT $\rightarrow$ Guided-OPD	$81.0{\pm}1.6$	$67.4{\pm}1.8$	$87.8{\pm}1.7$	$96.6{\pm}0.8$	$91.5{\pm}1.9$	84.9
3	WeVisCut (Ours)	$\mathbf{88.4}{\pm}2.1$	$\mathbf{74.6}{\pm}2.3$	$\mathbf{94.0}{\pm}2.0$	$\mathbf{98.8}{\pm}0.5$	$\mathbf{96.7}{\pm}1.8$	90.5

Table 3: Cumulative training-stage analysis with a fixed Qwen3.5-9B backbone and evaluation environment. Phase 1 includes SFT and RFT, Phase 2 adds Guided-OPD, and Phase 3 adds RCF-DAPO . Component scores are mean

\pm

standard deviation across five runs on the 100 held-out tasks. Overall is the equal-weight mean of the component means. The best results are bold.

Figure 5: Cumulative training progression for (a) SCF, (b) VNSM, and (c) LHC. OPD and RL denote Guided-OPD and RCF-DAPO , respectively. Error bars show standard deviations across five evaluation runs on the 100 held-out tasks. Horizontal lines show Qwen3.5-9B, Qwen3.5-397B-A17B, and Claude Opus 5 under the shared tool environment. Results at these checkpoints on all five metrics are shown in Table 3 .

Method	Theme fidelity	Content richness	Narrative coherence	Editing smoothness	Visual quality	Overall preference
Crayotter [ 38 ]	3.58	3.42	3.35	3.48	3.55	3.48
VideoAgent [ 49 ]	3.46	3.60	3.30	3.42	3.52	3.46
WeVisCut (Ours)	4.45	4.32	4.58	4.52	4.38	4.56

Table 4: Blinded human evaluation on 100 held-out tasks with three raters. Entries are mean 1–5 ratings (higher is better). Overall preference is a human rating, distinct from the Overall Score.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Key configuration
Guided-OPD	Qwen3.5-9B student with a Qwen3.5-397B-A17B-FP8 teacher and $K_{\rm vocab}=50$ . Per-call prompt/response caps: 12,288/4,096 tokens. Batch 16, 90 steps, $\beta_{\rm start}=0.8$ decaying over 70 steps, and learning rate $10^{-6}$ . At most 50 assistant turns with early termination.
RCF-DAPO	Per-call prompt/response caps: 24,576/12,288 tokens. Train/generation/mini-batch: 48/72/24, matched call group size 8, and separate O/G/C batches. Stratified dynamic sampling with coverage replay. Clip $[0.8,1.28]$ , temperature 0.9, 16-turn deployment horizon, learning rate $10^{-6}$ , LoRA rank/alpha 32/64, and no KL penalty.

Appendix

Table 5: Registered post-training configuration.

Figure 6: Auditable VNSM evaluator path. A shot packet binds the narration window, timestamps, video and manifest digests, sampler/prompt versions, judge snapshot, and decoding configuration. The frozen sampler exposes four timestamped frames and, only when motion is necessary, one bounded clip. The held-out Claude Opus 5 evaluator must first record visible evidence, then choose one anchored grade and emit schema-validated JSON. Aggregation excludes unavailable calls rather than converting infrastructure failure into a zero-quality judgment.

System	Paradigm	Fixed narration	Shot plan	Trace / repair	Fixed sources	Matched native track
Crayotter	Artifact-centric multi-agent	✓	✓	✓	✓	✓
VideoAgent	Large agent library	✓	✓	✓	✓	✓
CutClaw	Specialized multi-agent	✗	✓	✓	✓	✗
CineAgents	Design-and-compose agents	✗	✓	✓	✓	✗
DirectorAgent	Script-to-video generation	✗	✓	✗	✗	✗

Appendix

Table 6: Baseline task-contract audit. Systems satisfying every required contract element enter the task-matched native-system comparison.

Figure 7: Visual Input to the Evaluation Harness: Case 1. A contact sheet of temporally sampled and timestamped video frames used for evidence-grounded diagnostic evaluation.

Figure 8: Visual Input to the Evaluation Harness: Case 2.

Agentic Video Editing

See all 13 papers

Crayotter: Traceable Multi-Agent Workflows for Long-Form Video Editing
May 31, 2026Lecheng Yan, Yichong Zhang, Ben Pan +5Video Editing Multi-Agent Collaboration
Crayotter Project
VideoAgent: All-in-One Framework for Video Understanding and Editing
Jun 22, 2026Hengji Zhou, Lingxuan Huang, Jian Wang +4Video Editing Benchmarks Agentic Video Editing
1Harbin Institute of Technology, Shenzhen · 2South China University of Technology · 3The University of Hong Kong +1
VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generation
Jun 6, 2026Jianhui Wei, Yan Zhang, Jie Tan +7Long Video Generation Agentic Video Generation
Zhejiang University · ByteDance

May 31, 2026J/K move · Enter open · S save

Crayotter: Traceable Multi-Agent Workflows for Long-Form Video Editing

Lecheng Yan, Yichong Zhang, Ben Pan, Xiaoyu Zheng+4

Crayotter Project

Video Editing Multi-Agent Collaboration Instruction-Based Video Editing Multimodal Agents Multimodal Tool Use Agentic Video Editing

Open paper AskPDF arXiv

Editing a long-form video from heterogeneous footage requires more than selecting clips: an agent must preserve narrative intent across material preparation, timeline construction, post-production, and revision while leaving enough evidence to diagnose failures. We present Crayotter, an open-source multimodal multi-agent system for prompt-driven video editing. Crayotter organizes production into three phases: coverage-aware material preparation, artifact-based editing research, and tool-grounded timeline execution. Each phase externalizes inspectable artifacts, including coverage reports, multimodal analyses, editing blueprints, tool calls, and intermediate renders. These artifacts make an editing run traceable and allow failed segments to be diagnosed and selectively revised instead of requiring a full restart. We evaluate Crayotter on 23 editing themes against CapCut-Mate and CutClaw. Under human evaluation, Crayotter achieves an average score of 3.40/5, compared with 2.44 and 1.70 for the two baselines, with consistent gains in theme alignment, narrative coherence, and editing smoothness. We additionally describe a replayable trajectory schema and verifiable reward design that prepare these workflows for future policy optimization. Code, traces, and examples are publicly available at https://github.com/idwts/Crayotter.

Abstract

Agentic Video Editing

Explore similar work

Crayotter: Traceable Multi-Agent Workflows for Long-Form Video Editing

VideoAgent: All-in-One Framework for Video Understanding and Editing

VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generation