Editing the video content with audio alignment forms a digital human-made art in current social media. However, the time-consuming and repetitive nature of manual video editing has long been a challenge for filmmakers and professional content creators alike. In this paper, we introduce CutClaw, an autonomous multi-agent framework designed to edit hours-long raw footage into meaningful short videos that leverages the capabilities of multiple Multimodal Language Models~(MLLMs) as an agent system. It produces videos with synchronized music, followed by instructions, and a visually appealing appearance. In detail, our approach begins by employing a hierarchical multimodal decomposition that captures both fine-grained details and global structures across visual and audio footage. Then, to ensure narrative consistency, a Playwriter Agent orchestrates the whole storytelling flow and structures the long-term narrative, anchoring visual scenes to musical shifts. Finally, to construct a short edited video, Editor and Reviewer Agents collaboratively optimize the final cut via selecting fine-grained visual content based on rigorous aesthetic and semantic criteria. We conduct detailed experiments to demonstrate that CutClaw significantly outperforms state-of-the-art baselines in generating high-quality, rhythm-aligned videos. The code is available at: https://github.com/GVCLab/CutClaw.
Figures & tables
Figure 1: We present an automated music-driven video editing system that transforms hours-long footage into high-quality short videos based on user instructions and given music rhythm, so that the resulting video demonstrates precise music synchronization, faithful instruction following, and visually appealing aesthetics.
Figure 2: The whole workflow of the CutClaw. The multi-modal footage is first Deconstructed , and then, the shot plan is generated by the Playwriter , scene retrieval and editing by the Editor , and quality validation by the Reviewer .
Figure 3: Left: Video Shots Aggregation. We first perform shot detection on the entire video and conduct a caption for a detailed understanding. Then, we use an LLM to aggregate similar content for a scene-level description. Right: The workflow of Playwriter. Playwriter generate the whole storyline according to the input, and gives the detailed shot plan of each specific shot of music.
Figure 4: Editor and Reviewer are used to perform segment selection and validation. SNR stands for Semantic Neighborhood Retrieval, and FGST stands for Fine-Grained Shot Trimming.
Figure 5: A sample execution of a single-shot cutting , utilizing footage from movie “Interstellar” and the music “Moon.” Actions performed by the Playwright, Editor, and Reviewer are color-coded in blue , yellow , and green , respectively. The orange background traces the execution path leading to the final clip selection.
Table 6
Figure 6: Qualitative comparison between CutClaw and baseline methods. The two cases utilize full-length footage from the films “Paprika” and “La La Land”, paired with the musical tracks “Luv(sic) Pt.2” and “Norman F**king Rockwell”, respectively. Shot boundary detection is performed using PySceneDetect [ 6 ] .
Method
Visual Quality
Instruction Follow
Audio-Visual Harmony
Human-Like
Nar
Obj
Film
Vlog
Avg
Nar
Obj
Film
Vlog
Avg
Nar
Obj
Film
Vlog
Avg
Nar
Obj
Film
Vlog
Avg
NarratoAI*
11.6%
11.2%
27.3%
-
11.4%
14.8%
10.8%
28.7%
-
12.8%
11.2%
12.4%
25.3%
-
11.8%
12.0%
8.4%
23.3%
-
10.2%
UVCOM
18.0%
16.8%
7.3%
23.6%
17.4%
18.8%
13.2%
7.3%
22.0%
16.0%
17.2%
13.2%
6.0%
22.8%
15.2%
18.8%
15.6%
9.3%
23.6%
17.2%
Time-R1
25.2%
17.6%
18.0%
22.4%
21.4%
22.4%
19.6%
16.7%
24.0%
21.0%
23.2%
16.8%
17.3%
18.4%
20.0%
24.8%
22.8%
16.7%
25.6%
23.8%
CutClaw
45.2%
54.4%
47.3%
54.0%
49.8%
44.0%
56.4%
47.3%
53.6%
50.2%
48.4%
57.6%
51.3%
57.6%
53.0%
44.4%
53.2%
50.7%
50.4%
48.8%
Table 3: User Study Results . We report the percentage of user votes across four metrics: Visual Quality, Instruction Following, Audio-Visual Harmony, and Human-Likeness. For each metric, we break down performance by instruction type (Narrative/Object) and video type (Film/Vlog). Avg. denotes the average score. Our method outperforms baselines across all categories. * Note that NarratoAI cannot deal with the VLOG as it does not have dense subtitles.
Video editing has become essential in digital media creation, yet existing automated systems are restricted to short segment processing and domain-specific tasks. They face two critical limitations: i) inability to handle diverse video comprehension and editing operations, and ii) lack of long-video understanding for coherent narrative creation. We propose VideoAgent, an all-in-one agentic framework addressing these challenges through two key innovations. First, we develop automated video shot creation with shot planning agents for coherent narratives and cross-modal retrieval for aligned visual content. Second, we design a multi-agent orchestration framework integrating over thirty specialized editing agents. Intent parsing filters relevant tools while textual-gradient graph optimization assembles complex editing pipelines. Extensive experiments on our newly-proposed VideoEdit benchmark and public datasets demonstrate VideoAgent's superiority over existing multimodal LLMs and agentic systems. VideoAgent achieves 87-95% orchestration success rates while reducing API costs by 60%. Human evaluation across six video categories shows VideoAgent produces professional-quality content approaching human-level performance, with ratings only 4% below human-created videos. We release our code at https://github.com/HKUDS/VideoAgent.
Hengji Zhou, Lingxuan Huang, Jian Wang +4
1Harbin Institute of Technology, Shenzhen · 2South China University of Technology · 3The University of Hong Kong +1
Autonomous video editing from open-world resources remains a challenging problem in multimedia content creation. Retrieval-based methods are constrained by fixed source material and lack creative flexibility, while generative models struggle with long-horizon coherence and often diverge from user intent. Both paradigms fail to replicate the structured, multi-stage reasoning that underlies professional post-production. To address these limitations, we present Auto-Director, a tool-augmented multi-agent framework that mirrors professional editorial workflows: a Planning Agent decomposes abstract topics into fact-grounded scripts via retrieval-augmented planning; a Material Agent indexes open-world footage through shot segmentation and visual tagging; an Editor Agent assembles semantically aligned clips with MLLM-driven spatial adaptation; and a Critic Agent governs an iterative Reflect-and-Revise loop that checks scene-level alignment and pacing. To rigorously evaluate this new paradigm, we introduce VEdit-Bench, a benchmark of 100 expert-curated topics with human-annotated Golden Scripts and a five-dimensional metric suite covering Narrative Logic, Factual Accuracy, Semantic Alignment, Compositional Quality, and Stylistic Cohesion. Extensive experiments demonstrate that Auto-Director significantly outperforms baseline methods in both automated metrics and human evaluation, effectively bridging the gap between amateurs and professional-grade video production.
Yuxia Chen, Wenbo Zhu, Yangguang Ji +3
University of Electronic Science and Technology of China, China · University of California, Berkeley · Nanyang Technological University +3
Editing a long-form video from heterogeneous footage requires more than selecting clips: an agent must preserve narrative intent across material preparation, timeline construction, post-production, and revision while leaving enough evidence to diagnose failures. We present Crayotter, an open-source multimodal multi-agent system for prompt-driven video editing. Crayotter organizes production into three phases: coverage-aware material preparation, artifact-based editing research, and tool-grounded timeline execution. Each phase externalizes inspectable artifacts, including coverage reports, multimodal analyses, editing blueprints, tool calls, and intermediate renders. These artifacts make an editing run traceable and allow failed segments to be diagnosed and selectively revised instead of requiring a full restart. We evaluate Crayotter on 23 editing themes against CapCut-Mate and CutClaw. Under human evaluation, Crayotter achieves an average score of 3.40/5, compared with 2.44 and 1.70 for the two baselines, with consistent gains in theme alignment, narrative coherence, and editing smoothness. We additionally describe a replayable trajectory schema and verifiable reward design that prepare these workflows for future policy optimization. Code, traces, and examples are publicly available at https://github.com/idwts/Crayotter.