Organizations: UC Davis Davis, California, USA · Fudan University Shanghai, China · Stanford University Palo Alto, California, USA · University of Rochester Rochester, New York, USA · Meta AI Menlo Park, California, USA
Music-grounded mashup video creation is a challenging form of video non-linear editing, where a system must compose a coherent timeline from large collections of source videos while aligning with music rhythm, user intent, story completeness, and long-range structural constraints. Existing approaches typically rely on fixed pipelines or simplified retrieval-and-concatenation paradigms, limiting their ability to adapt to diverse prompts and heterogeneous source materials. In this paper, we present GLANCE, a global-local coordination multi-agent framework for music-grounded nonlinear video editing. GLANCE adopts a bi-loop architecture for better editing practice: an outer loop performs long-horizon planning and task-graph construction, and an inner loop adopts the "Observe-Think-Act-Verify" flow for segment-wise editing tasks and their refinements. To address the cross-segment and global conflict emerging after subtimelines composition, we introduce a dedicated global-local coordination mechanism with both preventive and corrective components, which includes a novelly designed context controller, conflict region decomposition module, and a bottom-up dynamic negotiation mechanism. To support rigorous evaluation, we construct MVEBench, a new benchmark that factorizes editing difficulty along task type, prompt specificity, and music length, and propose an agent-as-a-judge evaluation framework for scalable multi-dimensional assessment. Experimental results show that GLANCE consistently outperforms prior research baselines and open-source product baselines under the same backbone models. With GPT-4o-mini as the backbone, GLANCE improves over the strongest baseline by 33.2% and 15.6% on two task settings, respectively. Human evaluation further confirms the quality of the generated videos and validates the effectiveness of the proposed evaluation framework.
Figures & tables
Figure 1. The illustration of MVEBench.
Figure 2. Overview of the GLANCE framework.
On-beat Video
Story-driven Video
Method
Rhythm Alignment ↑
Emotion Alignment ↑
Instruction Following ↑
Overall Quality ↑
Story Completeness ↑
Character Continuity ↑
Instruction Following ↑
Overall Quality ↑
LLM-based Agentic Framework
CoT
2.06
1.95
1.98
1.47
2.12
2.05
2.04
1.65
GPTSwarm
2.31
2.22
2.24
1.64
2.34
2.29
2.28
1.95
Multi-modal Video Editing Agentic Framework
TeaserGen
2.01
1.90
1.94
1.76
2.43
2.33
2.34
2.10
Table 1. Overall comparison on the full MVEBench. Judge-based metrics are reported on a 1–5 scale. Best results are in bold ; second-best are underlined. All baselines utilize GPT-4o-mini as the backbone model, except for specific demonstrations for GLANCE . The results are the average score of all types of configuration as described in Section 4 .
Config
#. Samples
Avg. Music Len.
Avg. Source Video Len.
O-Sh-GP
45
27.9s
1.7h
O-Me-GP
35
83.7s
3.1h
O-Me-DP
35
83.7s
3.1h
O-Lo-DP
44
154.2s
5.5h
S-Me-GP
37
85.6s
2.9h
S-Lo-GP
45
182.1s
5.2h
Table 2. Statistics of MVEBench under different task configurations. The unit of music length is second (s), while the unit of video source length is hour (h). “#. Samples” represent the number of tasks for each configuration.
Variant
Preventive
Region Decomp.
Bottom-up Negot.
Rhythm Alignment ↑
Emotion Alignment ↑
Instruction Following ↑
Overall Quality ↑
Efficiency ↑
Only Preventive Controller
✓
3.18
3.07
3.12
3.05
1.18
w/o Bottom-up Negotiation
✓
✓
3.36
3.25
3.30
3.24
1.10
w/o Region Decomposition
✓
✓
3.46
3.35
3.39
3.33
0.76
w/o Preventive Controller
✓
✓
3.58
3.46
3.52
3.40
0.96
Full GLANCE
✓
✓
✓
3.72
3.54
3.66
3.45
1.00
Table 3. Ablation of global–local coordination modules on on-beat mashup video creation. Efficiency is reported as normalized processing efficiency measured by token usage, with higher being better.
Figure 3. Overall quality across different task configurations.
Task Type
Metric
Gemini
Qwen-30B
EditDuet
On-Beat
Rhythm Alignment
3.43
2.91
2.57
On-Beat
Emotion Alignment
3.31
2.78
2.45
On-Beat
Instruction Following
3.38
2.83
2.48
On-Beat
Overall Quality
3.11
2.54
2.29
Story-Dri.
Story Completeness
3.49
2.97
2.68
Story-Dri.
Character Continuity
3.41
2.88
2.53
Table 4. Human evaluation results on sampled On-Beat and Story-Driven subsets.
Task Type
Metric
Spearman ρ↑
Kendall τ↑
On-Beat
Rhythm Alignment
0.64
0.48
On-Beat
Emotion Alignment
0.62
0.46
On-Beat
Instruction Following
0.60
0.47
On-Beat
Overall Quality
0.66
0.50
Story-Driven
Story Completeness
0.71
0.55
Story-Driven
Character Continuity
0.68
0.52
Table 5. Agreement between the proposed judge and human evaluation.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Variant
Preventive
Region Decomp.
Bottom-up Negot.
Story Completeness ↑
Character Continuity ↑
Instruction Following ↑
Overall Quality ↑
Efficiency ↑
Only Preventive Controller
✓
3.20
3.10
3.38
3.07
1.18
w/o Bottom-up Negotiation
✓
✓
3.45
3.37
3.63
3.28
1.10
w/o Region Decomposition
✓
✓
3.56
3.48
3.72
3.34
0.76
w/o Preventive Controller
✓
✓
3.69
3.59
3.82
3.38
0.95
Full GLANCE
✓
✓
✓
3.75
3.68
3.95
3.42
1.00
Appendix
Table 6. Ablation of global–local coordination modules on story-driven mashup video creation. Efficiency is reported as normalized processing efficiency, with higher being better.
Figure 4. Evaluation scores across different task configurations..
School of Electronics Engineering and Computer Science, Peking University · School of Computer Science, Peking University · Huazhong University of Science and Technology