MotionCoach: Constructing Multimodal Guidance for Video Professional Motion Editing

Siyi Qian; Hanqing Yang; Weihuang Lin; Zihao Yang; Yichen Lu; Yuanhong Zheng; Sheng Li

cs.CVOct 9, 2026

MotionCoach: Constructing Multimodal Guidance for Video Professional Motion Editing

Authors: Siyi Qian, Hanqing Yang, Weihuang Lin, Zihao Yang, Yichen Lu, Yuanhong Zheng, Sheng Li

Organizations: Peking University · Alibaba Token Hub, Alibaba Group · University of Science and Technology of China · Zhejiang University

Abstract

Video editing models achieve strong performance in modifying appearance, backgrounds, styles, and motions, empowered by large-scale training data. Beyond general scenarios, professional motions are significant in performance, sports, and martial arts. This paper focuses on professional motion editing, which involves fine-grained motion technique and temporal structure. Given professional motion names but lacking detailed descriptions, existing models often fail to generate professional motions, such as incorrectly lifting the front wheel under a Stoppie instruction. To address this challenge, we introduce MotionCoach, a trained multimodal agent that uses multi-turn tools and performs self-verification, producing a grounded prompt and a reference clip as reliable and sufficient multimodal guidance to inject external professional motion knowledge into video editing models. To supply training data, we build a data pipeline to construct trajectories. We cold-start the agent on MotionCoach-SFT-10k and apply Group Relative Policy Optimization (GRPO) on MotionCoach-RL-2k with rewards on the motion guidance prompt, the motion outcome video, and the format. For evaluation, we introduce VPMEBench, comprising 314 professional motion editing tasks with fine-grained references and annotations. Extensive experiments show that MotionCoach improves the overall score from 55.1 to 65.6 on Seedance 2.0 and from 51.2 to 64.5 on MiniMax-H3. Learning to construct multimodal guidance enables precise professional motion editing without retraining video editing models.

Figures & tables

Figure 1: Motivation for professional motion editing. Prompting Seedance 2.0 [ Team Seedance et al., 2026 ] with Change the rider’s motion to Stoppie yields a Wheelie (wrong wheel lifted). MotionCoach instead supplies a grounded prompt and reference clip specifying the correct technique.

Figure 2: Illustration of the data pipeline. (1) Data preparation prepares source videos and edit prompts. (2) Agentic trajectory construction acquires and verifies motion references to produce MotionCoach-SFT-10k and MotionCoach-RL-2k. (3) Human-verified benchmark construction establishes VPMEBench.

Figure 3: Overview of VPMEBench . 314 professional motion editing tasks under a hierarchical taxonomy over dance, sports, and martial arts. Each task provides visual references and textual annotations of the target motion for evaluation along motion professionalism (MP), physical plausibility (PP), and overall video quality (VQ).

Figure 4: Overview of MotionCoach . Top: the multi-turn agentic loop during rollout and inference uses search, inspect, and clip tools with self-verification to produce reliable and sufficient motion guidance. Bottom: Agentic RL updates the policy with motion guidance prompt, motion outcome video, and format rewards while keeping the video editing model frozen.

Open-source models
Model	Dance			Sports			Martial Arts			All			Overall
	MP	PP	VQ	MP	PP	VQ	MP	PP	VQ	MP	PP	VQ
Ditto	0.7	\cellcolor rankthirdblue88.2	59.5	0.9	\cellcolor rankfirstblue 94.1	79.1	0.0	\cellcolor ranksecondblue88.3	63.9	0.6	\cellcolor ranksecondblue90.3	67.2	38.5
Kiwi-Edit	1.7	61.8	72.6	0.9	81.4	\cellcolor rankthirdblue91.2	0.0	72.5	79.7	1.1	70.7	80.5	38.8
InsV2V	0.3	82.6	78.1	0.0	87.7	88.9	0.0	\cellcolor rankthirdblue85.8	85.8	0.2	85.0	83.4	42.1
Lucy-Edit-Dev	0.0	85.8	86.3	0.0	\cellcolor rankthirdblue88.2	87.0	0.0	85.0	84.2	0.0	86.5	86.1	43.1

Table 1: Professional motion editing on VPMEBench . Existing video editing models use short edit prompts. We evaluate MotionCoach on frozen MiniMax-H3 and Seedance 2.0, where the agent provides multimodal guidance for each model.

Table 5: Policy and tool execution, in percent.

Figure 5: Qualitative results of MotionCoach.

	Total	Metric Performance
Model	Total	SC	BC	TF	MS	VTSS	OSC	PSC	IS	SF	CF
MiniMax-H3	48.0	90.9	95.3	96.5	98.9	6.4	19.3	19.1	2.73	69.5	1.40
MiniMax-H3 + Ours	53.8	92.3	95.2	97.5	99.0	6.1	24.5	19.2	2.62	95.3	2.05

Table 6: Generalization on the subject-motion editing subset of IVEBench [ Chen et al., 2026b ] .

Table 9: Agreement between the automatic evaluation and human judgments.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Representative VPMEBench tasks.

Panel of Figure 7 — Figure 7: Additional qualitative results of MotionCoach on VPMEBench. MotionCoach (blue) is compared across dance, sports, and martial arts; the first eight are success cases, while the last two (gray) are failure cases in challenging settings.

Panel of Table 10 — Table 10: Comparison with representative video-editing benchmarks. None indicates that motion-editing tasks are included without target-motion references or annotations. A dash indicates that no standalone motion-editing split is defined. Coarse denotes general pose or action descriptions without named atomic-motion labels or phase-level annotations.

Figure 8: A trajectory of the MotionCoach multi-turn agent loop. The agent calls search, inspect, and clip, each turn follows the think, tool call, and tool response format, and the final answer gives the grounded prompt and the verified reference clip.

Configuration	Value
Base model	Qwen3-VL-8B-Instruct
Training data	MotionCoach-SFT-10k
Update mode	Full-parameter fine-tuning
Hardware	4 nodes $\times$ 8 GPUs
Framework	LLaMA-Factory with DeepSpeed ZeRO-3
Optimizer	Fused AdamW ( $\beta_{1}=0.9$ , $\beta_{2}=0.999$ , $\epsilon=10^{-8}$ )

Appendix

Table 11: Main hyperparameters for supervised fine-tuning.

Configuration	Value
Initialization	Final SFT checkpoint
Training data	MotionCoach-RL-2k
Algorithm	GRPO; full-parameter FSDP training
Framework	verl with Ray orchestration
Policy hardware	2 nodes $\times$ 8 GPUs
Video-generation hardware	1 node $\times$ 8 GPUs

Appendix

Table 12: Main hyperparameters and system configuration for agentic RL.

Video Editing

Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.

See all 178 papers

ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos
Apr 12, 2026Arjun Somayazulu, Kristen GraumanVideo Editing
The University of Texas at Austin, USA
CoVEBench: Can Video Editing Models Handle Complex Instructions?
Jun 7, 2026Jiangtao Wu, Jiaming Wang, Yiwen He +7Video Editing Benchmarks Video Editing
NJU-LINK Team, Nanjing University · Kling Team, Kuaishou Technology
Sound Sparks Motion: Audio and Text Tuning for Video Editing
May 14, 2026AmirHossein Naghi Razlighi, Aryan Mikaeili, Ali Mahdavi-Amiri +2Audio-Video Generation Inference-Time Optimization
University of Cyprus, Cyprus · Simon Fraser University, Canada · Tel Aviv University, Israel +1

Apr 12, 2026J/K move · Enter open · S save

ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos

Arjun Somayazulu, Kristen Grauman

The University of Texas at Austin, USA

Video Editing

Open paper AskPDF arXiv

Visual feedback is critical for motor skill acquisition in sports and rehabilitation, and psychological studies show that observing near-perfect versions of one's own performance accelerates learning more effectively than watching expert demonstrations alone. We propose to enable such personalized feedback by automatically editing a person's motion to reflect higher skill. Existing motion editing approaches are poorly suited for this setting because they assume paired input-output data -- rare and expensive to curate for skill-driven tasks -- and explicit edit guidance at inference. We introduce ExpertEdit, a framework for skill-driven motion editing trained exclusively on unpaired expert video demonstrations. ExpertEdit learns an expert motion prior with a masked language modeling objective that infills masked motion spans with expert-level refinements. At inference, novice motion is masked at skill-critical moments and projected into the learned expert manifold, producing localized skill improvements without paired supervision or manual edit guidance. Across eight diverse techniques and three sports from Ego-Exo4D and Karate Kyokushin, ExpertEdit outperforms state-of-the-art supervised motion editing methods on multiple metrics of motion realism and expert quality. Project page: https://vision.cs.utexas.edu/projects/expert_edit/ .

Abstract

Video Editing

Explore similar work

ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos

CoVEBench: Can Video Editing Models Handle Complex Instructions?

Sound Sparks Motion: Audio and Text Tuning for Video Editing