ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos

Arjun Somayazulu; Kristen Grauman

cs.CVApr 12, 2026

ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos

Authors: Arjun Somayazulu, Kristen Grauman

Organizations: The University of Texas at Austin, USA

Abstract

Visual feedback is critical for motor skill acquisition in sports and rehabilitation, and psychological studies show that observing near-perfect versions of one's own performance accelerates learning more effectively than watching expert demonstrations alone. We propose to enable such personalized feedback by automatically editing a person's motion to reflect higher skill. Existing motion editing approaches are poorly suited for this setting because they assume paired input-output data -- rare and expensive to curate for skill-driven tasks -- and explicit edit guidance at inference. We introduce ExpertEdit, a framework for skill-driven motion editing trained exclusively on unpaired expert video demonstrations. ExpertEdit learns an expert motion prior with a masked language modeling objective that infills masked motion spans with expert-level refinements. At inference, novice motion is masked at skill-critical moments and projected into the learned expert manifold, producing localized skill improvements without paired supervision or manual edit guidance. Across eight diverse techniques and three sports from Ego-Exo4D and Karate Kyokushin, ExpertEdit outperforms state-of-the-art supervised motion editing methods on multiple metrics of motion realism and expert quality. Project page: https://vision.cs.utexas.edu/projects/expert_edit/ .

Figures & tables

Figure 1: Skill-driven motion editing. Given a 3D motion sequence extracted from a novice activity video, ExpertEdit produces personalized skill edits by refining poses within regions where skill differences are pronounced. Our model tweaks them to exhibit expert-like precision and form, while preserving the original execution’s motion path and body orientation, as well as its source poses at all non-skill-critical moments. We learn to perform these edits solely from expert videos, without paired supervision or heavy edit guidance from text or reference motions—whether during inference or training. Preserved source poses are shown in blue; edited poses are shown in orange.

Figure 2: ExpertEdit approach. We tokenize expert pose motion sequences and mask the key action phase as determined by task-specific kinematic criteria. We train a bidirectional transformer, MotionInfiller , to predict the expert pose tokens at the masked positions. At inference, we mask skill-critical action phases in a novice motion (see Sec. 4 ) and infill these regions with expert-like motion.

Figure 3: Example novice–expert pseudo-pair sequence in our testbed. See text.

Figure 4: ExpertEdit sequence visualization: We show novice source pose (blue) and edited pose (orange) at several frames for all techniques. ExpertEdit makes subtle pose refinements that improve form at skill-critical action moments, including raising the knee on the shooting hand-side higher during layups (Mikan, reverse), extending legs further on kicks (spin back, roundhouse), moving the shooting hand under the ball during jumpshots, and improving follow-through from the kicking leg on penalty kicks.

	Basketball and Soccer
	Mikan		Rev. Layup		Jumpshot		Pen. Kick
Method	$P(\uparrow$ )	$F(\uparrow$ )	$P(\uparrow$ )	$F(\uparrow$ )	$P(\uparrow$ )	$F(\uparrow$ )	$P(\uparrow$ )	$F(\uparrow$ )
SimMotionEdit [ 32 ]	2.26	1.28	2.31	4.88	3.95	1.24	3.20	2.38
TMED [ 5 ]	1.87	1.00	2.20	4.30	2.58	2.03	2.57	1.95
FLAME* [ 30 ]	2.35	1.45	2.09	6.13	3.13	1.54	2.90	2.22
ExpertEdit* (Ours)	6.18	5.72	5.87	12.08	5.34	7.66	6.03	9.14

Table 1: Results on four basketball and soccer techniques .

P

and

F

represent relative improvement in PA-MPJPE and alignment with the expert distribution respectively over the source motion. Higher is better for both (

\uparrow

).*Indicates no access to paired supervision for training. PA-MPJPE is averaged over

k=3

expert reference pairs to account for natural variation in expert behavior. See Supp. for absolute metrics.

	Kyokushin Karate
	Rev. Punch		Spin BK		Front		High RH
Method	$P(\uparrow$ )	$F(\uparrow$ )	$P(\uparrow$ )	$F(\uparrow$ )	$P(\uparrow$ )	$F(\uparrow$ )	$P(\uparrow$ )	$F(\uparrow$ )
SimMotionEdit [ 32 ]	2.20	1.30	1.43	3.30	1.45	5.05	3.30	5.88
TMED [ 5 ]	0.92	1.08	0.86	2.25	0.38	2.90	2.25	3.37
ExpertEdit (Ours)	2.07	1.36	1.79	4.23	1.88	9.73	2.96	6.18

Table 2: Results on Kyokushin Karate Dataset, four techniques .

P

denotes relative improvement in PA-MPJPE.

F

denotes relative improvement in FID over novice motion. Higher is better for both (

\uparrow

Preferred edit	Basketball (78)	Soccer (34)	Overall (112)
ExpertEdit	42.3%	47.1%	43.8%
SimMotionEdit [ 32 ]	26.9%	23.5%	25.9%
Neither	30.8%	29.4%	30.4%
ExpertEdit among non-neutral	61.1%	66.7%	62.8%

Table 3: Blinded human preference study. Participants compare anonymized ExpertEdit and SimMotionEdit (strongest baseline) rendered edits and choose which would be more useful for improving form.

Table of Contents
1) Dataset preprocessing and technique clip extraction pipeline .	Sec. F
2) Details of the procedure for constructing test set pseudo-pairs .	Sec. G
3) Additional model architecture and training details .	Sec. H
4) Performance vs. train set size scaling analysis .	Sec. I
5) Experiments with different text prompts for baselines .	Sec. J
6) Ablation on mask span fraction $\alpha$ .	Sec. K

Table 8

Figure 5: ExpertEdit performance as a function of training data. We report Pose Improvement ( $P$ ) and FID Improvement ( $F$ ) metrics averaged across all techniques as a function of train set size. ExpertEdit performance scales well as it observes more unpaired expert video during training.

	Basketball and Soccer
	Mikan		Rev. Layup		Jumpshot		Pen. Kick
Prompt	$P(\uparrow)$	$F(\uparrow)$	$P(\uparrow)$	$F(\uparrow)$	$P(\uparrow)$	$F(\uparrow)$	$P(\uparrow)$	$F(\uparrow)$
Expert-like	1.44	0.96	0.97	0.81	1.20	0.93	1.04	1.12
Smooth	2.01	1.11	1.92	2.90	3.12	1.18	1.74	1.33
Smooth + Technique	2.26	1.28	2.31	4.88	3.95	1.24	3.20	2.38

Table 3: Effect of different text-edit prompts on baseline motion editing performance . We train and evaluate a representative motion-editing baseline [ 32 ] with different text-edit prompts.

P

and

F

denote relative improvement in PA-MPJPE and alignment with the expert distribution, respectively, over the source motion. Prompts encouraging general improvements in smoothness and control led to better performance than explicitly requesting expert-like motion, and adding the technique name to the prompt led to further gains.

Variant	Avg. $P\uparrow$	Avg. $F\uparrow$
$\alpha=0.15$	4.09	6.10
$\alpha=0.3$ (default)	4.02	7.01
$\alpha=0.45$	3.88	6.41

Table 4: Mask-span ablation. Average Pose Improvement (

P

) and FID Improvement (

F

) across techniques.

Method	PA-MPJPE $\downarrow$	FID $\downarrow$
Source novice	204.0	220.5
SimMotionEdit	198.9	213.5
ExpertEdit	195.8	205.1

Table 5: Absolute metrics, averaged across techniques vs. strongest baseline SimMotionEdit. ExpertEdit achieves the lowest raw PA-MPJPE and FID, confirming that the relative improvement metrics in the main paper are consistent with absolute metric values.

Video Editing

Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.

See all 178 papers

MotionCoach: Constructing Multimodal Guidance for Video Professional Motion Editing
Oct 9, 2026Siyi Qian, Hanqing Yang, Weihuang Lin +4Video Editing
Peking University · Alibaba Token Hub, Alibaba Group · University of Science and Technology of China +1
AIDE: Automated Instruction via Distilled Expertise for Reference-Free Motor Skill Coaching
Aug 4, 2026Yoshiki ItoText Generation
Research & Development Group Hitachi, Ltd. Tokyo, Japan
SportSkills: Physical Skill Learning from Sports Instructional Videos
Mar 26, 2026Kumar Ashutosh, Chi Hsuan Wu, Kristen GraumanSports Video Analysis Video Representation Learning
University of Texas at Austin

Oct 9, 2026J/K move · Enter open · S save

MotionCoach: Constructing Multimodal Guidance for Video Professional Motion Editing

Siyi Qian, Hanqing Yang, Weihuang Lin, Zihao Yang+3

Peking University · Alibaba Token Hub, Alibaba Group · University of Science and Technology of China+1

Video Editing

Open paper AskPDF arXiv

Video editing models achieve strong performance in modifying appearance, backgrounds, styles, and motions, empowered by large-scale training data. Beyond general scenarios, professional motions are significant in performance, sports, and martial arts. This paper focuses on professional motion editing, which involves fine-grained motion technique and temporal structure. Given professional motion names but lacking detailed descriptions, existing models often fail to generate professional motions, such as incorrectly lifting the front wheel under a Stoppie instruction. To address this challenge, we introduce MotionCoach, a trained multimodal agent that uses multi-turn tools and performs self-verification, producing a grounded prompt and a reference clip as reliable and sufficient multimodal guidance to inject external professional motion knowledge into video editing models. To supply training data, we build a data pipeline to construct trajectories. We cold-start the agent on MotionCoach-SFT-10k and apply Group Relative Policy Optimization (GRPO) on MotionCoach-RL-2k with rewards on the motion guidance prompt, the motion outcome video, and the format. For evaluation, we introduce VPMEBench, comprising 314 professional motion editing tasks with fine-grained references and annotations. Extensive experiments show that MotionCoach improves the overall score from 55.1 to 65.6 on Seedance 2.0 and from 51.2 to 64.5 on MiniMax-H3. Learning to construct multimodal guidance enables precise professional motion editing without retraining video editing models.

Abstract

Video Editing

Explore similar work

MotionCoach: Constructing Multimodal Guidance for Video Professional Motion Editing

AIDE: Automated Instruction via Distilled Expertise for Reference-Free Motor Skill Coaching

SportSkills: Physical Skill Learning from Sports Instructional Videos