MoCA-Video: Motion-Aware Concept Alignment for Consistent Video Editing

Tong Zhang; Victor Escorcia; Juan C Leon Alcazar; Bernard Ghanem

cs.CVJun 1, 2025

MoCA-Video: Motion-Aware Concept Alignment for Consistent Video Editing

Authors: Tong Zhang, Victor Escorcia, Juan C Leon Alcazar, Bernard Ghanem

Organizations: Department of Computer Science King Abdullah University of Science and Technology Thuwal, Saudi Arabia · Independent Researcher

Abstract

Unlike traditional video editing or inpainting, video semantic mixing fuses a reference concept with a moving target entity to produce a hybrid while preserving the source video's motion and layout. We propose MoCA-Video, a training-free framework that steers a frozen video-diffusion denoising trajectory through concept-localized reference injection. At selected low-noise steps, MoCA-Video uses concept attention to localize the target object and injects the reference latent into the localized region, where object structure has formed but appearance remains editable. A momentum-based correction carries the injected prediction across frames to encourage coherent concept integration through the sequence. We further introduce CASS, a CLIP-based metric that measures the output's directional alignment shift toward the reference and away from the source prompt. Using the denoiser's internal attention avoids an external localization model; in our A100 FP16 setup, MoCA-Video takes 3.2 seconds per output frame, excluding preprocessing. Across the evaluated baselines, MoCA-Video achieves the highest CASS, rel-CASS, and ImageReward, while LPIPS-T and FVD expose separate temporal-coherence and video-quality trade-offs.

Figures & tables

Figure 1: Video semantic mixing with \modelnameabbr . Given a video and reference image, \modelnameabbr blends their concepts across diverse object categories. Prompts are provided in the Supplementary Material.

Figure 2: Overview of \modelnameabbr . Given a video and reference image, we inject $z^{\mathrm{ref}}=E(I^{\mathrm{ref}})$ into concept-attention regions of $\hat{x}_{0}$ during low-noise steps ( $t\leq t^{\prime}$ ) (Sec. 3.4 ), use diagonal scheduling and momentum correction for bounded cross-frame propagation (Sec. 3.5 ), and decode the edited latents with frozen decoder $D$ .

Figure 3: Entity-blending dataset pipeline. CTIB and DAVIS-16 entities are paired to create intra- and inter-category video–image blends. The taxonomy is extensible to custom categories.

	Method	CASS $\uparrow$	rel-CASS $\uparrow$	LPIPS-T $\downarrow$	FVD $\downarrow$	ImageReward $\uparrow$
		Directional alignment		Temporal	Video quality
Pretrained
	AnimateDiffV2V	0.68	-0.41	0.009	1266	0.246
	TokenFlow PnP	2.87	0.02	0.010	4580	-2.087
	TokenFlow SDEdit	1.98	0.05	0.150	4417	-1.431
Training-free

Table 1: Quantitative comparison. We report directional concept alignment (CASS, rel-CASS), temporal coherence (LPIPS-T), and video quality (FVD, ImageReward). Best in bold , second-best underlined . Full-frame structural metrics are excluded, as the task deliberately alters the target object.

Figure 4: Qualitative comparison of video semantic mixing. In this astronaut–cat example, \modelnameabbr forms a visible hybrid: feline features persist across the displayed frames while source spacesuit and scene elements remain. AnimateDiffV2V largely preserves the source, whereas several stronger edits overwrite its structure or introduce artifacts. Red and blue borders mark pretrained and training-free baselines; full sequences appear in the Supplementary Material.

Variant	CASS $\uparrow$	rel-CASS $\uparrow$	LPIPS-T $\downarrow$	FVD $\downarrow$	ImageReward $\uparrow$
MoCA-Video (ours)	7.15	0.135	0.111	3520	0.263
w/o injection	4.74	0.090	0.127	3447	0.256
w/o momentum	5.98	0.108	0.119	3602	0.324
w/o concept attention	5.25	0.098	0.108	3670	0.324

Table 2: Component ablations. Each row disables one mechanism of \modelnameabbr , on the full evaluation set. Defaults:

\lambda=0.8

\tau=0.4

c=2.0

. Best in bold , second-best underlined .

$\lambda$	0.5	0.8 ∗	1.0	1.5	2.0
CASS	2.69	4.09	1.02	0.54	0.40
$\tau$	0.1	0.2	0.3	0.4 ∗	0.5
CASS	0.89	1.60	1.56	4.09	2.37
$c$	off	0.5	1.0	2.0 ∗	5.0
CASS	0.64	0.69	1.33	4.09	0.64

Table 3: Hyperparameter ablation (CASS). Each parameter is varied while the others remain fixed at their default values ( ∗ ). Among the tested values, CASS is highest at

\lambda=0.8

\tau=0.4

, and

c=2.0

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Additional qualitative results on DAVIS-derived pairs. Each row shows an input-video frame (left), reference image (center), and frames from the \modelnameabbr output (right). The examples span intra- and inter-category blends across animals, objects, and human activities.

Figure 6: Additional qualitative results. Examples across diverse concept pairs illustrate how source and reference appearances blend.

Method	CLIP-BS $\uparrow$	DINO-BS $\uparrow$
FreeBlend	6.65	0.27
MoCA-Video	4.00	0.12

Appendix

Table 4: FreeBlend’s blend metrics on 100 test samples. FreeBlend scores higher on both metrics, whereas Fig. 4 and Tab. 1 provide separate qualitative and video-quality comparisons.

Figure 7: Mean ratings from 20 participants, each completing eight trials, for MoCA-Video, AnimateDiffV2V, and FreeBlend+DynamiCrafter. MoCA-Video receives the highest mean Blending Quality and Overall Quality ratings; Video Consistency and Character Consistency are reported as separate subjective measures.

Method	Frames	Time (s/frame)
AnimateDiffV2V	32	3.8
FreeBlend + DynamiCrafter a	16	3.3
TokenFlow (PnP)	16	9.4
TokenFlow (SDEdit)	16	1.9
RAVE	16	3.0
MoCA-Video (ours)	50	3.2

Appendix

Table 5: Per-frame inference time on a single A100 (FP16), excluding preprocessing. Sequence lengths differ, and FreeBlend’s image-mixing time is excluded from its row.

Source prompt	Condition
A surfer riding a wave at sunset	panda
A superhero in a dynamic pose against a city skyline, cape flowing in the wind	deer
A majestic eagle soaring through the sky with its wings spread wide	corgi
A horse galloping across an open meadow	unicorn
A firefighter in full gear, standing in front of a burning building	bear
An astronaut floating in space against a starry background, wearing a detailed white spacesuit	cat

Appendix

Table 6: Source and conditioning prompts for the qualitative results.

Source prompt	Condition
A goat walking on rocks	wolf
A man windsurfing with the sail	dog
A rolling soccer ball	airplane
A man wearing hockey skates	panda
A cow with a bell around its neck	corgi

Appendix

Table 7: Source and conditioning prompts for the DAVIS qualitative results. Source prompts are intentionally minimal to isolate the effect of visual guidance.

Explore similar work

Unified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising
Nov 5, 2025Shuangquan Lyu, Jian Mao, Yue MaText-To-Video Diffusion Models Long Video Generation
Carnegie Mellon University · Jilin University · Tsinghua University
MiVE: Multiscale Vision-language features for reference-guided video Editing
May 14, 2026Tong Wang, Meng Zou, Chengjing Wu +4Video Editing Vision Encoders
MT Lab, Meitu Inc., Beijing 100083, China · Beijing University of Posts and Telecommunications, Beijing 100876, China · Department of Computer Science and Technology, BNRist, IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China
One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing
Sep 3, 2026Adheesh Sunil Juvekar, Onkar Kishor Susladkar, Kiet A. Nguyen +6Video Editing Edit
University of Illinois Urbana-Champaign

Nov 5, 2025, cs.CVJ/K move · Enter open · S save

Unified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising

Shuangquan Lyu, Jian Mao, Yue Ma

Carnegie Mellon University · Jilin University · Tsinghua University

Text-To-Video Diffusion Models Long Video Generation Inpainting Iterative Denoising Methods

Open paper AskPDF arXiv

Diffusion-based text-to-video models are increasingly capable, but mask-based editing over hundreds of frames remains challenging: naïve long-video generation suffers from memory blow-up, window seams, and temporal drift, while existing editors often require specialized modules or heavy fine-tuning. We present Overlapping High-Order Co-Denoising, a lightweight framework that turns a single pre-trained text-to-video model into a unified inpainting-outpainting editor. We train only LoRA adapters using mixed interior and border masks together with a dual-region loss that improves synthesis inside the mask while explicitly preserving known content. At inference, we denoise long latent sequences using overlapping windows, apply second-order Heun sampling within each window, and fuse overlaps with Hamming-weighted blending to reduce boundary artifacts and improve temporal coherence. On InpaintBench (30 real-world videos, 81--300 frames), our method outperforms Wan 2.1 variants and VACE in background faithfulness (SSIM/LPIPS), temporal consistency (tLPIPS), and text alignment (CLIP), and scales to long horizons, demonstrated up to 800 frames, with memory bounded by the chosen window size.

Abstract

Explore similar work

Unified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising

MiVE: Multiscale Vision-language features for reference-guided video Editing

One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing