Regional Anchor Routing for Multi-Object Motion Transfer

Yuze Li; Fuchen Long; Xinlong Sun; Cheng Yan; Xinyu Zhang; Xi Chen

cs.CVOct 29, 2026

Regional Anchor Routing for Multi-Object Motion Transfer

Authors: Yuze Li, Fuchen Long, Xinlong Sun, Cheng Yan, Xinyu Zhang, Xi Chen

Organizations: Tianjin University · Smart Creation Platform Department, Online Video BU, Tencent · University of Auckland

Abstract

Compositional motion transfer requires a model to extract motions from multiple reference videos, and apply each motion to a user-specified object in one target scene. Existing methods normally struggle to reliably assign different reference motions to the correct target objects, thus a motion intended for one object may affect another. Meanwhile, some motion-specific tuning approaches are still facing another issue: they must run a new optimization whenever the reference video changes, making repeated transfers inefficient. To address these limitations, we propose RART, a framework that unifies object-wise motion extraction, target-region anchoring, and frame-aligned injection into a frozen video Diffusion Transformer for multi-object motion transfer. For each reference motion, RART extracts temporally ordered features from the masked reference video, and then conditions them on the target first-frame mask to form mask-anchored motion tokens. Frame-aligned motion cross-attention allows each generated frame to read only the motion tokens at the corresponding time slots, composing all object motions in one denoising process. During training, a motion-attention routing loss encourages each target region to attend to its corresponding motion tokens and reduces interference between objects. Therefore, RART can handle novel reference-target combinations without per-video optimization. Experiments show that RART achieves accurate motion preservation, efficient inference, and strong compositional generalization.

Figures & tables

Figure 1: Compositional motion transfer with RART. Masked reference videos provide object-wise motions, which are assigned to colored target regions and jointly rendered in the generated video. The examples show (a) and (b) motion reassignment, (c) cross-species transfer, and (d) multi-object composition.

Figure 2: Overview of RART. (a) The motion encoder extracts temporally ordered features from each masked reference video and injects the corresponding target-mask anchor to produce object-wise motion tokens. (b) Frame-aligned motion cross-attention injects the motion tokens of each latent frame into the frozen DiT.

Figure 3: Motion-Attention Routing Loss. For each object, RART encourages its target region to attend to the corresponding motion tokens while discouraging other object regions from attending to the same token block.

Method	TS $\uparrow$	TC $\uparrow$	AC $\uparrow$	FF $\uparrow$	MF $\uparrow$	App. $\uparrow$	Temp. $\uparrow$	Motion $\uparrow$	Time $\downarrow$
	Automatic Metrics					Human Evaluations (%)
AnyV2V	0.261	0.879	0.742	0.571	0.338	0.0	0.0	0.0	176s/ref + 50s
DisMo	0.266	0.944	0.913	0.652	0.451	3.0	3.0	2.6	113s
SCAIL-2	0.266	0.922	0.906	0.648	0.428	15.8	9.0	24.1	233s
FlexiMMT	0.267	0.925	0.899	0.671	0.462	14.7	27.1	21.8	2h/ref + 432s
RART	0.265	0.928	0.903	0.713	0.480	66.5	60.9	51.5	41s

Table 1: Quantitative and human evaluations. TS, TC, AC, FF, and MF denote text similarity, temporal consistency, appearance consistency, flow fidelity, and motion fidelity, respectively. Human evaluation columns report five-way selection rates (%), while Time denotes runtime. For AnyV2V and FlexiMMT, the first time term denotes per-reference preparation.

Figure 4: Qualitative comparison. The top row shows the reference motions, target image, and motion-to-object assignments; the remaining rows show frames generated by the baselines and RART. RART transfers each reference motion to its designated target object while preserving target appearance.

Figure 5: Qualitative ablation. The top row shows reference motions and their assignments to target regions; the bottom panels compare RART with variants without frame alignment, target-mask anchoring, or routing loss.

Figure 6: Motion-attention heat maps. Columns correspond to the two reference motions, while rows compare the model without the routing loss and RART. Warmer colors indicate stronger attention to each motion-token block.

Variant	FF $\uparrow$	MF $\uparrow$
w/o Frame Alignment	0.678	0.400
w/o Mask Anchor	0.690	0.413
w/o Routing Loss	0.727	0.448
RART	0.713	0.480

Table 2: Quantitative ablations. We replace frame-aligned motion attention with global attention, use all-ones target masks, or remove the motion-attention routing loss.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	Value
Video backbone	Wan2.2 TI2V-5B
Motion extractor	DisMo Encoder
Training steps	200K
Learning rate	$1\times 10^{-5}$
Hardware	8 $\times$ NVIDIA H200 GPUs
Batch size per GPU	1

Appendix

Table 3: Training configuration of the reported RART model.

Figure 7: Additional qualitative results for compositional motion transfer. Each example shows color-coded reference sequences at the top, followed by the target-object assignments and generated frames at the bottom. Blue and red indicate the correspondence between each reference motion and its target object.

Figure 8: Further qualitative results for compositional motion transfer. RART jointly transfers independently specified reference motions to their color-matched target objects while preserving the appearance and scene of the target image.

Controllable Video Generation

Controllable video generation steers how videos look, move, and change over time using user-provided instructions and structured controls.

See all 324 papers

Let Your Image Move with Your Motion! -- Implicit Multi-Object Multi-Motion Transfer
Mar 1, 2026Yuze Li, Dong Gong, Xiao Cao +6Controllable Video Generation Image-to-Video Generation
Tianjin University · University of New South Wales · University of Electronic Science and Technology of China +2
MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer
Apr 1, 2026Samuel Teodoro, Yun Chen, Agus Gunawan +3Controllable Video Generation Video Diffusion Models
School of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South Korea · Adobe Research, San Jose, California · CMLab, Chung-Ang University, Seoul, South Korea
FlowMotion: Training-Free Flow Guidance for Video Motion Transfer
Mar 6, 2026Zhen Wang, Youcan Xu, Jun Xiao +1Controllable Video Generation Temporal Consistency in Video Generation
The Hong Kong University of Science and Technology · State key lab of CAD&CG, Zhejiang University

Mar 1, 2026J/K move · Enter open · S save

Let Your Image Move with Your Motion! -- Implicit Multi-Object Multi-Motion Transfer

Yuze Li, Dong Gong, Xiao Cao, Junchao Yuan+5

Tianjin University · University of New South Wales · University of Electronic Science and Technology of China+2

Controllable Video Generation Image-to-Video Generation Video Diffusion Models Representation Disentanglement

Open paper AskPDF arXiv

Motion transfer has emerged as a promising direction for controllable video generation, yet existing methods largely focus on single-object scenarios and struggle when multiple objects require distinct motion patterns. In this work, we present FlexiMMT, the first implicit image-to-video (I2V) motion transfer framework that explicitly enables multi-object, multi-motion transfer. Given a static multi-object image and multiple reference videos, FlexiMMT independently extracts motion representations and accurately assigns them to different objects, supporting flexible recombination and arbitrary motion-to-object mappings. To address the core challenge of cross-object motion entanglement, we introduce a Motion Decoupled Mask Attention Mechanism that uses object-specific masks to constrain attention, ensuring that motion and text tokens only influence their designated regions. We further propose a Differentiated Mask Propagation Mechanism that derives object-specific masks directly from diffusion attention and progressively propagates them across frames efficiently. Extensive experiments demonstrate that FlexiMMT achieves precise, compositional, and state-of-the-art performance in I2V-based multi-object multi-motion transfer. Our project page is: https://ethan-li123.github.io/FlexiMMT_page/

Abstract

Controllable Video Generation

Explore similar work

Let Your Image Move with Your Motion! -- Implicit Multi-Object Multi-Motion Transfer

MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer

FlowMotion: Training-Free Flow Guidance for Video Motion Transfer