Search2Motion: Training-Free Object-Level Motion Control via Attention-Consensus Search

Sainan Liu; Tz-Ying Wu; Hector A Valdez; Subarna Tripathi

cs.CVMar 17, 2026

Search2Motion: Training-Free Object-Level Motion Control via Attention-Consensus Search

Authors: Sainan Liu, Tz-Ying Wu, Hector A Valdez, Subarna Tripathi

Organizations: Intel Corporation

Abstract

We present Search2Motion, a training-free framework for object-level motion editing in image-to-video generation. Unlike prior methods requiring trajectories, bounding boxes, masks, or motion fields, Search2Motion adopts target-frame-based control, leveraging first-last-frame motion priors to realize object relocation while preserving scene stability without fine-tuning. Reliable target-frame construction is achieved through semantic-guided object insertion and robust background inpainting. We further show that early-step self-attention maps predict object and camera dynamics, offering interpretable user feedback and motivating ACE-Seed (Attention Consensus for Early-step Seed selection), a lightweight search strategy that improves motion fidelity without look-ahead sampling or external evaluators. Noting that existing benchmarks conflate object and camera motion, we introduce S2M-DAVIS and S2M-OMB for stable-camera, object-only evaluation, alongside FLF2V-obj metrics that isolate object artifacts without requiring ground-truth trajectories. Search2Motion consistently outperforms baselines on FLF2V-obj and VBench.

Figures & tables

Figure 1: Left: Search2Motion is a training-free pipeline for object-level motion editing. Given a single image and a user-specified target location, Search2Motion constructs a target frame and leverages pretrained FLF2V motion priors to synthesize realistic object motion, without retraining or auxiliary control signals. Right: Sample pairs from the Search2Motion Benchmark, two stable-camera datasets for object-only motion evaluation.

Figure 2: The Search2Motion Pipeline is constructed with three components, where the user can interact with the application at the target frame construction stage (Background Inpainting and Object Placement). Then the original input image and the user-edited last frame are sent to a first-frame last-frame (FLF2V) video generator to generate the final video based on the given input image and user preference, while utilizing ACE-Seed , a novel search criterion in the noise space based on attention-consensus, to improve the generated video quality automatically.

Table 5

Dataset	Method	ObjMC $\downarrow$	CA-IoU $\uparrow$
S2M-OMB	DragAnything [ 30 ]	58.12	0.5619
	TTM - Wan2.2-5B [ 25 ]	175.02	0.4460
	Search2Motion - Wan2.2-5B	9.96	0.8818
S2M-DAVIS	DragAnything [ 30 ]	36.61	0.4290
	TTM - Wan2.2-5B [ 25 ]	105.79	0.3686
	Search2Motion - Wan2.2-5B	3.72	0.8211

Table 3: Object center distance (ObjMC) and center-aligned object mask IoU (CA-IoU) between the penultimate frame in the generated video and the given target frame.

Panel of Figure 7 — Figure 7: Qualitative results of Search2Motion video generation comparing against DragAnything and TTM on S2M-OMB (top two) and S2M-DAVIS (bottom).

Dataset	Seed Selection	Subject $\uparrow$	Background $\uparrow$	Temporal $\uparrow$	Motion $\uparrow$	DINOv2 $\uparrow$	DINOv2 $\uparrow$	LPIPS $\downarrow$	LPIPS $\downarrow$
		Consistency	Consistency	Flickering	Smoothness		first frame		first frame
S2M-OMB	none $\dagger$	95.14	95.88	98.68	99.30	96.99	82.60	.0435	.2207
	BANSA [ 11 ]	94.81	95.49	98.36	99.25	97.03	83.02	.0446	.2177
	ACE-Seed	95.19	96.07	99.00	99.45	97.09	84.31	.0419	.2075
S2M-DAVIS	none $\dagger$	96.45	96.52	99.30	99.57	95.70	78.31	.0637	.2739
	BANSA [ 11 ]	95.85	96.10	98.85	99.45	95.87	77.87	.0568	.2684

Table 4: Seed selection ablation on VBench and FLF2V-obj metrics using Wan2.2-5B.

\dagger

denotes the average performance of non-selected seeds.

Ranking	R $@$ top-5	R $@$ bottom-5
ACE-Seed	62.50	54.16
VBench	63.88*	61.11*
FLF2V-obj	70.83 *	78.12 *

Table 5: Recall scores using human ranking as relevant items from S2M-DAVIS. FLF2V-obj aligns best to human preference, highlighting its value as a set of new metrics. * denotes average recall over respective set of metrics.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Comparison of our Search2Motion using LaMa [ 26 ] inpainting versus Generative Omnimatte [ 13 ] . With LaMa, last-frame’s inpainting artifacts persist throughout the generated sequence, whereas our earlier method utilizing Generative Omnimatte produces cleaner backgrounds and yields overall higher video quality.

Figure 11: Qualitative comparison of image object removal and background inpainting. The top row displays the source, mask, and target of sample $\text{real}\_001$ from the ObjectMover set A. The bottom row, from left to right, displays images generated by ObjectClear, RORem, and our inpainting methods. ObjectClear and our methods can remove shadows, whereas RORem does not.

$\text{{Search2Motion}{}}_{GenOmni}$	✓	21.37	0.69	30.90	0.13	1160.80
	Training
Method	Free	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	fMSE $\downarrow$
$\text{{Search2Motion}{}}_{ROSE}$	✓	21.67	0.68	27.93	0.11	1148.89
RORem [ 14 ]	-	26.70	0.83	29.91	0.08	1181.25
ObjectClear [ 37 ]	-	31.58	0.93	15.01	0.03	600.51

Appendix

Table 6: Image object removal and background inpainting with

512\times 512

resolution. We evaluate these methods on the ObjectMover [ 34 ] dataset. More specifically, the ObjMove-A set, which contains 200 (source, mask, target) samples.

Figure 12: Synthetic data generation for S2M-DAVIS through the semantic-guided object placement pipeline.

Panel of Figure 13 — Figure 13: Qualitative results of Search2Motion video generation comparing against DragAnything and TTM on S2M-OMB (top two) and S2M-DAVIS (bottom).

Panel of Figure 14 — Figure 14: Object end state evaluation (green: ground truth; red: prediction).

Dataset	Seed Selection	Subject $\uparrow$	Background $\uparrow$	Temporal $\uparrow$	Motion $\uparrow$	DINOv2 $\uparrow$	DINOv2 $\uparrow$	LPIPS $\downarrow$	LPIPS $\downarrow$
		Consistency	Consistency	Flickering	Smoothness		first frame		first frame
S2M-OMB	none $\dagger$	$94.07$	$95.51$	$97.53$	$98.53$	$96.26$	$82.28$	$.0592$	$.2229$
	ACE-Seed	${\bf 94.77}$	${\bf 95.98}$	${\bf 97.99}$	${\bf 98.75}$	96.92	84.46	.0498	.1965
S2M-DAVIS	none $\dagger$	$94.39$	$95.14$	$97.91$	$98.69$	$93.93$	$76.39$	$.0904$	$.2849$
	ACE-Seed	${\bf 95.69}$	${\bf 95.89}$	${\bf 98.53}$	${\bf 99.00}$	95.19	81.16	.0809	.2644

Appendix

Table 7: Seed selection ablation on VBench and FLF2V-obj metrics using VACE-1.3B.

\dagger

denotes the average performance of non-selected seeds.

Figure 15: Additional qualitative Best-Worst comparison ranked by ACE-Seed algorithm from the S2M-DAVIS dataset.

Figure 16: Additional qualitative Best-Worst comparison ranked by ACE-Seed algorithm from the S2M-OMB dataset.

Panel of Figure 17 — Figure 17: VBench ablation for ACE-Seed with different numbers of noise seeds. Higher is better.

Panel of Figure 18 — Figure 18: FLF2V-obj ablation for ACE-Seed with different numbers of noise seeds.

Step	Subject $\uparrow$	Background $\uparrow$	Temporal $\uparrow$	Motion $\uparrow$	DINOv2 $\uparrow$	DINOv2 $\uparrow$	LPIPS $\downarrow$	LPIPS $\downarrow$
	Consistency	Consistency	Flickering	Smoothness		First Frame		First Frame
1	95.44	95.52	98.39	98.94	94.76	78.02	0.0834	0.2831
4	95.35	95.47	98.43	98.94	95.06	79.11	0.0787	0.2705
7	95.53	95.82	98.49	98.98	94.93	80.48	0.0824	0.2723
10	95.69	95.89	98.53	99.00	95.19	81.16	0.0809	0.2644

Appendix

Table 8: VBench and FLF2V-obj ablation results on S2M-DAVIS for attention maps of different diffusion steps used in ACE-Seed . All experiments share the same set of noise seeds. Best , Second Best .

Figure 19: Early-stage self-attention visualization under different token-masking strategies from S2M-DAVIS dataset. For each sample, from top to bottom: attention computed with the foreground (fg) object mask , inverse foreground mask (bg) , placement mask (pl) . It shows that bg can provide the most rich information (comparing to fg) with a relatively more consistent area coverage among different samples (comparing to pl).

Figure 20: Attention overlays on the final generated video across sampled denoising steps (ranges from 1 to 50). The attention maps begin to resemble the final object layout very early in the diffusion process, with the strongest structural alignment emerging around step 10.

Panel of Figure 21 — Figure 21: Self-attention overlays on the final video across attention blocks 0 to 29, shown for denoising step 1 (top) and step 10 (bottom). For both steps, blocks 22-26 produce the cleanest and most spatially coherent attention maps relative to other layers, given the first-frame foreground-mask tokens. Compared with step 1, the attention maps at step 10 exhibit noticeably clearer object contours that align better with the final video layout.

Controllable Video Generation

Controllable video generation steers how videos look, move, and change over time using user-provided instructions and structured controls.

See all 309 papers

MotiMotion: Motion-Controlled Video Generation with Visual Reasoning
May 21, 2026Lee Hsin-Ying, Hanwen Jiang, Yiqun Mei +3Physical Consistency in Video Generation Image-to-Video Generation
University of California, Merced · Adobe Research
FlowMotion: Training-Free Flow Guidance for Video Motion Transfer
Mar 6, 2026Zhen Wang, Youcan Xu, Jun Xiao +1Controllable Video Generation Temporal Consistency in Video Generation
The Hong Kong University of Science and Technology · State key lab of CAD&CG, Zhejiang University
TRACE: Object Motion Editing in Videos with First-Frame Trajectory Guidance
Mar 26, 2026Quynh Phung, Long Mai, Cusuh Ham +3Video Editing Controllable Video Generation
University of Maryland, College Park · Adobe Research

May 21, 2026J/K move · Enter open · S save

MotiMotion: Motion-Controlled Video Generation with Visual Reasoning

Lee Hsin-Ying, Hanwen Jiang, Yiqun Mei, Jing Shi+2

University of California, Merced · Adobe Research

Physical Consistency in Video Generation Image-to-Video Generation Causal Reasoning in Videos Controllable Video Generation

Open paper AskPDF arXiv

Current motion-controlled image-to-video generation models rigidly follow user-provided trajectories that are often sparse, imprecise, and causally incomplete. Such reliance often yields unnatural or implausible outcomes, especially by missing secondary causal consequences. To address this, we introduce MotiMotion, a novel framework that reformulates motion control as a reasoning-then-generation problem. To encourage causally grounded and commonsense-consistent interactions, we leverage a training-free vision-language reasoner to refine image-space coordinates of primary trajectories and to hallucinate plausible secondary motions. To further improve motion naturalness, we propose a confidence-aware control scheme that modulates guidance strength, enabling the model to closely follow high-confidence plans while correcting artifacts under low-confidence inputs with its internal generative priors. To support systematic evaluation, we curate a new image-to-video benchmark, MotiBench, consisting of interaction-centric scenes where new events are triggered by motion. Both VLM-based evaluation and a human study on MotiBench demonstrate that MotiMotion produces videos with more plausible object behaviors and interaction, and is preferred over existing approaches.

Abstract

Controllable Video Generation

Explore similar work

MotiMotion: Motion-Controlled Video Generation with Visual Reasoning

FlowMotion: Training-Free Flow Guidance for Video Motion Transfer

TRACE: Object Motion Editing in Videos with First-Frame Trajectory Guidance