Drag as Evidence: Motion-Grounded Latent Recomposition for Drag-Based Editing

Xinyu Pu; Hongsong Wang; Jie Gui; Pan Zhou

cs.CVSep 29, 2026

Drag as Evidence: Motion-Grounded Latent Recomposition for Drag-Based Editing

Authors: Xinyu Pu, Hongsong Wang, Jie Gui, Pan Zhou

Organizations: Southeast University · Southeast University Purple Mountain Laboratories · Singapore Management University

Abstract

Modern image editors excel at semantic manipulation and visual synthesis, yet remain limited in precise spatial control, motivating the development of drag-based editing. However, existing drag-based methods often struggle to balance drag accuracy with natural, plausible, and intent-aligned generation. We propose MoRe-Drag, a motion-grounded drag-based editing method. Our key insight is to treat pixel-space warping as coarse motion evidence, and to inject this evidence into the generative sampling trajectory. Specifically, MoRe-Drag performs region-aware latent recomposition over refinement, inpainting, and anchor regions, coupled with stage-adaptive conditioning that progressively shifts from motion-grounded structure formation to semantic refinement. We further support an instruction-free interface by adapting the MLLM-based text encoder for drag-aware instruction inference. Experiments on DragBench-SR and DragBench-DR show that MoRe-Drag substantially improves drag precision over strong base editors and achieves superior drag accuracy among SOTA drag-based methods, while delivering strong semantic consistency and visually realistic results. Code and dataset will be publicly released.

Figures & tables

Figure 1: Editing comparison. Pixel-space warping provides explicit but imperfect motion evidence. MoRe-Drag injects this evidence into the generative trajectory, enabling the editor to follow the specified motion while restoring natural structure and appearance.

Figure 2: Conceptual comparison with existing drag-based and text-based editing paradigms. Existing drag methods use warped image as an intermediate result, leading to accurate but unnatural edits, while text-based editors produce natural results without precise drag control. Our method uses warped image as motion evidence to achieve both accurate dragging and natural appearance.

Figure 3: Overview of MoRe-Drag . (a) Region-aware recomposition: the evidence and original image are decomposed into role-specific regions, producing role masks that guide latent recomposition, so motion cues are injected into the corresponding regions. (b) Generation-stage adaptive conditioning: at early timesteps $(t\geq t_{c})$ , the model uses the warped image with a fixed prompt to enforce motion consistency; at later timesteps $(t<t_{c})$ , it switches to the original image with the user/auto prompt for semantic and appearance refinement.

Figure 4: Toy editing from different initial timesteps. Top: starting latent timestep; bottom: LPIPS to the original image.

Figure 5: Instruction interaction and data construction. Top: drag annotation construction from ByteMorph . Bottom: LoRA-based MLLM fine-tuning with AR loss, enabling automatic instruction inference from drag annotations.

Test-time Optimization-based Methods
Model	Params	DragBench-SR			DragBench-DR
Model	Params	MD $\downarrow$	CP $\uparrow$	PF $\uparrow$	MD $\downarrow$	CP $\uparrow$	PF $\uparrow$
DragDiffusion	2.1B	$41.08_{\pm 0.43}$	$92.75_{\pm 0.38}$	$77.15_{\pm 1.49}$	$31.61_{\pm 0.23}$	$96.46_{\pm 0.43}$	$78.45_{\pm 0.95}$
DragNoise	2.1B	$47.64_{\pm 1.08}$	$79.00_{\pm 0.25}$	$61.56_{\pm 0.31}$	$30.05_{\pm 0.01}$	$91.46_{\pm 1.34}$	$81.02_{\pm 0.18}$
GoodDrag	2.1B	$28.94_{\pm 0.40}$	$94.50_{\pm 0.13}$	$76.50_{\pm 0.39}$	$23.21_{\pm 0.12}$	$95.36_{\pm 0.24}$	$83.94_{\pm 0.04}$
Optimization-free Methods

Table 1: Results on DragBench-SR/DR . CP and PF are scaled by 100. Metrics of Qwen-Image-Edit and LongCat-Image-Edit are not highlighted, as they are not drag-based methods. “+ FLUX.1-Fill-dev” denotes an Inpaint4Drag variant using a stronger inpainting model. (*) indicates results from the original papers, as the models were not publicly available at the time of evaluation. In the Params column, (-) indicates that MoRe-Drag keeps the generative backbone frozen.

Figure 6: Qualitative comparisons with state-of-the-art drag-based editing models. Blue and red points denote handle and target points, respectively. LongCat-Image-Edit is a text-based editing method; Inpaint4Drag and MoRe-Drag take the warped image as input, while the other methods are driven by point inputs. We provide zoomed-in views of the edited regions for the two warped-image-based methods to highlight local structural fidelity. More results are shown in Fig. 11

Figure 9: Instruction interaction examples. Compared with vanilla instruction inference, LoRA adaptation produces drag-aware instructions that better match the visual annotations, reducing unintended artifacts and hallucinated content.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Main models	Key settings	Size
Raw dataset	-	-	780K
Motion-edit filtering	Qwen3.5-9B	local human/object motion; no global camera change	220K
Drag annotation mining	Grounding DINO, SAM, RAFT	flow threshold $1.5$ ; cycle error $\leq 1$ ; max $16$ points/sample	220K
Quality scoring and filtering	Qwen3.5-9B + Human	alignment scores in $\{0,1,2,3\}$ ; keep $S_{\mathrm{avg}}\geq 0.6$	68K

Appendix

Table 4: Summary of the drag-instruction dataset construction pipeline.

Figure 10: Illustration of bidirectional warping and mask decomposition. Given handle–target drag annotations, bidirectional warping produces a warped observation that provides pixel-space motion evidence. We decompose the result into preservation, refinement, and inpainting regions for region-aware latent recomposition: blue denotes the anchor preservation $\boldsymbol{M}_{p}$ , yellow denotes the refinement mask $\boldsymbol{M}_{r}$ , and red denotes the inpainting mask $\boldsymbol{M}_{i}$ .

Figure 11: Additional qualitative comparisons on diverse drag-based editing examples. MoRe-Drag faithfully follows the specified drag motion while preserving object structure, texture details, and background consistency.

Setting (RAR steps, GAC switch)	DragBench-SR
Setting (RAR steps, GAC switch)	MD $\downarrow$	CP $\uparrow$	PF $\uparrow$
(6, 2)	$37.00_{\pm 0.48}$	$92.75_{\pm 0.00}$	$69.72_{\pm 0.28}$
(11, 9)	$21.09_{\pm 0.20}$	$95.75_{\pm 0.25}$	$80.63_{\pm 0.38}$
(16, 12)	${18.54_{\pm 0.22}}$	${96.36_{\pm 0.38}}$	${86.36_{\pm 4.06}}$
(21, 14)	$18.43_{\pm 0.31}$	$95.87_{\pm 0.88}$	$83.50_{\pm 2.36}$
(30, 20)	$17.98_{\pm 0.37}$	$94.25_{\pm 1.00}$	$80.75_{\pm 1.61}$

Appendix

Table 5: Hyperparameter analysis on DragBench-SR .

	MD $\downarrow$	CP $\uparrow$	PF $\uparrow$
Vanilla	$19.94_{\pm 0.27}$	$95.12_{\pm 0.13}$	$78.63_{\pm 1.38}$
w/ LoRA	${18.54_{\pm 0.22}}$	${96.36_{\pm 0.38}}$	${86.36_{\pm 4.06}}$

Appendix

Table 6: Quantitative analysis of instruction interaction on DragBench-SR .

Figure 12: Additional comparisons of drag-aware instruction inference. Each column shows a drag-annotated input, the instruction inferred by the vanilla MLLM, and the instruction inferred by the LoRA-adapted MLLM. LoRA adaptation improves grounding to the visual drag annotations and produces instructions that better reflect the intended object motion or deformation.

Figure 13: Failure cases. When the warped observation contains severe errors or ambiguous disocclusions, the injected motion evidence may not faithfully represent the intended edit. In these cases, MoRe-Drag may favor plausible restoration over the desired motion, such as incomplete head manipulation or unsuccessful lid articulation.

Figure 14: Failure cases of drag-aware instruction inference. The LoRA-adapted MLLM may still infer imperfect instructions when drag annotations are too close to each other, especially for fine-grained shape changes.

Step $k$	4	8	16	20	24	28
$t_{c}$	0.87	0.73	0.47	0.33	0.20	0.07
MD $\downarrow$	$21.01_{\pm 0.1}$	$19.27_{\pm 0.25}$	$18.59_{\pm 0.32}$	$17.98_{\pm 0.09}$	$17.52_{\pm 0.24}$	$17.36_{\pm 0.16}$
CP $\uparrow$	$96.09_{\pm 0.63}$	$97.10_{\pm 0.13}$	$97.47_{\pm 0.25}$	$97.10_{\pm 0.13}$	$95.96_{\pm 0.50}$	$96.46_{\pm 0.26}$
PF $\uparrow$	$83.33_{\pm 2.68}$	$85.10_{\pm 1.99}$	$80.08_{\pm 1.29}$	$81.06_{\pm 0.51}$	$79.31_{\pm 0.74}$	$80.94_{\pm 0.13}$

Appendix

Table 7: Effect of switching timestep

t_{c}

on DragBench-SR .

Figure 15: Comparison of base editor reconstruction ability. Under null conditions, LongCat-Image-Edit reconstructs the source images more faithfully than Qwen-Image-Edit, with better preservation of identity, structure, and appearance. This stronger reconstruction stability benefits motion-evidence injection and non-edited region preservation in MoRe-Drag .

Method	Sampling steps	Avg. time / image (s)	Peak GPU memory (GB)
Baseline editor	30	30.74	16.05
MoRe-Drag	30	41.31	17.18

Appendix

Table 8: Runtime and memory comparison. We report the average generation time per image and peak GPU memory over 10 samples.

Explore similar work

In-context Region-based Drag: Drag Any Region to Any Shape
Jun 24, 2026Jiacheng Sui, Tianyu Hao, Bingjie Gao +2Diffusion-Based Image Editing Region
Shanghai Jiao Tong University
Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network
Sep 17, 2026Yulong Chen, Ziqian Zhang, Haoyu Zhang +4Diffusion-Based Image Editing Image Editing
City University of Hong Kong (Dongguan), Guangdong, China · City University of Hong Kong, Hong Kong, China · Mohamed bin Zayed University of Artificial Intelligence, Masdar, Abu Dhabi
DrawAI: Agentic Benchmark and Workflow for Making Raster Images Editable
Aug 1, 2026Pu Cao, Qingye Kong, Xuedan Yin +5Agentic Workflow Design Agentic Benchmarks
Beijing University of Posts and Telecommunications · Tsinghua University · Beijing Institute of Technology

Jun 24, 2026, cs.CVJ/K move · Enter open · S save

In-context Region-based Drag: Drag Any Region to Any Shape

Jiacheng Sui, Tianyu Hao, Bingjie Gao, Li Niu+1

Shanghai Jiao Tong University

Diffusion-Based Image Editing Region In-Context Learning In-Context

Open paper AskPDF arXiv

Diffusion models have shown promise in drag-style editing. Previous works mainly focus on point-based drag, which is inherently ambiguous. This paper focuses on region-based drag and introduces a novel In-Context Region-based Drag (ICRDrag) method. Under the in-context learning framework, ICRDrag consumes a source image, a source region mask, and a target region mask, producing the target dragged image. Built upon the basic in-context learning model, we introduce two novel attention regularization: 1) image-mask attention consistency to ensure that a target region attends to similar source regions for image and mask modalities; 2) source-target attention correspondence to ensure the mutual correspondence between source and target regions. To facilitate region-based drag, we also construct Paired Region Dataset (PRD), a large-scale dataset with paired masks and images. Extensive experiments show that ICRDrag significantly outperforms existing methods in both quantitative metrics and user studies, achieving superior editing accuracy and visual fidelity. The dataset, code, and model are available at https://github.com/bcmi/ICRDrag-Region-Drag-Editing.

Abstract

Explore similar work

In-context Region-based Drag: Drag Any Region to Any Shape

Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network

DrawAI: Agentic Benchmark and Workflow for Making Raster Images Editable