Drag as Evidence: Motion-Grounded Latent Recomposition for Drag-Based Editing

Xinyu Pu; Hongsong Wang; Jie Gui; Pan Zhou

cs.CVSep 29, 2026

Drag as Evidence: Motion-Grounded Latent Recomposition for Drag-Based Editing

Authors: Xinyu Pu, Hongsong Wang, Jie Gui, Pan Zhou

Organizations: Southeast University · Southeast University Purple Mountain Laboratories · Singapore Management University

Abstract

Modern image editors excel at semantic manipulation and visual synthesis, yet remain limited in precise spatial control, motivating the development of drag-based editing. However, existing drag-based methods often struggle to balance drag accuracy with natural, plausible, and intent-aligned generation. We propose MoRe-Drag, a motion-grounded drag-based editing method. Our key insight is to treat pixel-space warping as coarse motion evidence, and to inject this evidence into the generative sampling trajectory. Specifically, MoRe-Drag performs region-aware latent recomposition over refinement, inpainting, and anchor regions, coupled with stage-adaptive conditioning that progressively shifts from motion-grounded structure formation to semantic refinement. We further support an instruction-free interface by adapting the MLLM-based text encoder for drag-aware instruction inference. Experiments on DragBench-SR and DragBench-DR show that MoRe-Drag substantially improves drag precision over strong base editors and achieves superior drag accuracy among SOTA drag-based methods, while delivering strong semantic consistency and visually realistic results. Code and dataset will be publicly released.

Figures & tables

Figure 1: Editing comparison. Pixel-space warping provides explicit but imperfect motion evidence. MoRe-Drag injects this evidence into the generative trajectory, enabling the editor to follow the specified motion while restoring natural structure and appearance.

Figure 2: Conceptual comparison with existing drag-based and text-based editing paradigms. Existing drag methods use warped image as an intermediate result, leading to accurate but unnatural edits, while text-based editors produce natural results without precise drag control. Our method uses warped image as motion evidence to achieve both accurate dragging and natural appearance.

Figure 3: Overview of MoRe-Drag . (a) Region-aware recomposition: the evidence and original image are decomposed into role-specific regions, producing role masks that guide latent recomposition, so motion cues are injected into the corresponding regions. (b) Generation-stage adaptive conditioning: at early timesteps $(t\geq t_{c})$ , the model uses the warped image with a fixed prompt to enforce motion consistency; at later timesteps $(t<t_{c})$ , it switches to the original image with the user/auto prompt for semantic and appearance refinement.

Figure 4: Toy editing from different initial timesteps. Top: starting latent timestep; bottom: LPIPS to the original image.

Figure 5: Instruction interaction and data construction. Top: drag annotation construction from ByteMorph . Bottom: LoRA-based MLLM fine-tuning with AR loss, enabling automatic instruction inference from drag annotations.

Test-time Optimization-based Methods
Model	Params	DragBench-SR			DragBench-DR
Model	Params	MD $\downarrow$	CP $\uparrow$	PF $\uparrow$	MD $\downarrow$	CP $\uparrow$	PF $\uparrow$
DragDiffusion	2.1B	$41.08_{\pm 0.43}$	$92.75_{\pm 0.38}$	$77.15_{\pm 1.49}$	$31.61_{\pm 0.23}$	$96.46_{\pm 0.43}$	$78.45_{\pm 0.95}$
DragNoise	2.1B	$47.64_{\pm 1.08}$	$79.00_{\pm 0.25}$	$61.56_{\pm 0.31}$	$30.05_{\pm 0.01}$	$91.46_{\pm 1.34}$	$81.02_{\pm 0.18}$
GoodDrag	2.1B	$28.94_{\pm 0.40}$	$94.50_{\pm 0.13}$	$76.50_{\pm 0.39}$	$23.21_{\pm 0.12}$	$95.36_{\pm 0.24}$	$83.94_{\pm 0.04}$
Optimization-free Methods

Table 1: Results on DragBench-SR/DR . CP and PF are scaled by 100. Metrics of Qwen-Image-Edit and LongCat-Image-Edit are not highlighted, as they are not drag-based methods. “+ FLUX.1-Fill-dev” denotes an Inpaint4Drag variant using a stronger inpainting model. (*) indicates results from the original papers, as the models were not publicly available at the time of evaluation. In the Params column, (-) indicates that MoRe-Drag keeps the generative backbone frozen.

Figure 6: Qualitative comparisons with state-of-the-art drag-based editing models. Blue and red points denote handle and target points, respectively. LongCat-Image-Edit is a text-based editing method; Inpaint4Drag and MoRe-Drag take the warped image as input, while the other methods are driven by point inputs. We provide zoomed-in views of the edited regions for the two warped-image-based methods to highlight local structural fidelity. More results are shown in Fig. 11

Figure 9: Instruction interaction examples. Compared with vanilla instruction inference, LoRA adaptation produces drag-aware instructions that better match the visual annotations, reducing unintended artifacts and hallucinated content.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Main models	Key settings	Size
Raw dataset	-	-	780K
Motion-edit filtering	Qwen3.5-9B	local human/object motion; no global camera change	220K
Drag annotation mining	Grounding DINO, SAM, RAFT	flow threshold $1.5$ ; cycle error $\leq 1$ ; max $16$ points/sample	220K
Quality scoring and filtering	Qwen3.5-9B + Human	alignment scores in $\{0,1,2,3\}$ ; keep $S_{\mathrm{avg}}\geq 0.6$	68K

Appendix

Table 4: Summary of the drag-instruction dataset construction pipeline.

Figure 10: Illustration of bidirectional warping and mask decomposition. Given handle–target drag annotations, bidirectional warping produces a warped observation that provides pixel-space motion evidence. We decompose the result into preservation, refinement, and inpainting regions for region-aware latent recomposition: blue denotes the anchor preservation $\boldsymbol{M}_{p}$ , yellow denotes the refinement mask $\boldsymbol{M}_{r}$ , and red denotes the inpainting mask $\boldsymbol{M}_{i}$ .

Figure 11: Additional qualitative comparisons on diverse drag-based editing examples. MoRe-Drag faithfully follows the specified drag motion while preserving object structure, texture details, and background consistency.

Setting (RAR steps, GAC switch)	DragBench-SR
Setting (RAR steps, GAC switch)	MD $\downarrow$	CP $\uparrow$	PF $\uparrow$
(6, 2)	$37.00_{\pm 0.48}$	$92.75_{\pm 0.00}$	$69.72_{\pm 0.28}$
(11, 9)	$21.09_{\pm 0.20}$	$95.75_{\pm 0.25}$	$80.63_{\pm 0.38}$
(16, 12)	${18.54_{\pm 0.22}}$	${96.36_{\pm 0.38}}$	${86.36_{\pm 4.06}}$
(21, 14)	$18.43_{\pm 0.31}$	$95.87_{\pm 0.88}$	$83.50_{\pm 2.36}$
(30, 20)	$17.98_{\pm 0.37}$	$94.25_{\pm 1.00}$	$80.75_{\pm 1.61}$

Appendix

Table 5: Hyperparameter analysis on DragBench-SR .

	MD $\downarrow$	CP $\uparrow$	PF $\uparrow$
Vanilla	$19.94_{\pm 0.27}$	$95.12_{\pm 0.13}$	$78.63_{\pm 1.38}$
w/ LoRA	${18.54_{\pm 0.22}}$	${96.36_{\pm 0.38}}$	${86.36_{\pm 4.06}}$

Appendix

Table 6: Quantitative analysis of instruction interaction on DragBench-SR .

Figure 12: Additional comparisons of drag-aware instruction inference. Each column shows a drag-annotated input, the instruction inferred by the vanilla MLLM, and the instruction inferred by the LoRA-adapted MLLM. LoRA adaptation improves grounding to the visual drag annotations and produces instructions that better reflect the intended object motion or deformation.

Figure 13: Failure cases. When the warped observation contains severe errors or ambiguous disocclusions, the injected motion evidence may not faithfully represent the intended edit. In these cases, MoRe-Drag may favor plausible restoration over the desired motion, such as incomplete head manipulation or unsuccessful lid articulation.

Figure 14: Failure cases of drag-aware instruction inference. The LoRA-adapted MLLM may still infer imperfect instructions when drag annotations are too close to each other, especially for fine-grained shape changes.

Step $k$	4	8	16	20	24	28
$t_{c}$	0.87	0.73	0.47	0.33	0.20	0.07
MD $\downarrow$	$21.01_{\pm 0.1}$	$19.27_{\pm 0.25}$	$18.59_{\pm 0.32}$	$17.98_{\pm 0.09}$	$17.52_{\pm 0.24}$	$17.36_{\pm 0.16}$
CP $\uparrow$	$96.09_{\pm 0.63}$	$97.10_{\pm 0.13}$	$97.47_{\pm 0.25}$	$97.10_{\pm 0.13}$	$95.96_{\pm 0.50}$	$96.46_{\pm 0.26}$
PF $\uparrow$	$83.33_{\pm 2.68}$	$85.10_{\pm 1.99}$	$80.08_{\pm 1.29}$	$81.06_{\pm 0.51}$	$79.31_{\pm 0.74}$	$80.94_{\pm 0.13}$

Appendix

Table 7: Effect of switching timestep

t_{c}

on DragBench-SR .

Figure 15: Comparison of base editor reconstruction ability. Under null conditions, LongCat-Image-Edit reconstructs the source images more faithfully than Qwen-Image-Edit, with better preservation of identity, structure, and appearance. This stronger reconstruction stability benefits motion-evidence injection and non-edited region preservation in MoRe-Drag .

Method	Sampling steps	Avg. time / image (s)	Peak GPU memory (GB)
Baseline editor	30	30.74	16.05
MoRe-Drag	30	41.31	17.18

Appendix

Table 8: Runtime and memory comparison. We report the average generation time per image and peak GPU memory over 10 samples.

Explore similar work

Jun 24, 2026cs.CV

In-context Region-based Drag: Drag Any Region to Any Shape

Diffusion models have shown promise in drag-style editing. Previous works mainly focus on point-based drag, which is inherently ambiguous. This paper focuses on region-based drag and introduces a novel In-Context Region-based Drag (ICRDrag) method. Under the in-context learning framework, ICRDrag consumes a source image, a source region mask, and a target region mask, producing the target dragged image. Built upon the basic in-context learning model, we introduce two novel attention regularization: 1) image-mask attention consistency to ensure that a target region attends to similar source regions for image and mask modalities; 2) source-target attention correspondence to ensure the mutual correspondence between source and target regions. To facilitate region-based drag, we also construct Paired Region Dataset (PRD), a large-scale dataset with paired masks and images. Extensive experiments show that ICRDrag significantly outperforms existing methods in both quantitative metrics and user studies, achieving superior editing accuracy and visual fidelity. The dataset, code, and model are available at https://github.com/bcmi/ICRDrag-Region-Drag-Editing.

Jiacheng Sui, Tianyu Hao, Bingjie Gao +2

Shanghai Jiao Tong University

Diffusion-Based Image Editing Region In-Context Learning In-Context

Ask PDF

Sep 17, 2026cs.CV

Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network

Text-guided image editing must introduce the requested changes while preserving unrelated source content. In training-free editing, diffusion editors often use spatial controls whose inaccuracies can leave edits incomplete or alter unrelated regions. Causal autoregressive editors face a further constraint: their fixed decoding order limits revision of earlier decisions. As the first to explore training-free image editing with Generative Refinement Networks (GRN), we observe that its refinement process is inherently suitable for editing and offers a promising way to address these limitations. Motivated by this observation, we introduce RefineEdit, a training-free prompt-to-prompt image editing framework built on the GRN. Our key idea is to couple edit localization with content generation through the global refinement of binary image codes, allowing editing evidence to be revised as the image evolves. More specifically, RefineEdit combines bit routing with two stabilization mechanisms: adaptive spatial freezing and finite bit locking. Bit routing starts from an intermediate source state and uses signed probability differences between the two branches to identify editable positions and bits. It directs selected bits toward editing refinement while anchoring the rest to the evolving source trajectory. Adaptive spatial freezing limits unnecessary expansion of the editing region, while finite bit locking maintains recent bit activations to support continued editing. The overall framework requires no additional training, external masks, or attention control. Across nine editing categories of PIE-Bench, RefineEdit achieves the best background-preservation scores in PSNR, LPIPS, MSE, and SSIM, together with the highest whole-image and edited-region CLIP scores among the evaluated methods. Code is available at https://github.com/mura1n/RefineEdit.

Yulong Chen, Ziqian Zhang, Haoyu Zhang +4

City University of Hong Kong (Dongguan), Guangdong, China · City University of Hong Kong, Hong Kong, China · Mohamed bin Zayed University of Artificial Intelligence, Masdar, Abu Dhabi

Diffusion-Based Image Editing Image Editing Multi-Reference Image Generation Refinement

Ask PDF

Aug 1, 2026cs.CV

DrawAI: Agentic Benchmark and Workflow for Making Raster Images Editable

Recent image-generation models and multimodal agents can produce high-quality visuals for increasingly complex visual communication tasks. Yet their raster outputs remain difficult to use directly because meaningful content and relationships are flattened into pixels, preventing users from inspecting, modifying, rearranging, or reusing individual components. We formulate image-to-editable reconstruction, which recovers a structured, directly manipulable artifact from a raster image while preserving its visual and semantic content. The central challenge is to jointly satisfy Fidelity and Editability, which often trade off in practice. To study this task, we introduce DrawAI, comprising an agentic benchmark, DrawAI-Bench, and a reconstruction workflow, DrawAI-Flow. DrawAI-Bench spans scientific figures, presentation slides, posters, and diagrams, combining real and AI-generated images to reflect practical visual-creation scenarios. It evaluates Fidelity and Editability through a hybrid protocol of 39 criteria: deterministic rule-based metrics measure properties with direct correspondences, while asset-specific vision-language rubrics capture semantic and perceptual qualities for which exact matching is misleading. Besides, we propose DrawAI-Flow, a two-stage agentic workflow in which a Parser Agent turns extracted elements evidence into an explicit reconstruction plan, and a Reconstruction Agent realizes the plan as executable graphics code through an iterative code-render-validate-revise loop. On DrawAI-Bench, we systematically evaluate thirteen models across five agent harnesses to study the effects of model capability, harness choice, and workflow design. The results show that reconstruction quality and costs vary substantially across model-harness configurations, while DrawAI-Flow consistently improves editable structure.

Pu Cao, Qingye Kong, Xuedan Yin +5

Beijing University of Posts and Telecommunications · Tsinghua University · Beijing Institute of Technology

Agentic Workflow Design Agentic Benchmarks

Ask PDF