Object-Aware Background-Controlled Editing via Weighted Velocity Guidance

Wuji Wang; Yue Wu; Chenhao Yi; Shuhui Wang

cs.CVSep 5, 2026

Object-Aware Background-Controlled Editing via Weighted Velocity Guidance

Authors: Wuji Wang, Yue Wu, Chenhao Yi, Shuhui Wang

Organizations: Language Technologies Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Department of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China

Abstract

Training-free image editing steers diffusion or flow-matching generative models at inference time by modifying prompt-conditioned denoising velocities. Existing velocity-based editors often apply prompt-induced residuals globally over the latent space and rely on the model to localize semantic changes implicitly. For object-centric edits, these residuals are rarely zero outside the target object, so small non-target components can accumulate during multi-step integration, causing background drift and unstable object boundaries. We propose Object-Aware Velocity Control (OAVC), a training-free framework that introduces object-level control into the velocity-integration process. OAVC decouples where semantic residuals are allowed to act from how they are injected into the dynamics. It constructs a background-anchored reference interface under the source prompt and then performs object-localized safe semantic injection under the target prompt. A constrained injection operator suppresses drift-inducing velocity components, while time-adaptive spatial weighting stabilizes the transition near object boundaries. OAVC requires no training or modification of pretrained model parameters. Experiments on object-centric image and video benchmarks with image and video rectified-flow backbones show improved background preservation, structural fidelity, boundary stability, and temporal consistency while retaining effective localized editability.

Figures & tables

Figure 1. Object-centric image and video editing. Our approach enables localized edits while keeping non-target regions stable during multi-step integration. Prior training-free velocity-steering methods often accumulate small background residuals, leading to background drift, boundary artifacts, and temporal inconsistency. By enforcing object-localized semantic updates with explicit non-target preservation, we produce cleaner object removal/inpainting, faithful attribute or color changes, and temporally consistent video edits.

Figure 2. Illustration of Stage-2: Projected and Constrained Semantic Injection. We explicitly constrain the edit to the object region and a safe injection direction.

Figure 3. Overview of Object-Aware Velocity Control. Global steering applies prompt-induced residuals over the full latent trajectory, causing non-target accumulation and background drift. OAVC first constructs a background-anchored reference interface, then applies explicit where/how control: object supports specify where semantic residuals are integrated, while a safe injection operator specifies how they are transformed before solver integration.

Panel of Figure 4 — Figure 4. Qualitative comparison on PIE-Bench. We compare OAVC against representative training-free editors across SD3.5 and FLUX. OAVC better preserves non-target background and object boundaries while achieving the intended edits (red boxes).

Method	Model	Structure	Background Preservation				CLIP Similarity
Method	Model	Distance $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	MSE $\downarrow$	SSIM $\uparrow$	Whole $\uparrow$	Edited $\uparrow$
PnP ( Tumanyan et al., 2023 )	SD1.5	27.35	22.31	112.76	82.95	79.25	25.41	22.52
DI+PnP ( Ju et al., 2023 )	SD1.5	23.35	22.46	105.51	79.94	79.88	25.49	22.64
InfEdit ( Xu et al., 2024 )	LCM	19.31	27.31	56.32	47.80	85.30	24.90	22.14
RF-Inv ( Rout et al., 2025 )	FLUX	42.29	20.20	179.54	139.20	69.91	24.57	22.20
RFEdit ( Wang et al., 2024 )	FLUX	21.79	24.83	113.15	52.46	83.38	25.57	22.26

Table 1. Quantitative comparison on PIE-Bench. OAVC improves structural fidelity and background preservation while retaining effective localized editability. The best and second-best results are indicated by bold and underlined text, respectively.

Setting	Structure	Background Preservation				CLIP Similarity
Setting	Distance $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	MSE $\downarrow$	SSIM $\uparrow$	CLIP ${}_{\text{whole}}\uparrow$	CLIP ${}_{\text{edit}}\uparrow$
DNAEdit (baseline; global)	14.19	26.66	74.57	32.76	88.63	25.63	22.71
Stage-1 only (Anchor-DNA)	14.68	26.28	77.75	35.44	88.42	25.76	22.87
Stage-2 (Mask+Ring+Proj; w/o Gate)	6.60	31.74	27.65	11.95	92.61	24.60	21.97
Stage-2 (+Mask only; $W^{\mathrm{edit}}$ )	5.27	32.27	25.78	10.48	92.99	24.55	21.87
DNAEdit + final latent hard blend	6.28	–	24.29	9.56	–	24.46	21.98

Table 2. PIE-Bench ablation (all metrics scaled to match Tab. 1 ). We report the SD3.5 setting. The best results are indicated by bold text.

Method	BG-PSNR $\uparrow$	BG-L1 $\downarrow$	TW-BG-PSNR $\uparrow$	TW-BG-L1 $\downarrow$
DNAEdit	19.85	0.0692	26.56	0.0281
UniEdit-Flow	22.87	0.0467	27.97	0.0227
OAVC	27.53	0.0284	29.65	0.0212

Table 3. DAVIS quantitative video results. BG metrics measure per-frame background fidelity, while TW-BG metrics measure temporal background stability using source-video optical flow.

Figure 5. DAVIS video editing example for target-object removal across consecutive frames.

Method	Backbone	Structure	Background Preservation				CLIP Similarity $\uparrow$
Method	Backbone	Distance $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	MSE $\downarrow$	SSIM $\uparrow$	CLIP ${}_{\text{whole}}\uparrow$	CLIP ${}_{\text{edit}}\uparrow$
FlowEdit	FLUX	27.82	21.96	112.19	94.99	83.08	25.25	22.58
FlowEdit	SD3	27.12	22.22	104.12	85.96	93.22	26.53	23.57
Ours (OAVC on FlowEdit)	SD3	35.72	27.65	81.05	21.57	86.98	24.46	22.20
Ours (OAVC on FlowEdit)	FLUX	9.32	30.77	30.09	16.80	93.57	24.18	23.66

Table 4. Transfer to FlowEdit backbones on PIE-Bench (all metrics scaled to match Tab. 1 ).

Method	Struct. $\downarrow$	BG LPIPS $\downarrow$	BG MSE $\downarrow$	CLIP ${}_{\rm whole}\uparrow$	CLIP ${}_{\rm edit}\uparrow$
DNAEdit (global)	14.19	74.57	32.76	25.63	22.71
Final latent hard blend	6.28	24.29	9.56	24.46	21.98
Mask-only velocity gating	5.27	25.78	10.48	24.55	21.87
Full OAVC	4.11	24.40	9.61	24.21	21.51

Table 5. Controlled comparison on PIE-Bench using SD3.5. All localized variants use the identical cached SAM3 support, thereby isolating the effect of trajectory-level control from that of object localization. Metrics follow the scaling convention of Table 1 .

Criterion	OAVC	DNAEdit	Final blend	None/Tie	No consensus
Edit realization	15	22	28	17	18
Background preservation	74	0	20	0	6
Absolute task success	Yes	Partially	Yes+Partially	No	–
OAVC	59.0%	20.5%	79.5%	20.5%	–
DNAEdit	65.5%	14.5%	80.0%	20.0%	–
Final blend	55.5%	21.5%	77.0%	23.0%	–

Table 6. Human evaluation. Top: majority-vote preference counts over 100 examples. Bottom: absolute task-success rates over 200 judgments per method.

Input-prior perturbation	IoU w.r.t. $M_{0}$	Area ratio	Empty	Struct. $\downarrow$	BG LPIPS $\downarrow$	BG MSE $\downarrow$	CLIP ${}_{\rm edit}\uparrow$
Erode 4	0.397	0.397 $\times$	13%	2.07	19.53	6.35	20.65
Erode 2	0.593	0.593 $\times$	6%	2.84	20.10	6.55	21.29
Original	1.000	1.000 $\times$	0%	4.21	23.15	8.08	21.47
Dilate 2	0.712	1.581 $\times$	0%	4.67	26.04	9.34	21.36
Dilate 4	0.594	2.236 $\times$	0%	4.93	28.88	10.38	21.42

Table 7. Robustness to input-prior corruption on 100 PIE-Bench examples. The cached SAM3 prior

M_{0}

is eroded or dilated before the unchanged OAVC support-construction step. All conditions retain

r_{\mathrm{fg}}=2

and

r_{\mathrm{bd}}=4

. IoU and area ratio characterize

M_{r}

relative to

M_{0}

; region-conditioned metrics use the same independent PIE-Bench annotation, while Structure Distance is global. Metrics are scaled as in Table 1 .

Figure 6. Robustness to generation-time input-prior corruption on a fixed 100-example PIE-Bench subset. Points denote means and error bars denote bootstrap 95% confidence intervals. The dashed line marks the unperturbed input prior (

r=0

). All conditions retain the same OAVC support-construction parameters, including

r_{\mathrm{fg}}=2

and

r_{\mathrm{bd}}=4

. Region-conditioned metrics use the same independent PIE-Bench annotation; Structure Distance is global. Erosion restricts edit coverage, whereas dilation increasingly exposes non-target regions to semantic residuals.

Figure 7. Qualitative examples on GEdit-Bench. Columns show Original, DNAEdit, and OAVC. Top: microphone $\rightarrow$ ice-cream cone. Bottom: remove the white railing. OAVC better localizes the requested edit and preserves surrounding content. These two cases provide a qualitative generalization check, not full-benchmark evaluation.

Method	FG-Gain		BG-Gain		FG-BG Gap
Method	Mean $\uparrow$	Std	Mean $\downarrow$	Std	Mean $\uparrow$	Std
DNAEdit (global)	0.0189	0.0299	0.0108	0.0245	0.0080	0.0336
OAVC	0.0130	0.0261	0.0008	0.0123	0.0122	0.0260

Table 8. Foreground/background target-semantic decomposition. FG-Gain and BG-Gain measure target-prompt gain in foreground and background regions, respectively, while FG-BG Gap measures localization strength.

Figure 8. Large-deformation object replacements from the same source image. Rows show car-to-train and car-to-plane edits, while columns compare the original image, DNAEdit, and OAVC. DNAEdit introduces more extensive changes to the road and surrounding scene and, in the car-to-plane case, incompletely removes the source car. OAVC concentrates the transformation around the source object while more closely preserving non-target scene content.

Figure 9. Representative two-object edit from a goat and a cat to a horse and a dog. The orange and blue overlays indicate the source goat and cat supports, respectively; their union is used as the editable support. Both methods modify the two target objects, while OAVC more closely retains the original scene layout and non-target background.

Backbone/Method	Time (s/img) $\downarrow$	Peak Mem (MiB) $\downarrow$
SD3.5 / DNAEdit	3.08	19,459
SD3.5 / Ours	3.12	23,139
FLUX / DNAEdit	9.01	35,839
FLUX / Ours	9.85	39,571

Table 9. Runtime and peak GPU memory (per image) for DNAEdit vs. OAVC on two rectified-flow backbones.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10. Interactive brush refinement for challenging support extraction cases. When the automatic SAM3 support is imperfect, users can manually refine the editable region with a brush, and OAVC applies the same velocity-control mechanism using the corrected support.

Support source	Struct.	Background Preservation				CLIP Similarity
Support source	Distance $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	MSE $\downarrow$	SSIM $\uparrow$	CLIP ${}_{\text{whole}}\uparrow$	CLIP ${}_{\text{edit}}\uparrow$
DNAEdit (global)	18.87	24.99	95.06	50.45	85.71	25.79	22.87
Attention-derived support	4.58	30.55	35.74	16.41	92.53	23.60	20.76
SAM3 support (default)	4.07	33.30	23.70	8.75	94.38	23.69	21.08

Appendix

Table 10. Support-source comparison on PIE-Bench. All metrics are scaled to match the main tables. Attention-derived support can replace SAM3 and still provides strong preservation gains over global DNAEdit, showing that OAVC is support-source agnostic.

Figure 11. Qualitative examples using attention-derived support instead of SAM3. Attention maps provide an internal support prior for OAVC, enabling object-localized velocity control without an external segmenter. Although attention-derived supports are coarser than SAM3 masks, they remain sufficient to reduce background drift and preserve non-target regions compared with global steering.

Figure 12. Stage-1 hyperparameter sweep on PIE-Bench for SD3.5 and FLUX. We vary foreground relaxation $\lambda_{fg}$ and background suppression $\rho_{bg}$ and report background preservation (LPIPS $\downarrow$ , MSE $\downarrow$ on non-target regions; scaled as shown) together with semantic alignment (CLIP ${}_{\text{whole}}$ /CLIP ${}_{\text{edit}}$ ). The dashed boxes highlight the stable region around our default setting ( $\lambda_{fg}{=}0.6$ , $\rho_{bg}{=}0.1$ ).

Setting	Structure	Background Preservation				CLIP Similarity $\uparrow$
Setting	Distance $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	MSE $\downarrow$	SSIM $\uparrow$	CLIP ${}_{\text{whole}}\uparrow$	CLIP ${}_{\text{edit}}\uparrow$
$r_{\mathrm{fg}}{=}3$	4.42	32.22	25.60	10.26	92.86	24.29	21.50
$r_{\mathrm{fg}}{=}1$	3.15	34.10	21.68	7.86	93.35	23.83	21.25
$r_{\mathrm{fg}}{=}0$	3.80	33.26	23.05	8.78	93.17	24.10	21.45
$r_{\mathrm{fg}}{=}2$ (default)	4.11	32.64	24.40	9.61	93.00	24.21	21.51

Appendix

Table 11. Ablation on the interior-support dilation radius

r_{\mathrm{fg}}

on PIE-Bench (scaled to match Tab. 1 ). The radius controls a preservation–editability trade-off;

r_{\mathrm{fg}}=2

is the default operating point used in all main experiments.

Setting	Structure	Background Preservation				CLIP Similarity $\uparrow$
Setting	Distance $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	MSE $\downarrow$	SSIM $\uparrow$	CLIP ${}_{\text{whole}}\uparrow$	CLIP ${}_{\text{edit}}\uparrow$
$\beta_{\max}{=}0.60$	4.10	32.64	24.39	9.60	93.00	24.20	21.51
$\beta_{\max}{=}0.15$	6.10	32.14	26.84	10.73	92.70	24.39	21.73
$\beta_{\max}{=}0.00$ (no ring coupling)	6.12	32.13	26.87	10.74	92.69	24.40	21.73
$\beta_{\max}{=}0.45$ (default)	4.11	32.64	24.40	9.61	93.00	24.21	21.51

Appendix

Table 12. Ablation on maximum ring coupling

\beta_{\max}

\beta(\sigma_{t})

(Eq. ( 34 )) on PIE-Bench (scaled to match Tab. 1 ).

Setting	Structure	Background Preservation				CLIP Similarity $\uparrow$
Setting	Distance $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	MSE $\downarrow$	SSIM $\uparrow$	CLIP ${}_{\text{whole}}\uparrow$	CLIP ${}_{\text{edit}}\uparrow$
$r_{\mathrm{bd}}{=}2$	4.12	32.63	24.42	9.62	93.00	24.19	21.51
$r_{\mathrm{bd}}{=}6$	4.09	32.65	24.39	9.59	93.00	24.19	21.51
$r_{\mathrm{bd}}{=}0$ (no ring band)	4.13	32.62	24.46	9.63	92.99	24.19	21.52
$r_{\mathrm{bd}}{=}4$ (default)	4.11	32.64	24.40	9.61	93.00	24.21	21.51

Appendix

Table 13. Ablation on boundary-band dilation radius

r_{\mathrm{bd}}

used to form

W_{t}^{\mathrm{ring}}

(Eq. 68 ) on PIE-Bench (scaled to match Tab. 1 ).

Panel of Figure 13 — Figure 13. Qualitative transfer of our Stage-2 where/how control to FlowEdit. Left: SD3. Right: FLUX. Each column shows (Original, FlowEdit, and OAVC applied to FlowEdit) on three examples.

Figure 14. Representative weak-benefit case 1 for OAVC.

Figure 15. Representative weak-benefit case 2 for OAVC.

Panel of Figure 16 — Figure 16. Qualitative comparison of image editing methods on 10 samples.

Panel of Figure 17 — Figure 17. Qualitative comparison of 5 image editing methods.

Figure 18. Qualitative video editing results on DAVIS using Wan2.1. OAVC reduces background drift and improves temporal consistency by constraining semantic updates to the target object region (and a safe injection direction), compared to global velocity steering baselines.

Panel of Figure 19 — Figure 19. Qualitative comparison on Stable Diffusion (left) and FLUX (right).

Image Editing

Image editing changes selected visual content while preserving the parts of an image that should remain unchanged.

See all 315 papers

VeloEdit: Training-Free Consistent and Continuous Instruction-Based Image Editing via Velocity Field Decomposition
Mar 11, 2026Zongqing Li, Zhihui Liu, Yujie Xie +3Training-Free Image Editing Image Editing
Xiamen University · Truesight
DecFlowEdit: Self-Localized Flow-based Image Editing via Guidance Decoupling
Sep 28, 2026Zheyuan Zhan, Can Wang, Jiawei Chen +4Flow Matching Classifier-Free Guidance
Zhejiang University · University at Buffalo · University of California, Berkeley
IDAG-Edit: Multi-Object Video Editing via Instance-Decoupled Attention and Guidance
Jun 20, 2026Yuan-Zhih Lin, Huu-Thang Nguyen, Huu-Phu Do +2Diffusion Model Guidance Video Editing
Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan

Mar 11, 2026J/K move · Enter open · S save

VeloEdit: Training-Free Consistent and Continuous Instruction-Based Image Editing via Velocity Field Decomposition

Zongqing Li, Zhihui Liu, Yujie Xie, Shansiyuan Wu+2

Xiamen University · Truesight

Training-Free Image Editing Image Editing Text-Guided Image Editing Flow Matching

Open paper AskPDF arXiv

Instruction-based image editing aims to modify source content according to textual instructions. However, existing methods built upon flow matching often struggle to maintain consistency in non-edited regions due to denoising-induced reconstruction errors that cause drift in preserved content. Moreover, they typically lack fine-grained control over edit strength. To address these limitations, we propose VeloEdit, a training-free method that enables highly consistent and continuously controllable editing. VeloEdit dynamically identifies editing regions by quantifying the discrepancy between the velocity fields responsible for preserving source content and those driving the desired edits. Based on this partition, we enforce consistency in preservation regions by substituting the editing velocity with the source-restoring velocity, while enabling continuous modulation of edit intensity in target regions via velocity interpolation. Unlike prior works that rely on complex attention manipulation or auxiliary trainable modules, VeloEdit operates directly on the velocity fields. Extensive experiments on Flux.1 Kontext and Qwen-Image-Edit demonstrate that VeloEdit improves visual consistency and editing continuity with negligible additional computational cost. Code is available at https://github.com/xmulzq/VeloEdit.

Abstract

Image Editing

Explore similar work

VeloEdit: Training-Free Consistent and Continuous Instruction-Based Image Editing via Velocity Field Decomposition

DecFlowEdit: Self-Localized Flow-based Image Editing via Guidance Decoupling

IDAG-Edit: Multi-Object Video Editing via Instance-Decoupled Attention and Guidance