Object-Aware Background-Controlled Editing via Weighted Velocity Guidance
Authors: Wuji Wang, Yue Wu, Chenhao Yi, Shuhui Wang
Organizations: Language Technologies Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Department of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China
Training-free image editing steers diffusion or flow-matching generative models at inference time by modifying prompt-conditioned denoising velocities. Existing velocity-based editors often apply prompt-induced residuals globally over the latent space and rely on the model to localize semantic changes implicitly. For object-centric edits, these residuals are rarely zero outside the target object, so small non-target components can accumulate during multi-step integration, causing background drift and unstable object boundaries. We propose Object-Aware Velocity Control (OAVC), a training-free framework that introduces object-level control into the velocity-integration process. OAVC decouples where semantic residuals are allowed to act from how they are injected into the dynamics. It constructs a background-anchored reference interface under the source prompt and then performs object-localized safe semantic injection under the target prompt. A constrained injection operator suppresses drift-inducing velocity components, while time-adaptive spatial weighting stabilizes the transition near object boundaries. OAVC requires no training or modification of pretrained model parameters. Experiments on object-centric image and video benchmarks with image and video rectified-flow backbones show improved background preservation, structural fidelity, boundary stability, and temporal consistency while retaining effective localized editability.
Figures & tables
Figure 1. Object-centric image and video editing. Our approach enables localized edits while keeping non-target regions stable during multi-step integration. Prior training-free velocity-steering methods often accumulate small background residuals, leading to background drift, boundary artifacts, and temporal inconsistency. By enforcing object-localized semantic updates with explicit non-target preservation, we produce cleaner object removal/inpainting, faithful attribute or color changes, and temporally consistent video edits.
Figure 2. Illustration of Stage-2: Projected and Constrained Semantic Injection. We explicitly constrain the edit to the object region and a safe injection direction.
Figure 3. Overview of Object-Aware Velocity Control. Global steering applies prompt-induced residuals over the full latent trajectory, causing non-target accumulation and background drift. OAVC first constructs a background-anchored reference interface, then applies explicit where/how control: object supports specify where semantic residuals are integrated, while a safe injection operator specifies how they are transformed before solver integration.
Figure 4. Qualitative comparison on PIE-Bench. We compare OAVC against representative training-free editors across SD3.5 and FLUX. OAVC better preserves non-target background and object boundaries while achieving the intended edits (red boxes).
Method
Model
Structure
Background Preservation
CLIP Similarity
Distance ↓
PSNR ↑
LPIPS ↓
MSE ↓
SSIM ↑
Whole ↑
Edited ↑
PnP ( Tumanyan et al., 2023 )
SD1.5
27.35
22.31
112.76
82.95
79.25
25.41
22.52
DI+PnP ( Ju et al., 2023 )
SD1.5
23.35
22.46
105.51
79.94
79.88
25.49
22.64
InfEdit ( Xu et al., 2024 )
LCM
19.31
27.31
56.32
47.80
85.30
24.90
22.14
RF-Inv ( Rout et al., 2025 )
FLUX
42.29
20.20
179.54
139.20
69.91
24.57
22.20
RFEdit ( Wang et al., 2024 )
FLUX
21.79
24.83
113.15
52.46
83.38
25.57
22.26
Table 1. Quantitative comparison on PIE-Bench. OAVC improves structural fidelity and background preservation while retaining effective localized editability. The best and second-best results are indicated by bold and underlined text, respectively.
Setting
Structure
Background Preservation
CLIP Similarity
Distance ↓
PSNR ↑
LPIPS ↓
MSE ↓
SSIM ↑
CLIP whole↑
CLIP edit↑
DNAEdit (baseline; global)
14.19
26.66
74.57
32.76
88.63
25.63
22.71
Stage-1 only (Anchor-DNA)
14.68
26.28
77.75
35.44
88.42
25.76
22.87
Stage-2 (Mask+Ring+Proj; w/o Gate)
6.60
31.74
27.65
11.95
92.61
24.60
21.97
Stage-2 (+Mask only; Wedit )
5.27
32.27
25.78
10.48
92.99
24.55
21.87
DNAEdit + final latent hard blend
6.28
–
24.29
9.56
–
24.46
21.98
Table 2. PIE-Bench ablation (all metrics scaled to match Tab. 1 ). We report the SD3.5 setting. The best results are indicated by bold text.
Method
BG-PSNR ↑
BG-L1 ↓
TW-BG-PSNR ↑
TW-BG-L1 ↓
DNAEdit
19.85
0.0692
26.56
0.0281
UniEdit-Flow
22.87
0.0467
27.97
0.0227
OAVC
27.53
0.0284
29.65
0.0212
Table 3. DAVIS quantitative video results. BG metrics measure per-frame background fidelity, while TW-BG metrics measure temporal background stability using source-video optical flow.
Figure 5. DAVIS video editing example for target-object removal across consecutive frames.
Method
Backbone
Structure
Background Preservation
CLIP Similarity ↑
Distance ↓
PSNR ↑
LPIPS ↓
MSE ↓
SSIM ↑
CLIP whole↑
CLIP edit↑
FlowEdit
FLUX
27.82
21.96
112.19
94.99
83.08
25.25
22.58
FlowEdit
SD3
27.12
22.22
104.12
85.96
93.22
26.53
23.57
Ours (OAVC on FlowEdit)
SD3
35.72
27.65
81.05
21.57
86.98
24.46
22.20
Ours (OAVC on FlowEdit)
FLUX
9.32
30.77
30.09
16.80
93.57
24.18
23.66
Table 4. Transfer to FlowEdit backbones on PIE-Bench (all metrics scaled to match Tab. 1 ).
Method
Struct. ↓
BG LPIPS ↓
BG MSE ↓
CLIP whole↑
CLIP edit↑
DNAEdit (global)
14.19
74.57
32.76
25.63
22.71
Final latent hard blend
6.28
24.29
9.56
24.46
21.98
Mask-only velocity gating
5.27
25.78
10.48
24.55
21.87
Full OAVC
4.11
24.40
9.61
24.21
21.51
Table 5. Controlled comparison on PIE-Bench using SD3.5. All localized variants use the identical cached SAM3 support, thereby isolating the effect of trajectory-level control from that of object localization. Metrics follow the scaling convention of Table 1 .
Criterion
OAVC
DNAEdit
Final blend
None/Tie
No consensus
Edit realization
15
22
28
17
18
Background preservation
74
0
20
0
6
Absolute task success
Yes
Partially
Yes+Partially
No
–
OAVC
59.0%
20.5%
79.5%
20.5%
–
DNAEdit
65.5%
14.5%
80.0%
20.0%
–
Final blend
55.5%
21.5%
77.0%
23.0%
–
Table 6. Human evaluation. Top: majority-vote preference counts over 100 examples. Bottom: absolute task-success rates over 200 judgments per method.
Input-prior perturbation
IoU w.r.t. M0
Area ratio
Empty
Struct. ↓
BG LPIPS ↓
BG MSE ↓
CLIP edit↑
Erode 4
0.397
0.397 ×
13%
2.07
19.53
6.35
20.65
Erode 2
0.593
0.593 ×
6%
2.84
20.10
6.55
21.29
Original
1.000
1.000 ×
0%
4.21
23.15
8.08
21.47
Dilate 2
0.712
1.581 ×
0%
4.67
26.04
9.34
21.36
Dilate 4
0.594
2.236 ×
0%
4.93
28.88
10.38
21.42
Table 7. Robustness to input-prior corruption on 100 PIE-Bench examples. The cached SAM3 prior M0 is eroded or dilated before the unchanged OAVC support-construction step. All conditions retain rfg=2 and rbd=4 . IoU and area ratio characterize Mr relative to M0 ; region-conditioned metrics use the same independent PIE-Bench annotation, while Structure Distance is global. Metrics are scaled as in Table 1 .
Figure 6. Robustness to generation-time input-prior corruption on a fixed 100-example PIE-Bench subset. Points denote means and error bars denote bootstrap 95% confidence intervals. The dashed line marks the unperturbed input prior ( r=0 ). All conditions retain the same OAVC support-construction parameters, including rfg=2 and rbd=4 . Region-conditioned metrics use the same independent PIE-Bench annotation; Structure Distance is global. Erosion restricts edit coverage, whereas dilation increasingly exposes non-target regions to semantic residuals.
Figure 7. Qualitative examples on GEdit-Bench. Columns show Original, DNAEdit, and OAVC. Top: microphone → ice-cream cone. Bottom: remove the white railing. OAVC better localizes the requested edit and preserves surrounding content. These two cases provide a qualitative generalization check, not full-benchmark evaluation.
Method
FG-Gain
BG-Gain
FG-BG Gap
Mean ↑
Std
Mean ↓
Std
Mean ↑
Std
DNAEdit (global)
0.0189
0.0299
0.0108
0.0245
0.0080
0.0336
OAVC
0.0130
0.0261
0.0008
0.0123
0.0122
0.0260
Table 8. Foreground/background target-semantic decomposition. FG-Gain and BG-Gain measure target-prompt gain in foreground and background regions, respectively, while FG-BG Gap measures localization strength.
Figure 8. Large-deformation object replacements from the same source image. Rows show car-to-train and car-to-plane edits, while columns compare the original image, DNAEdit, and OAVC. DNAEdit introduces more extensive changes to the road and surrounding scene and, in the car-to-plane case, incompletely removes the source car. OAVC concentrates the transformation around the source object while more closely preserving non-target scene content.
Figure 9. Representative two-object edit from a goat and a cat to a horse and a dog. The orange and blue overlays indicate the source goat and cat supports, respectively; their union is used as the editable support. Both methods modify the two target objects, while OAVC more closely retains the original scene layout and non-target background.
Backbone/Method
Time (s/img) ↓
Peak Mem (MiB) ↓
SD3.5 / DNAEdit
3.08
19,459
SD3.5 / Ours
3.12
23,139
FLUX / DNAEdit
9.01
35,839
FLUX / Ours
9.85
39,571
Table 9. Runtime and peak GPU memory (per image) for DNAEdit vs. OAVC on two rectified-flow backbones.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10. Interactive brush refinement for challenging support extraction cases. When the automatic SAM3 support is imperfect, users can manually refine the editable region with a brush, and OAVC applies the same velocity-control mechanism using the corrected support.
Support source
Struct.
Background Preservation
CLIP Similarity
Distance ↓
PSNR ↑
LPIPS ↓
MSE ↓
SSIM ↑
CLIP whole↑
CLIP edit↑
DNAEdit (global)
18.87
24.99
95.06
50.45
85.71
25.79
22.87
Attention-derived support
4.58
30.55
35.74
16.41
92.53
23.60
20.76
SAM3 support (default)
4.07
33.30
23.70
8.75
94.38
23.69
21.08
Appendix
Table 10. Support-source comparison on PIE-Bench. All metrics are scaled to match the main tables. Attention-derived support can replace SAM3 and still provides strong preservation gains over global DNAEdit, showing that OAVC is support-source agnostic.
Figure 11. Qualitative examples using attention-derived support instead of SAM3. Attention maps provide an internal support prior for OAVC, enabling object-localized velocity control without an external segmenter. Although attention-derived supports are coarser than SAM3 masks, they remain sufficient to reduce background drift and preserve non-target regions compared with global steering.
Figure 12. Stage-1 hyperparameter sweep on PIE-Bench for SD3.5 and FLUX. We vary foreground relaxation λfg and background suppression ρbg and report background preservation (LPIPS ↓ , MSE ↓ on non-target regions; scaled as shown) together with semantic alignment (CLIP whole /CLIP edit ). The dashed boxes highlight the stable region around our default setting ( λfg=0.6 , ρbg=0.1 ).
Setting
Structure
Background Preservation
CLIP Similarity ↑
Distance ↓
PSNR ↑
LPIPS ↓
MSE ↓
SSIM ↑
CLIP whole↑
CLIP edit↑
rfg=3
4.42
32.22
25.60
10.26
92.86
24.29
21.50
rfg=1
3.15
34.10
21.68
7.86
93.35
23.83
21.25
rfg=0
3.80
33.26
23.05
8.78
93.17
24.10
21.45
rfg=2 (default)
4.11
32.64
24.40
9.61
93.00
24.21
21.51
Appendix
Table 11. Ablation on the interior-support dilation radius rfg on PIE-Bench (scaled to match Tab. 1 ). The radius controls a preservation–editability trade-off; rfg=2 is the default operating point used in all main experiments.
Setting
Structure
Background Preservation
CLIP Similarity ↑
Distance ↓
PSNR ↑
LPIPS ↓
MSE ↓
SSIM ↑
CLIP whole↑
CLIP edit↑
βmax=0.60
4.10
32.64
24.39
9.60
93.00
24.20
21.51
βmax=0.15
6.10
32.14
26.84
10.73
92.70
24.39
21.73
βmax=0.00 (no ring coupling)
6.12
32.13
26.87
10.74
92.69
24.40
21.73
βmax=0.45 (default)
4.11
32.64
24.40
9.61
93.00
24.21
21.51
Appendix
Table 12. Ablation on maximum ring coupling βmax in β(σt) (Eq. ( 34 )) on PIE-Bench (scaled to match Tab. 1 ).
Setting
Structure
Background Preservation
CLIP Similarity ↑
Distance ↓
PSNR ↑
LPIPS ↓
MSE ↓
SSIM ↑
CLIP whole↑
CLIP edit↑
rbd=2
4.12
32.63
24.42
9.62
93.00
24.19
21.51
rbd=6
4.09
32.65
24.39
9.59
93.00
24.19
21.51
rbd=0 (no ring band)
4.13
32.62
24.46
9.63
92.99
24.19
21.52
rbd=4 (default)
4.11
32.64
24.40
9.61
93.00
24.21
21.51
Appendix
Table 13. Ablation on boundary-band dilation radius rbd used to form Wtring (Eq. 68 ) on PIE-Bench (scaled to match Tab. 1 ).
Figure 13. Qualitative transfer of our Stage-2 where/how control to FlowEdit. Left: SD3. Right: FLUX. Each column shows (Original, FlowEdit, and OAVC applied to FlowEdit) on three examples.
Figure 14. Representative weak-benefit case 1 for OAVC.
Figure 15. Representative weak-benefit case 2 for OAVC.
Figure 16. Qualitative comparison of image editing methods on 10 samples.
Figure 17. Qualitative comparison of 5 image editing methods.
Figure 18. Qualitative video editing results on DAVIS using Wan2.1. OAVC reduces background drift and improves temporal consistency by constraining semantic updates to the target object region (and a safe injection direction), compared to global velocity steering baselines.
Figure 19. Qualitative comparison on Stable Diffusion (left) and FLUX (right).
Instruction-based image editing aims to modify source content according to textual instructions. However, existing methods built upon flow matching often struggle to maintain consistency in non-edited regions due to denoising-induced reconstruction errors that cause drift in preserved content. Moreover, they typically lack fine-grained control over edit strength. To address these limitations, we propose VeloEdit, a training-free method that enables highly consistent and continuously controllable editing. VeloEdit dynamically identifies editing regions by quantifying the discrepancy between the velocity fields responsible for preserving source content and those driving the desired edits. Based on this partition, we enforce consistency in preservation regions by substituting the editing velocity with the source-restoring velocity, while enabling continuous modulation of edit intensity in target regions via velocity interpolation. Unlike prior works that rely on complex attention manipulation or auxiliary trainable modules, VeloEdit operates directly on the velocity fields. Extensive experiments on Flux.1 Kontext and Qwen-Image-Edit demonstrate that VeloEdit improves visual consistency and editing continuity with negligible additional computational cost. Code is available at https://github.com/xmulzq/VeloEdit.
Flow-based image editing (FlowEdit) enables inversion-free semantic changes through the difference between source and target velocities. In this paper, we observe that FlowEdit's default classifier-free guidance (CFG) configuration, with asymmetric source and target scales, causes substantial background leakage. Matching these guidance scales, for example by removing CFG, improves edit-relevant localization but severely degrades editability. To get the best of both worlds, we propose DecFlowEdit, which decouples the optimal guidance scales for localization and for editing in flow-based generative models. In particular, DecFlowEdit first extracts an edit-relevant prior by temporally aggregating velocity differences evaluated without CFG, and then uses this prior to reweight the original updates under default CFG. Our method remains training-free and inversion-free, requiring neither external spatial masks nor attention manipulation. Experiments on PIE-Bench across FLUX, SD3, and SD3.5 show that DecFlowEdit improves background preservation, reducing structure distance by approximately 61 to 73 percent and background LPIPS by 68 to 80 percent relative to FlowEdit at comparable editing fidelity.
Zheyuan Zhan, Can Wang, Jiawei Chen +4
Zhejiang University · University at Buffalo · University of California, Berkeley
Diffusion-based video editing has made significant progress; however, achieving precise and temporally consistent object-level control, especially in multi-object scenarios, remains challenging due to attention leakage, identity drift, and unstable temporal dynamics. In this work, we propose IDAGEdit, a training-free framework for fine-grained multi-object video editing with strong temporal consistency. The framework adopts Layout-guided Attention Modulation to facilitate coherent multi-object editing, while Instance-level Masks are introduced to preserve individual object identity and enforce localized attention within each object region, thereby enabling fine-grained, object-level editing. Extensive qualitative and quantitative evaluations demonstrate that our method improves temporal stability and multi-object controllability over state-of-the-art video editing approaches.
Yuan-Zhih Lin, Huu-Thang Nguyen, Huu-Phu Do +2
Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan