One-step text-guided diffusion editing is efficient but prone to spatially misallocated updates that distort the edited object and alter the background. Existing methods often improve stability by averaging the editing field across timesteps. We instead identify spatial energy misallocation as a distinct and measurable failure mode: across two independent noise draws, the residual field is essentially unrepeatable, making the background field unreliable for direct transport, while its total energy still sets a usable magnitude for the draw at hand. BudEdit turns that magnitude into an explicit budget and reallocates it to edit-relevant regions selected jointly by residual energy and cross-attention, controlling where editing energy is spent rather than averaging over timesteps. The resulting training-free, inversion-free editor spends the budget on transport and reuses it to scale a correction in a lower-noise gated refinement. The budgeted injection field matches its prescribed budget exactly and vanishes on the identified background support, by construction. On PIE-Bench with SD-Turbo, BudEdit outperforms ChordEdit under each method's reported default settings on all 11 evaluated metrics, including a 2.1,dB gain in background PSNR, 31% lower DINO, and 36% lower LPIPS, while improving all five editing-quality metrics and reporting the lowest runtime in the comparison.
Figures & tables
Figure 2: One-step editors on PIE-Bench. PSNR-bg vs. CLIP-D, bubble size = runtime; BudEdit leads both axes and reports the lowest runtime in this comparison.
Figure 3: Comparison of editing paradigms. (a) multi-step editing buys stability by spreading the update over many small steps; (b) one-step editing integrates the measured field in a single large step; (c) chord transport forms the one-step update from a time-weighted average of residual fields at two timesteps; (d) energy transport converts background residual energy into a budget and spends it inside the effective injection support.
Table 1: Quantitative comparison on PIE-Bench. T-free: Training-free. I-free: Inversion-free. Best and second-best among one-step methods are highlighted in yellow and blue , ties included.
Table 2: ChordEdit chord window ablation. δ=0.15 is default; δ=0 disables the window.
Figure 6: Annotation-free localization. Two edits, from birds to origami birds and from a dog to a lion. The columns show the Real Image, Energy Mask, Edit-Attention Map, Derived Mask, GT Mask, Overlap, background energy, field energy of the composed transport field, and output.
Table 3: Factorial ablation of energy reallocation and cleanup. Bold: column best.
Dataset
Backbone
PSNR ↑
CLIP-E ↑
ChordEdit
BudEdit
ChordEdit
BudEdit
PIE-Bench
SD-Turbo
22.63
24.71
22.15
22.52
SwiftBrushV2
22.93
25.49
21.18
21.26
PIE-Bench++
SD-Turbo
22.87
25.06
24.12
24.27
SwiftBrushV2
23.16
25.83
22.46
22.70
Table 4: Quantitative comparison on different datasets and backbones. Bold: column best.
Text-guided diffusion image editing aims to modify semantic attributes of an image while preserving its identity, layout, and background. However, naïvely switching the text condition during sampling often causes global drift, as denoising dynamics propagate changes across tokens and can disrupt unedited regions. To address this issue, we propose Asynchronous Token Decoding Edit (ATDEdit), an inference-time framework that views each sampler step as a parallel update of a globally coupled token matrix and enables token-indexed condition switching with differentiated update policies. Instead of applying synchronous target-conditioned updates to all tokens, ATDEdit estimates editable locations using token-wise conditional surprisal and applies target-conditioned corrections to the selected token set. It supplies source key/value memory at keep-token positions and projects selected keep-token latent rows back to their source values; these operations promote background preservation but do not constitute a pixel-level invariance guarantee. This approach combines local editing and background preservation without external or user-provided spatial masks and without model fine-tuning. On PIE-Bench, ATDEdit achieves the strongest reported preservation metrics, including 27.44~dB PSNR and 0.055 LPIPS, while retaining competitive semantic alignment.
Yang Shi, Liangsi Lu, Minzhe Guo +4
Guangdong University of Technology Guangzhou, China · Hong Kong Baptist University Hong Kong, China · Peking University Beijing, China +1
Diffusion-based text-to-image models have recently been distilled into few-step samplers and fast generators, reducing inference cost while maintaining high image quality. However, text-guided editing with such accelerated backbones remains challenging: inversion in a small number of steps is unreliable, local errors accumulate along the trajectory, and existing fast-editing methods typically rely on backbone-specific inversion procedures, which limits their transferability. In this work, we propose a simple and universal framework for text-guided image editing with arbitrary accelerated diffusion and flow-matching models. Instead of optimizing local, per-timestep objectives, we directly minimize a global editing objective over the entire inversion-generation trajectory. The framework fine-tunes only 1.5-7 % of the base-model parameters through two lightweight LoRA adapters (forward and backward), trained on fewer than 10 K triplets from the publicly available HQ-Edit dataset. We validate our method on three architecturally distinct backbones: consistency-distilled SD1.5 LCM-Dreamshaper, adversarial-distilled SDXL-Turbo, and rectified-flow-distilled Z-Image-Turbo. On PIE-Bench, our method achieves the strongest editing alignment within each backbone family while ensuring a strong level of content preservation. Inference adds no extra steps beyond the standard inversion and editing passes. The project page with visual results is available at iliabeletskii.github.io/inverse-and-edit.
One-step diffusion editors are fast because they avoid inversion and iterative optimization, but a single transport update must be aggressive enough to realize the target prompt and conservative enough to preserve the source image--and no fixed update strength satisfies both demands across edit types. We treat this tension as a post-hoc candidate-selection problem on top of energy-field transport rather than as a new editing model. Our proposed method, Riemannian Residual Line Search, first builds a stronger edit by estimating the local time curvature of the prompt-delta field and projecting the corrected direction back onto the update norm of the original first-order energy-field transport estimation. It then forms a small residual path from the source image to this strong edit, retains the original first-order output as one candidate, and picks the final image by maximizing target-prompt CLIP alignment. On a 700-sample PIE-Bench++ evaluation across 10 edit type IDs, our method achieves state-of-the-art (SOTA) performance among current one-step update algorithms.