cs.CVMar 22, 2026

GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editing

Authors: Zifeng Zhu, Jiaming Han, Jiaxiang Zhao, Minnan Luo, Xiangyu Yue

Organizations: Xi’an Jiaotong University · MMLab, The Chinese University of Hong Kong

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

While Diffusion Large Language Models (DLLMs) have demonstrated remarkable capabilities in multi-modal generation, performing precise, training-free image editing remains an open challenge. Unlike continuous diffusion models, the discrete tokenization inherent in DLLMs hinders the application of standard noise inversion techniques, often leading to structural degradation during editing. In this paper, we introduce GIDE (Grounded Inversion for DLLM Image Editing), a unified framework designed to bridge this gap. GIDE incorporates a novel Discrete Noise Inversion mechanism that accurately captures latent noise patterns within the discrete token space, ensuring high-fidelity reconstruction. We then decompose the editing pipeline into grounding, inversion, and refinement stages. This design enables GIDE supporting various editing instructions (text, point and box) and operations while strictly preserving the unedited background. Furthermore, to overcome the limitations of existing single-step evaluation protocols, we introduce GIDE-Bench, a rigorous benchmark comprising 805 compositional editing scenarios guided by diverse multi-modal inputs. Extensive experiments on GIDE-Bench demonstrate that GIDE significantly outperforms prior training-free methods, improving Semantic Correctness by 51.83% and Perceptual Quality by 50.39%. Additional evaluations on ImgEdit-Bench confirm its broad applicability, demonstrating consistent gains over trained baselines and yielding photorealistic consistency on par with leading models.

Figures & tables

Figure 1: Demonstration of our proposed GIDE framework . Given a source image (left of each pair) and an editing instruction guided by flexible spatial modalities (points, bounding boxes, or pure text), GIDE enables precise and localized image editing. The edited results (right of each pair) demonstrate high-quality visual synthesis and strict background preservation, all achieved in a completely training-free manner.
Figure 2: The overall architecture of GIDE . It decouples the editing process into three sequential stages: Grounding locates the target region via a segmentation foundation model; Inversion executes the edit by reconstructing the discrete latent space; and Refinement enhances the visual coherence and fidelity of the final output.
Figure 3: Statistical overview of GIDE-Bench . It contains 805 cases and 1,610 sub-instructions (left), stratified across three input modalities (middle). The right chart shows the distribution of compositional editing pairs (Rem.: Remove, Rep.: Replace).
Method Non-edit EditGPT\textbf{Edit}_{\mathrm{GPT}} EditGemini\textbf{Edit}_{\mathrm{Gemini}}
 MSE ↓\downarrowPSNR ↑\uparrowSSIM ↑\uparrow SC ↑\uparrowPQ ↑\uparrow SC ↑\uparrowPQ ↑\uparrow
End-to-End Image Editing Models
OneDiffusion [ 26 ] 1247.7420.610.6831 1.811.79 1.941.81
InstructPix2Pix [ 2 ] 2540.0617.710.7108 1.891.86 1.871.68
MagicBrush [ 72 ] 2410.4419.360.7210 2.132.04 2.081.74
Lumina-DiMOO [ 66 ] 1208.2219.800.6461 3.102.92 3.102.34
Table 1: Performance comparison on GIDE-Bench . Best results within each group are highlighted in bold . Gray indicates our method. Notably, our GIDE integrated with Lumina-DiMOO achieves state-of-the-art performance among training-free methods, steadily narrowing the performance gap to top-tier fully supervised models.
Figure 4: Qualitative comparison on GIDE-Bench . GIDE accurately follows instructions and preserves structural fidelity, avoiding unintended alterations (e.g., DICE) and aspect ratio distortions (e.g., GPT-Image-1) while achieving photorealistic consistency. Zoom in for better view.
MethodRep. ↑\uparrowAdd ↑\uparrowRem. ↑\uparrow
MagicBrush [ 72 ]1.972.841.58
AnyEdit [ 71 ]2.473.182.23
UltraEdit [ 74 ]2.963.441.45
ICEdit [ 73 ]3.153.582.93
Step1X-Edit [ 32 ]3.403.882.41
Table 2: Evaluation on ImgEdit [ 70 ] benchmark . Rep.: Replace, Rem.: Remove.
Figure 5: Visual ablation on the proposed components . Removing the discrete inversion degrades structural fidelity and distorts object shapes. Omitting the spatial grounding leads to the loss of background context. Disabling the visual refinement blurs local details and compromises the seamless preservation of the unedited regions.
MethodNon-edit EditGPT\textbf{Edit}_{\mathrm{GPT}} EditGemini\textbf{Edit}_{\mathrm{Gemini}}
MSE ↓\downarrowPSNR ↑\uparrowSSIM ↑\uparrow SC ↑\uparrowPQ ↑\uparrow SC ↑\uparrowPQ ↑\uparrow
Ours (Full)1224.8920.400.7083 4.473.98 4.263.78
w/o discrete inversion2196.29 79%↑17.85 13%↓0.6628 6%↓ 4.30 4%↓3.83 4%↓ 4.08 4%↓3.28 13%↓
w/o spatial grounding8446.17 590%↑9.42 54%↓0.4161 41%↓ 3.60 19%↓3.43 14%↓ 2.68 37%↓2.21 42%↓
w/o visual refinement2476.78 102%↑17.23 16%↓0.6444 9%↓ 3.73 17%↓3.25 18%↓ 3.56 16%↓2.67 29%↓
w/o intrinsic refinement1424.29 16%↑19.80 3%↓0.6987 1%↓ 3.96 11%↓3.51 12%↓ 3.87 9%↓3.11 18%↓
Table 3: Ablation experiments . We compare the full method against variants with specific modules removed. The results confirm that the full framework achieves the best performance in both background preservation and editing quality.
λ\lambda Non-edit EditGPT\textbf{Edit}_{GPT} EditGemini\textbf{Edit}_{Gemini}
 MSE ↓\downarrowPSNR ↑\uparrowSSIM ↑\uparrow SC ↑\uparrowPQ ↑\uparrow SC ↑\uparrowPQ ↑\uparrow
0.0 1224.6220.350.7086 4.303.86 4.123.71
0.1 1306.4720.220.7066 4.263.82 4.123.69
0.2 1224.8920.400.7083 4.473.98 4.263.78
0.3 1308.5720.330.7075 4.283.85 4.093.70
0.4 1225.4020.510.7084 4.213.83 4.033.75
Table 4: Sensitivity analysis of the mixing coefficient λ\lambda . Best scores are highlighted in bold . The default setting λ=0.2\lambda=0.2 is highlighted in gray. GIDE achieves optimal editing quality at λ=0.2\lambda=0.2 with stable performance across the range.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: An example of Mask Relaxation. Expanding the tight source mask Msrc\mathbf{M}_{\text{src}} to its bounding box accommodates noticeable shape variations (e.g., bicycle to bench), ensuring complete object generation while preserving the unedited background.
Figure 7: An example of VQModel reconstruction characteristics. GIDE executes semantic edits accurately, but the VQModel’s decompression introduces minor color variations in high-frequency details like the eyes. Zoom in for better view.
MethodGeneration Setup
GIDE +Lumina-DiMOOcfg_scale = 4.0, timesteps = 64, temperature = 1.0
GIDE +MMaDAcfg_scale = 4.0, timesteps = 64, temperature = 1.0
DICE+Lumina-DiMOOcfg_scale = 4.0, timesteps = 64, temperature = 1.0
DirectInversion+PnPguidance_scale = 7.5, num_steps = 50
DirectInversion+P2Pguidance_scale = 7.5, cross_replace_steps = 0.4, self_replace_steps = 0.6
Edit-R1true_cfg_scale = 4.0, num_inference_steps = 40
Appendix
Table 5: Generating parameters for training-free and training-based methods.
More in this topic

Training-Free Image Editing

See all 19 papers

Explore similar work

CardsList
Feb 24, 2026

Training-Free Multi-Concept Image Editing

Training-free image editing with diffusion models is highly desirable yet is complex and remains a significant challenge. While recent optimisation-based methods achieve strong zero-shot edits from text, they still struggle to preserve identity and capture intricate details, such as facial structure, surface texture, or object-specific geometry, that exist below the level of linguistic abstraction. To address this fundamental gap, we propose Concept Distillation Sampling (CDS). To the best of our knowledge, we are the first to introduce a unified, training-free framework for target-less, multi-concept image editing. CDS overcomes this linguistic bottleneck of previous methods by anchoring the editing process in the certainty of pretrained LoRA adapters. We integrate a highly stable distillation backbone (featuring ordered timesteps, regularisation, and negative-prompt guidance) with a novel dynamic weighting mechanism. This approach enables the composition and control of multiple visual concepts directly within the diffusion process, utilising spatially-aware priors from pretrained LoRA adapters without causing concept clashing. Our method preserves instance concept identity without requiring reference samples of the desired edit. Extensive quantitative and qualitative evaluations demonstrate that CDS establishes a new state-of-the-art over existing training-free editing and multi-LoRA composition methods on the InstructPix2Pix and ComposLoRA benchmarks. Project Page: https://nickyfot.github.io/cds/.

Niki Foteinopoulou, Ignas Budvytis, Stephan Liwicki

Cambridge Research Laboratory, Toshiba Europe, Cambridge, UK · Independent Researcher, UK

Training-Free Image EditingImage EditingDiffusion-Based Image EditingDiffusion Model GuidanceCompositional Image Generation
AskPDF
May 11, 2026

Masked Generative Transformer Is What You Need for Image Editing

Diffusion models dominate image editing, yet their global denoising mechanism entangles edited regions with surrounding context, causing modifications to propagate into areas that should remain intact. We propose a fundamentally different approach by leveraging Masked Generative Transformers (MGTs), whose localized token-prediction paradigm naturally confines changes to intended regions. We present EditMGT, an MGT-based editing framework that is the first of its kind. Our approach employs multi-layer attention consolidation to aggregate cross-attention maps into precise edit localization signals, and region-hold sampling to explicitly prevent token flipping in non-target areas. To support training, we construct CrispEdit-2M, a 2M-sample high-resolution (>1024) editing dataset spanning seven categories. With only 960M parameters, EditMGT achieves state-of-the-art image similarity on multiple benchmarks while delivering 6x faster editing, demonstrating that MGTs offer a compelling alternative to diffusion-based editing.

Wei Chow, Linfeng Li, Xian Sun +14

1ByteDance · 2National University of Singapore · 3Duke University +2

TransformerText-Guided Image EditingImage Editing
AskPDF
Sep 17, 2026

Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network

Text-guided image editing must introduce the requested changes while preserving unrelated source content. In training-free editing, diffusion editors often use spatial controls whose inaccuracies can leave edits incomplete or alter unrelated regions. Causal autoregressive editors face a further constraint: their fixed decoding order limits revision of earlier decisions. As the first to explore training-free image editing with Generative Refinement Networks (GRN), we observe that its refinement process is inherently suitable for editing and offers a promising way to address these limitations. Motivated by this observation, we introduce RefineEdit, a training-free prompt-to-prompt image editing framework built on the GRN. Our key idea is to couple edit localization with content generation through the global refinement of binary image codes, allowing editing evidence to be revised as the image evolves. More specifically, RefineEdit combines bit routing with two stabilization mechanisms: adaptive spatial freezing and finite bit locking. Bit routing starts from an intermediate source state and uses signed probability differences between the two branches to identify editable positions and bits. It directs selected bits toward editing refinement while anchoring the rest to the evolving source trajectory. Adaptive spatial freezing limits unnecessary expansion of the editing region, while finite bit locking maintains recent bit activations to support continued editing. The overall framework requires no additional training, external masks, or attention control. Across nine editing categories of PIE-Bench, RefineEdit achieves the best background-preservation scores in PSNR, LPIPS, MSE, and SSIM, together with the highest whole-image and edited-region CLIP scores among the evaluated methods. Code is available at https://github.com/mura1n/RefineEdit.

Yulong Chen, Ziqian Zhang, Haoyu Zhang +4

City University of Hong Kong (Dongguan), Guangdong, China · City University of Hong Kong, Hong Kong, China · Mohamed bin Zayed University of Artificial Intelligence, Masdar, Abu Dhabi

Text-Guided Image EditingTraining-Free Image EditingImage Editing
AskPDF