GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editing

Zifeng Zhu; Jiaming Han; Jiaxiang Zhao; Minnan Luo; Xiangyu Yue

cs.CVMar 22, 2026

GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editing

Authors: Zifeng Zhu, Jiaming Han, Jiaxiang Zhao, Minnan Luo, Xiangyu Yue

Organizations: Xi’an Jiaotong University · MMLab, The Chinese University of Hong Kong

Abstract

While Diffusion Large Language Models (DLLMs) have demonstrated remarkable capabilities in multi-modal generation, performing precise, training-free image editing remains an open challenge. Unlike continuous diffusion models, the discrete tokenization inherent in DLLMs hinders the application of standard noise inversion techniques, often leading to structural degradation during editing. In this paper, we introduce GIDE (Grounded Inversion for DLLM Image Editing), a unified framework designed to bridge this gap. GIDE incorporates a novel Discrete Noise Inversion mechanism that accurately captures latent noise patterns within the discrete token space, ensuring high-fidelity reconstruction. We then decompose the editing pipeline into grounding, inversion, and refinement stages. This design enables GIDE supporting various editing instructions (text, point and box) and operations while strictly preserving the unedited background. Furthermore, to overcome the limitations of existing single-step evaluation protocols, we introduce GIDE-Bench, a rigorous benchmark comprising 805 compositional editing scenarios guided by diverse multi-modal inputs. Extensive experiments on GIDE-Bench demonstrate that GIDE significantly outperforms prior training-free methods, improving Semantic Correctness by 51.83% and Perceptual Quality by 50.39%. Additional evaluations on ImgEdit-Bench confirm its broad applicability, demonstrating consistent gains over trained baselines and yielding photorealistic consistency on par with leading models.

Figures & tables

Figure 1: Demonstration of our proposed GIDE framework . Given a source image (left of each pair) and an editing instruction guided by flexible spatial modalities (points, bounding boxes, or pure text), GIDE enables precise and localized image editing. The edited results (right of each pair) demonstrate high-quality visual synthesis and strict background preservation, all achieved in a completely training-free manner.

Figure 2: The overall architecture of GIDE . It decouples the editing process into three sequential stages: Grounding locates the target region via a segmentation foundation model; Inversion executes the edit by reconstructing the discrete latent space; and Refinement enhances the visual coherence and fidelity of the final output.

Figure 3: Statistical overview of GIDE-Bench . It contains 805 cases and 1,610 sub-instructions (left), stratified across three input modalities (middle). The right chart shows the distribution of compositional editing pairs (Rem.: Remove, Rep.: Replace).

Method	Non-edit			$\textbf{Edit}_{\mathrm{GPT}}$		$\textbf{Edit}_{\mathrm{Gemini}}$
Method	MSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	SC $\uparrow$	PQ $\uparrow$	SC $\uparrow$	PQ $\uparrow$
End-to-End Image Editing Models
OneDiffusion [ 26 ]	1247.74	20.61	0.6831	1.81	1.79	1.94	1.81
InstructPix2Pix [ 2 ]	2540.06	17.71	0.7108	1.89	1.86	1.87	1.68
MagicBrush [ 72 ]	2410.44	19.36	0.7210	2.13	2.04	2.08	1.74
Lumina-DiMOO [ 66 ]	1208.22	19.80	0.6461	3.10	2.92	3.10	2.34

Table 1: Performance comparison on GIDE-Bench . Best results within each group are highlighted in bold . Gray indicates our method. Notably, our GIDE integrated with Lumina-DiMOO achieves state-of-the-art performance among training-free methods, steadily narrowing the performance gap to top-tier fully supervised models.

Figure 4: Qualitative comparison on GIDE-Bench . GIDE accurately follows instructions and preserves structural fidelity, avoiding unintended alterations (e.g., DICE) and aspect ratio distortions (e.g., GPT-Image-1) while achieving photorealistic consistency. Zoom in for better view.

Method	Rep. $\uparrow$	Add $\uparrow$	Rem. $\uparrow$
MagicBrush [ 72 ]	1.97	2.84	1.58
AnyEdit [ 71 ]	2.47	3.18	2.23
UltraEdit [ 74 ]	2.96	3.44	1.45
ICEdit [ 73 ]	3.15	3.58	2.93
Step1X-Edit [ 32 ]	3.40	3.88	2.41

Table 2: Evaluation on ImgEdit [ 70 ] benchmark . Rep.: Replace, Rem.: Remove.

Figure 5: Visual ablation on the proposed components . Removing the discrete inversion degrades structural fidelity and distorts object shapes. Omitting the spatial grounding leads to the loss of background context. Disabling the visual refinement blurs local details and compromises the seamless preservation of the unedited regions.

Method	Non-edit			$\textbf{Edit}_{\mathrm{GPT}}$		$\textbf{Edit}_{\mathrm{Gemini}}$
Method	MSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	SC $\uparrow$	PQ $\uparrow$	SC $\uparrow$	PQ $\uparrow$
Ours (Full)	1224.89	20.40	0.7083	4.47	3.98	4.26	3.78
w/o discrete inversion	2196.29 79%↑	17.85 13%↓	0.6628 6%↓	4.30 4%↓	3.83 4%↓	4.08 4%↓	3.28 13%↓
w/o spatial grounding	8446.17 590%↑	9.42 54%↓	0.4161 41%↓	3.60 19%↓	3.43 14%↓	2.68 37%↓	2.21 42%↓
w/o visual refinement	2476.78 102%↑	17.23 16%↓	0.6444 9%↓	3.73 17%↓	3.25 18%↓	3.56 16%↓	2.67 29%↓
w/o intrinsic refinement	1424.29 16%↑	19.80 3%↓	0.6987 1%↓	3.96 11%↓	3.51 12%↓	3.87 9%↓	3.11 18%↓

Table 3: Ablation experiments . We compare the full method against variants with specific modules removed. The results confirm that the full framework achieves the best performance in both background preservation and editing quality.

$\lambda$	Non-edit			$\textbf{Edit}_{GPT}$		$\textbf{Edit}_{Gemini}$
$\lambda$	MSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	SC $\uparrow$	PQ $\uparrow$	SC $\uparrow$	PQ $\uparrow$
0.0	1224.62	20.35	0.7086	4.30	3.86	4.12	3.71
0.1	1306.47	20.22	0.7066	4.26	3.82	4.12	3.69
0.2	1224.89	20.40	0.7083	4.47	3.98	4.26	3.78
0.3	1308.57	20.33	0.7075	4.28	3.85	4.09	3.70
0.4	1225.40	20.51	0.7084	4.21	3.83	4.03	3.75

Table 4: Sensitivity analysis of the mixing coefficient

\lambda

. Best scores are highlighted in bold . The default setting

\lambda=0.2

is highlighted in gray. GIDE achieves optimal editing quality at

\lambda=0.2

with stable performance across the range.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: An example of Mask Relaxation. Expanding the tight source mask $\mathbf{M}_{\text{src}}$ to its bounding box accommodates noticeable shape variations (e.g., bicycle to bench), ensuring complete object generation while preserving the unedited background.

Figure 7: An example of VQModel reconstruction characteristics. GIDE executes semantic edits accurately, but the VQModel’s decompression introduces minor color variations in high-frequency details like the eyes. Zoom in for better view.

Method	Generation Setup
GIDE +Lumina-DiMOO	cfg_scale = 4.0, timesteps = 64, temperature = 1.0
GIDE +MMaDA	cfg_scale = 4.0, timesteps = 64, temperature = 1.0
DICE+Lumina-DiMOO	cfg_scale = 4.0, timesteps = 64, temperature = 1.0
DirectInversion+PnP	guidance_scale = 7.5, num_steps = 50
DirectInversion+P2P	guidance_scale = 7.5, cross_replace_steps = 0.4, self_replace_steps = 0.6
Edit-R1	true_cfg_scale = 4.0, num_inference_steps = 40

Appendix

Table 5: Generating parameters for training-free and training-based methods.

Training-Free Image Editing

See all 19 papers

Training-Free Multi-Concept Image Editing
Feb 24, 2026Niki Foteinopoulou, Ignas Budvytis, Stephan LiwickiTraining-Free Image Editing Image Editing
Cambridge Research Laboratory, Toshiba Europe, Cambridge, UK · Independent Researcher, UK
Masked Generative Transformer Is What You Need for Image Editing
May 11, 2026Wei Chow, Linfeng Li, Xian Sun +14Transformer Text-Guided Image Editing
1ByteDance · 2National University of Singapore · 3Duke University +2
Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network
Sep 17, 2026Yulong Chen, Ziqian Zhang, Haoyu Zhang +4Text-Guided Image Editing Training-Free Image Editing
City University of Hong Kong (Dongguan), Guangdong, China · City University of Hong Kong, Hong Kong, China · Mohamed bin Zayed University of Artificial Intelligence, Masdar, Abu Dhabi

Feb 24, 2026J/K move · Enter open · S save

Training-Free Multi-Concept Image Editing

Niki Foteinopoulou, Ignas Budvytis, Stephan Liwicki

Cambridge Research Laboratory, Toshiba Europe, Cambridge, UK · Independent Researcher, UK

Training-Free Image Editing Image Editing Diffusion-Based Image Editing Diffusion Model Guidance Compositional Image Generation

Open paper AskPDF arXiv

Training-free image editing with diffusion models is highly desirable yet is complex and remains a significant challenge. While recent optimisation-based methods achieve strong zero-shot edits from text, they still struggle to preserve identity and capture intricate details, such as facial structure, surface texture, or object-specific geometry, that exist below the level of linguistic abstraction. To address this fundamental gap, we propose Concept Distillation Sampling (CDS). To the best of our knowledge, we are the first to introduce a unified, training-free framework for target-less, multi-concept image editing. CDS overcomes this linguistic bottleneck of previous methods by anchoring the editing process in the certainty of pretrained LoRA adapters. We integrate a highly stable distillation backbone (featuring ordered timesteps, regularisation, and negative-prompt guidance) with a novel dynamic weighting mechanism. This approach enables the composition and control of multiple visual concepts directly within the diffusion process, utilising spatially-aware priors from pretrained LoRA adapters without causing concept clashing. Our method preserves instance concept identity without requiring reference samples of the desired edit. Extensive quantitative and qualitative evaluations demonstrate that CDS establishes a new state-of-the-art over existing training-free editing and multi-LoRA composition methods on the InstructPix2Pix and ComposLoRA benchmarks. Project Page: https://nickyfot.github.io/cds/.

Abstract

Training-Free Image Editing

Explore similar work

Training-Free Multi-Concept Image Editing

Masked Generative Transformer Is What You Need for Image Editing

Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network