Training-free image editing with diffusion models is highly desirable yet is complex and remains a significant challenge. While recent optimisation-based methods achieve strong zero-shot edits from text, they still struggle to preserve identity and capture intricate details, such as facial structure, surface texture, or object-specific geometry, that exist below the level of linguistic abstraction. To address this fundamental gap, we propose Concept Distillation Sampling (CDS). To the best of our knowledge, we are the first to introduce a unified, training-free framework for target-less, multi-concept image editing. CDS overcomes this linguistic bottleneck of previous methods by anchoring the editing process in the certainty of pretrained LoRA adapters. We integrate a highly stable distillation backbone (featuring ordered timesteps, regularisation, and negative-prompt guidance) with a novel dynamic weighting mechanism. This approach enables the composition and control of multiple visual concepts directly within the diffusion process, utilising spatially-aware priors from pretrained LoRA adapters without causing concept clashing. Our method preserves instance concept identity without requiring reference samples of the desired edit. Extensive quantitative and qualitative evaluations demonstrate that CDS establishes a new state-of-the-art over existing training-free editing and multi-LoRA composition methods on the InstructPix2Pix and ComposLoRA benchmarks. Project Page: https://nickyfot.github.io/cds/.
Figures & tables
Figure 1: Our method enables training-free, concept-based image editing with diffusion models. While recent state-of-the-art optimisation methods rely solely on text guidance and often lose instance identity during complex, multi-element changes, we unify a regularised optimisation objective with LoRA-driven concept composition. This enables multiple semantic or visual concepts ( e.g . facial structure, clothing, objects, or backgrounds captured in Dreambooth-style LoRAs) to be combined and controlled directly within the diffusion process. This approach preserves concept identity and fine-grained details that would be impossible to adequately describe in text alone, bridging the gap between text-based and visual concept-driven control.
Figure 2: Overview of our CDS image editing framework. We integrate multiple LoRA adapters into the image optimisation loop, enabling spatially-aware, concept-preserving edits across multiple concepts. The denoising trajectory follows ordered timesteps, allowing precise control over pose, style, and semantic attributes while maintaining overall composition.
Figure 3: Visualisation of timestep-ordered denoising. Comparison between the source image, ∇θLCDS at intermediate optimisation steps, and the final target image. Unlike DDS [ 8 ] , which samples timesteps uniformly at random, our method enforces a strict descending timestep order, enabling a coarse-to-fine denoising trajectory as is evident from the visualised gradients. Early steps capture high-frequency structural details such as edges, while later steps refine lower-frequency and stylistic components, resulting in a more coherent and stable edit.
Figure 4: Comparison of our optimisation objective vs baselines. Our method produces better results for the target subject with similarly perceived changes in the background.
CLIPScore ↑
LPIPS ↓
DiffusionClip ‡
0.251 ± 0.022
0.572 ± 0.0594
PnP ‡
0.221 ± 0.036
0.310 ± 0.075
InstructPix2Pix ‡
0.219 ± 0.037
0.322 ± 0.215
DDS ‡
0.225 ± 0.031
0.104 ± 0.061
PDS†
0.298 ± 0.044
0.096 ± 0.033
Ours
0.308 ± 0.042 *
0.100 ± 0.042
Table 1: Quantitative Evaluation of our optimisation method against previous SoTA, on InstructPix2Pix. Our method achieves statistically significant improvements in CLIPScore while maintaining comparable LPIPS; the CLIPScore gain reflects a genuine semantic edge under a deliberate semantic–fidelity trade-off.
Table 2: Quantitative Evaluation of Concept Distillation Sampling (CDS). Performance comparison across four LoRA composition strategies for different numbers of LoRAs on ComposLoRA testbed. Bold values indicate statistical significance at the 99% confidence interval.
Figure 5: Comparison of our CDS framework against previous LoRA-composition methods, evaluated using GPT-4V. Our method achieves higher perceived image and composition quality, with consistently higher pairwise win rates.
Figure 6: Examples of our method under different editing conditions. Our approach enables consistent and coherent edits across diverse scenarios, including pose changes, facial expression modifications, and element or object swaps ( e.g . , between different characters and environmental components). Each pair demonstrates the method’s ability to preserve subject and visual fidelity while adapting to the intended transformation.
Avg. Rank ↓
Win Rate ↑
CDS
1.90
38%
Compose
2.62
31%
Switch
2.49
25%
Merge
3.00
5%
Table 3: Human evaluation comparing our CDS framework to previous LoRA-composition methods. In line with GPT-4V assessments, human raters consistently preferred our method, which achieved the lowest average rank and most frequent first-place selections (highest win rate).
CLIPScore ↑
LPIPS ↓
DDS (200 steps)
0.225 ± 0.031
0.104 ± 0.061
DDS
0.305 ± 0.039
0.264 ± 0.061
Strict Timesteps (+DDS)
0.332 ± 0.032
0.461 ± 0.076
Regularisation (+DDS)
0.291 ± 0.041
0.081 ± 0.033
Negative Prompts (+DDS)
0.318 ± 0.040
0.319 ± 0.076
Full CDS
0.308 ± 0.042
0.100 ± 0.042
Table 4: CDS Core Objective Ablation. We establish a baseline with standard DDS (300 steps unless indicated) and iteratively introduce our objective components to isolate their effects. Our unified formulation achieves the optimal balance between executing semantic changes and maintaining structural integrity.
Figure 7: Samples generated from the base model using text descriptions. We observe distortions in the face, additional limbs, missing concepts, and inconsistent knowledge of characters and objects, all of which affect downstream tasks.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Sensitivity of the CDS objective to η . When the intended semantic change is spatially small, the method is highly robust to large η values.
η
CLIP ↑
LPIPS ↓
0.01
0.316±0.040
0.407±0.151
0.05
0.308±0.042
0.100±0.042
1
0.304±0.042
0.090±0.047
5
0.299±0.044
0.079±0.034
10
0.299±0.044
0.089±0.035
Appendix
Table 5: Sensitivity of the CDS objective to the η hyperparameter, evaluated on InstructPix2Pix.
N=2
N=3
Avg.
2×2
0.340±0.030
0.361±0.022
0.351
4×4
0.335±0.029
0.356±0.022
0.346
8×8
0.329±0.031
0.349±0.020
0.339
16×16
0.322±0.029
0.341±0.022
0.332
Appendix
Table 6: Effect of patch size on a subset of the ComposLoRA testbed. Although the method is relatively robust to patch size variations, a more localised approach (smaller patches) yields higher CLIPScores.
N=2
N=3
Avg.
τ=0.002
0.340±0.030
0.361±0.022
0.351
τ=0.01
0.337±0.030
0.360±0.022
0.349
τ=0.1
0.333±0.029
0.360±0.022
0.347
τ=1
0.329±0.029
0.359±0.022
0.344
Appendix
Table 7: Effect of τ on a subset of the ComposLoRA testbed. Lower temperatures positively improve CLIPScore by sharpening concept isolation.
CLIPScore ↑
LPIPS ↓
Merge
0.273
0.38
CDS (Ours)
0.304
0.21
Appendix
Table 8: SDXL results at N=2 on a ComposLoRA-style protocol. CDS retains its identity-preservation and semantic-alignment advantages on a stronger backbone.
Figure 9: Sample character + style change using SDXL. Merge collapses the target identity; CDS preserves source structure while introducing both concepts.
Figure 10: Examples of multi-concept edits on the anime subset of the ComposLoRA testbed for various N .
Figure 11: Examples of multi-concept edits on the reality subset of the ComposLoRA testbed for various N .
Figure 12: Comparison of our CDS framework against previous LoRA-composition methods, evaluated using GPT-4V. Our method achieves consistently higher pairwise win rates across both subsets.
Table 9: Comparison of Concept Distillation Sampling (CDS) against previous LoRA-composition methods, evaluated using GPT-4V. Our method consistently achieves the highest total Image and Composition Quality scores.
Category
LoRA Name
Trigger Words
Source
Anime Style Subset
Character
Kamado Nezuko
kamado nezuko, black hair, pink eyes, forehead
Link
Character
Texas the Omertosa
omertosa, 1girl, wolf ears, long hair
Link
Character
Son Goku
son goku, spiked hair, muscular male, wristband
Link
Clothing
Garreg Mach Uniform
gmuniform, blue thighhighs, long sleeves
Link
Clothing
Zero Suit (Metroid)
zero suit, blue gloves, high heels
Link
Appendix
Table 10: Detailed descriptions of each LoRA in the ComposLoRA [ 41 ] Testbed.
Figure 13: Human Instructions for the Multi-Concept Editing task.
While Diffusion Large Language Models (DLLMs) have demonstrated remarkable capabilities in multi-modal generation, performing precise, training-free image editing remains an open challenge. Unlike continuous diffusion models, the discrete tokenization inherent in DLLMs hinders the application of standard noise inversion techniques, often leading to structural degradation during editing. In this paper, we introduce GIDE (Grounded Inversion for DLLM Image Editing), a unified framework designed to bridge this gap. GIDE incorporates a novel Discrete Noise Inversion mechanism that accurately captures latent noise patterns within the discrete token space, ensuring high-fidelity reconstruction. We then decompose the editing pipeline into grounding, inversion, and refinement stages. This design enables GIDE supporting various editing instructions (text, point and box) and operations while strictly preserving the unedited background. Furthermore, to overcome the limitations of existing single-step evaluation protocols, we introduce GIDE-Bench, a rigorous benchmark comprising 805 compositional editing scenarios guided by diverse multi-modal inputs. Extensive experiments on GIDE-Bench demonstrate that GIDE significantly outperforms prior training-free methods, improving Semantic Correctness by 51.83% and Perceptual Quality by 50.39%. Additional evaluations on ImgEdit-Bench confirm its broad applicability, demonstrating consistent gains over trained baselines and yielding photorealistic consistency on par with leading models.
Zifeng Zhu, Jiaming Han, Jiaxiang Zhao +2
Xi’an Jiaotong University · MMLab, The Chinese University of Hong Kong
In image editing, it is essential to incorporate a context image to convey the user's precise requirements, such as subject appearance or image style. Existing training-based visual context-aware editing methods incur data collection effort and training cost. On the other hand, the training-free alternatives are typically established on diffusion inversion, which struggles with consistency and flexibility. In this work, we propose VicoEdit, a training-free and inversion-free method to inject the visual context into the pretrained text-prompted editing model. More specifically, VicoEdit directly transforms the source image into the target one based on the visual context, thereby eliminating the need for inversion that can lead to deviated trajectories. Moreover, we design a posterior sampling approach guided by concept alignment to enhance the editing consistency. Empirical results demonstrate that our training-free method achieves even better editing performance than the state-of-the-art training-based models.
Rui Song, Guo-Hua Wang, Qing-Guo Chen +6
HKUST · Alibaba International Digital Commerce Group · Institute for AI Industry Research (AIR), Tsinghua University +1
Existing image editing frameworks predominantly follow the training paradigm of text-to-image diffusion models. However, extending this paradigm to image editing highlights two inherent discrepancies, specifically, the insufficient attention to edit concept granularity and the training inefficiency caused by sparse supervision signals. To address these issues, we establish a comprehensive hierarchical taxonomy featuring over 1,000 fine-grained edit concepts and build ConceptEdit-12M, a massive dataset of 12 million high-quality editing pairs via an improved synthesis framework. This library-driven approach effectively rectifies the distribution collapse of generated data while ensuring high data fidelity. Furthermore, we propose a dense supervision training strategy that synthesizes multiple non-interfering concepts into single image pairs. By providing richer learning signals, this strategy significantly enhances both training efficiency and overall model performance. Training results validate our strategy, significantly outperforming prior works. Finally, we present ConceptEdit-Bench, a granular evaluation suite designed to diagnose model capabilities across a vast array of real-world scenarios.