Organizations: Fudan University, Shanghai 200433, China. · School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, Shaanxi, China. · The University of Hong Kong, Hong Kong, China.
Diffusion models and flow-based models have recently become the dominant paradigms in generative modeling, largely due to their ability to learn rich, multi-level visual representations through large-scale training. This creates a bidirectional relationship between generative models and representation learning: improving representation learning enhances generation quality, while the learned representations can be leveraged for broader understanding tasks. This survey systematically explores this interplay with a focus on applications. We propose a three-tier progressive framework that organizes existing works from three perspectives: using representation learning to improve generative capabilities, exploiting generative models to extract representations for perception tasks, and ultimately moving toward general-purpose unified applications. We systematically categorize representative methods across a wide range of downstream tasks, including image classification, dense visual prediction, instance-level perception, and annotation-scarce scenarios. By providing a unified taxonomy and identifying key challenges, this survey aims to clarify the underlying logic of current research and suggest promising directions for future exploration. We hope this work can serve as a valuable reference for researchers interested in harnessing the representation power of generative models for applications beyond generation.
Figures & tables
Taxonomy Perspective
Category
Editing Type
Representative References
Visual target perspective
Content-level
Object editing
Paint-by-Inpaint Wasserman et al. (2025) , OmniPaint Yu et al. (2025c) , InsertAnything Song et al. (2026b) , PS-Diffusion Wang et al. (2025e) , EraDiff Liu et al. (2025b) , ObjectClear Zhao et al. (2026a) , YOEO Zhu et al. (2026d)
Scene-context editing
AnyEdit Yu et al. (2025a) , Step1X-Edit Liu et al. (2025a) , InsightEdit Xu et al. (2025b)
Expression-level
Structure editing
DragGAN Pan et al. (2023a) , DragDiffusion Shi et al. (2024) , GeoDrag Pu et al. (2026) , DragFlow Zhou et al. (2026)
Style editing
SCAdapter Trinh (2026)
Lighting editing
LightLab Magar et al. (2025) , DreamLight Liu et al. (2025c) , IC-Light Zhang et al. (2025e)
Texture editing
TextureDiffusion Su et al. (2025)
Table 1: Taxonomy of image editing methods.
Task
Role of Representation
Main Representation Levels
Representative References
Watermarking
A carrier whose identity signal must propagate to the output and remain recoverable.
Output/decoder, initial noise or latent, final latent, and inversion trajectory.
Stable Signature Fernandez et al. (2023) ; Tree-Ring Wen et al. (2023) ; Gaussian Shading Yang et al. (2024b)
Backdoor attacks
A representation in which a trigger, semantic association or synthetic-data signal is injected or modified.
Data poisoning, model training or fine-tuning, triggered inference, and downstream synthetic-data reuse.
UIBDiffusion Han et al. (2024) ; BadDiffusion Chou et al. (2023) ; Data-Chain Backdoor Lu et al. (2025)
Adversarial protection
A target representation used by personalization, editing, or student-model training.
Semantic/identity features, encoder latents, score or velocity objectives, attention/trajectory states, and released outputs.
AdvDM Liang et al. (2023) ; Anti-DreamBooth Van Le et al. (2023) ; EditShield Chen et al. (2024)
Table 2: Task-specific roles of representations in trustworthy generation. Representative references are illustrative rather than exhaustive.