Latent Diffusion Models

Follow this topic Ask about this topic

Latent diffusion models first encode data into a compact space, often with a variational autoencoder, then learn to generate it by reversing a noise process. Researchers use U-Nets and diffusion transformers, with text, images, masks, or physical variables as conditions, for tasks ranging from ImageNet image generation to medical imaging, audio, 3D shapes, and scientific reconstruction. Current work explores better latent spaces, faster sampling, and controls for detail, structure, and uncertainty. A central tension is that stronger compression can reduce compute but lose fine detail or physical fidelity, while good reconstruction alone does not guarantee that a latent space is easy to model.

Latest papers

207 papers19 papers in the last four weeks, up 36% on the four weeks before. 0.1% of all new papers.

Coevolving Representations in Joint Image-Feature Diffusion
Apr 19, 2026Theodoros Kouzelis, Spyros Gidaris, Nikos KomodakisRepresentation Learning Latent Diffusion Models
Archimedes, Athena RC · National Technical University of Athens · University of Crete +1
CLIMB: Controllable Longitudinal Brain Image Generation using Mamba-based Latent Diffusion Model and Gaussian-aligned Autoencoder
Apr 17, 2026Duy-Phuong Dao, Muhammad Taqiyuddin, Jahae Kim +4Latent Diffusion Models Longitudinal Medical Image Analysis
Department of Artificial Intelligence Convergence, Chonnam National University, South Korea · Department of Nuclear Medicine, Chonnam National University, South Korea · University of South Australia, Australia +2
Monthly Diffusion v0.9: A Latent Diffusion Model for the First AI-MIP
Apr 15, 2026Kyle J. C. Hall, Maria J. MolinaClimate Emulation Climate Modeling
Department of Atmospheric & Oceanic Sciences, University of Maryland, College Park, MD 20740
Domain-Specific Latent Representations Improve the Fidelity of Diffusion-Based Medical Image Super-Resolution
Apr 14, 2026Sebastian Cajas, Ashaba Judith, Rahul Gorijavolu +8Medical Image Super-Resolution Diffusion Models for Image Restoration
MIT Critical Data, Massachusetts Institute of Technology, Cambridge, MA, USA. · Department of Biomedical Engineering, Mbarara University of Science and Technology, Uganda. · School of Medicine, Johns Hopkins University, Baltimore, MD, USA. +10
Latent-Compressed Variational Autoencoder for Video Diffusion Models
Apr 12, 2026Jiarui Guan, Wenshuai Zhao, Zhengtao Zou +2Video Diffusion Models Variational Autoencoders
Aalto University · ELLIS Institute Finland · University of Oulu
FlowGuard: Towards Lightweight In-Generation Safety Detection for Diffusion Models via Linear Latent Decoding
Apr 9, 2026Jinghan Yang, Yihe Fan, Xudong Pan +1Diffusion Models Latent Diffusion Models
Fudan University, Shanghai, China · Fudan University, Shanghai Innovation Institute, Shanghai, China
Geo-EVS: Geometry-Conditioned Extrapolative View Synthesis for Autonomous Driving
Apr 8, 2026Yatong Lan, Rongkui Tang, Lei HeNovel View Synthesis Autonomous Driving
School of Vehicle and Mobility, Tsinghua University, Beijing, China · State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China · School of Science, Minzu University, Beijing, China +1
PhaseFlow4D: Physically Constrained 4D Beam Reconstruction via Feedback-Guided Latent Diffusion
Apr 4, 2026Alexander Scheinker, Alexander Plastun, Peter Ostroumov4D Reconstruction Physics-Informed Diffusion Models
Los Alamos National Laboratory Los Alamos 87545, NM, USA · Facility for Rare Isotopes Beams Michigan State University, East Lansing, MI 48824, USA
SD-FSMIS: Adapting Stable Diffusion for Few-Shot Medical Image Segmentation
Apr 3, 2026Meihua Li, Yang Zhang, Weizhao He +2Diffusion Models for Medical Image Segmentation Few-Shot Medical Image Segmentation
Computer Vision Institute, College of Computer Science and Software Engineering, Shenzhen University
Understanding Latent Diffusability via Fisher Geometry
Apr 3, 2026Jing Gu, Morteza Mardani, Wonjun Lee +2Information Geometry Neural Representation Geometry
School of Mathematics, University of Minnesota · NVIDIA · Department of Mathematics, The Ohio State University +1
Attention Frequency Modulation: Training-Free Spectral Modulation of Diffusion Cross-Attention
Mar 30, 2026Seunghun Oh, Unsang ParkAttention Control in Diffusion Models Cross-Attention
Sogang University, Seoul, Republic of Korea
Cone-Beam CT Image Quality Enhancement Using A Latent Diffusion Model Trained with Simulated CBCT Artifacts
Mar 27, 2026Naruki Murahashi, Mitsuhiro Nakamura, Megumi NakaoCone-Beam CT Diffusion Models for Image Restoration
Graduate School of Medicine, Kyoto University, Kyoto, 606-8507, JAPAN
CardioDiT: Latent Diffusion Transformers for 4D Cardiac MRI Synthesis
Mar 26, 2026Marvin Seyfarth, Sarah Kaye Müller, Arman Ghanaat +7Medical Image Synthesis Latent Diffusion Models
Institute for Artificial Intelligence in Cardiovascular Medicine, Medical Faculty of Heidelberg University, Heidelberg University, Germany · Department of Cardiology, Angiology, Pneumology, Heidelberg University Hospital, Heidelberg, Germany · DZHK (German Centre for Cardiovascular Research), Partner Site Heidelberg/Mannheim, Heidelberg, Germany +4
End-to-End Training for Unified Tokenization and Latent Denoising
Mar 23, 2026Shivam Duggal, Xingjian Bai, Zongze Wu +5Latent Diffusion Models Image Generation
Massachusetts Institute of Technology · Adobe
Repurposing Geometric Foundation Models for Multi-view Diffusion
Mar 23, 2026Wooseok Jang, Seonghu Jeon, Jisang Han +5Multi-View Diffusion Models Novel View Synthesis
KAIST AI · New York University · Intel Labs
DA-VAE: Plug-in Latent Compression for Diffusion via Detail Alignment
Mar 23, 2026Xin Cai, Zhiyuan You, Zhoutong Zhang +1Diffusion-Based Image Generation Diffusion Model Compression
Multimedia Laboratory, The Chinese University of Hong Kong · Shanghai AI Laboratory · Adobe NextCam +1
Taming Sampling Perturbations with Variance Expansion Loss for Latent Diffusion Models
Mar 22, 2026Qifan Li, Xingyu Zhou, Jinhua Zhang +2Latent-Space Regularization Latent Diffusion Models
University of Electronic Science and Technology of China
TAFG-MAN: Timestep-Adaptive Frequency-Gated Latent Diffusion for Efficient and High-Quality Low-Dose CT Image Denoising
Mar 21, 2026Tangtangfang Fang, Yang Jiao, Xiangjian He +2LDCT Denoising Diffusion Models for Image Restoration
University of Nottingham Ningbo China, Ningbo, Zhejiang, China
RPiAE: A Representation-Pivoted Autoencoder Enhancing Both Image Generation and Editing
Mar 19, 2026Yue Gong, Hongyu Li, Shanyuan Liu +8T2I Diffusion Models Diffusion-Based Image Generation
Beihang University · AI Research · The Chinese University of Hong Kong
LGESynthNet: Controlled Scar Synthesis for Improved Scar Segmentation in Cardiac LGE-MRI Imaging
Mar 18, 2026Athira J. Jacob, Puneet Sharma, Daniel RueckertMedical Image Synthesis Synthetic Data Augmentation
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · Al in Healthcare and Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany · Department of Computing, Imperial College London, UK
LaDe: Unified Multi-Layered Graphic Media Generation and Decomposition
Mar 18, 2026Vlad-Constantin Lungu-Stan, Ionut Mironica, Mariana-Iuliana GeorgescuImage Generation Latent Diffusion Models
Adobe Research, Romania
CytoSyn: a Foundation Diffusion Model for Histopathology -- Tech Report
Mar 18, 2026Thomas Duboudin, Xavier Fontaine, Etienne Andrier +7Medical Image Synthesis Pathology Foundation Models
Owkin, Inc
Semi-supervised Latent Disentangled Diffusion Model for Textile Pattern Generation
Mar 17, 2026Chenggong Hu, Yi Wang, Mengqi Xue +3Image Generation Latent Diffusion Models
Zhejiang University · Hangzhou City University
$D^3$ -RSMDE: 40 $\times$ Faster and High-Fidelity Remote Sensing Monocular Depth Estimation
Mar 17, 2026Ruizhi Wang, Weihan Li, Zunlei Feng +5Monocular Depth Estimation Diffusion Model Inference Acceleration
School of Software Technology, Zhejiang University · State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security +2
Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusion
Mar 15, 2026Mang Ning, Mingxiao Li, Le Zhang +4Latent Diffusion Models Diffusion-Based Image Generation
Utrecht University · KU Leuven · Mila +1
Multiscale Structure-Guided Latent Diffusion for Multimodal MRI Translation
Mar 13, 2026Jianqiang Lin, Zhiqiang Shen, Peng Cao +3Medical Image Synthesis Latent Diffusion Models
Northeastern University, Shenyang, China · Key Laboratory of Intelligent Computing in Medical Image, Shenyang, China · Computer Science and Engineering, Northeastern University, Shenyang, China +5
SNPgen: Phenotype-Supervised Genotype Representation and Synthetic Data Generation via Latent Diffusion
Mar 11, 2026Andrea Lampis, Michela Carlotta Massi, Nicola Pirastu +3Synthetic Data Generation Genomics
DEIB, Politecnico di Milano, Milan 20133, Italy · Health Data Science Centre, Human Technopole, Milan 20157, Italy · Genomics Research Centre, Human Technopole, Milan 20157, Italy +2
Co-Diffusion: An Affinity-Aware Two-Stage Latent Diffusion Framework for Generalizable Drug-Target Affinity Prediction
Mar 11, 2026Yining Qian, Pengjie Wang, Yixiao Li +4Drug Discovery Latent Diffusion Models
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China · College of Information Science and Engineering, Northeastern University, Shenyang 110819, China · Westlake University, Hangzhou 310000, China +2
Geometric Autoencoder for Diffusion Models
Mar 11, 2026Hangyu Liu, Jianyong Wang, Yutao SunLatent Diffusion Models Diffusion Models
Shanghai Innovation Institute · Tsinghua University
High-Fidelity Medical Shape Generation via Skeletal Latent Diffusion
Mar 8, 2026Guoqing Zhang, Jingyun Yang, Siqi Chen +23D Shape Generation Latent Diffusion Models
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China · School of AI, Chinese University of Hong Kong (Shenzhen)

Apr 19, 2026J/K move · Enter open · S save

Coevolving Representations in Joint Image-Feature Diffusion

Theodoros Kouzelis, Spyros Gidaris, Nikos Komodakis

Archimedes, Athena RC · National Technical University of Athens · University of Crete+1

Representation Learning Latent Diffusion Models Diffusion-Based Image Generation

Open paper AskPDF arXiv

Joint image-feature generative modeling has recently emerged as an effective strategy for improving diffusion training by coupling low-level VAE latents with high-level semantic features extracted from pre-trained visual encoders. However, existing approaches rely on a fixed representation space, constructed independently of the generative objective and kept unchanged during training. We argue that the representation space guiding diffusion should itself adapt to the generative task. To this end, we propose Coevolving Representation Diffusion (CoReDi), a framework in which the semantic representation space evolves during training by learning a lightweight linear projection jointly with the diffusion model. While naively optimizing this projection leads to degenerate solutions, we show that stable coevolution can be achieved through a combination of stop-gradient targets, normalization, and targeted regularization that prevents feature collapse. This formulation enables the semantic space to progressively specialize to the needs of image synthesis, improving its complementarity with image latents. We apply CoReDi to both VAE latent diffusion and pixel-space diffusion, demonstrating that adaptive semantic representations improve generative modeling across both settings. Experiments show that CoReDi achieves faster convergence and higher sample quality compared to joint diffusion models operating in fixed representation spaces.