Diffusion Models

Follow this topic Ask about this topic

Diffusion models learn a data distribution by gradually corrupting examples with noise and training a denoiser to reverse that process. Major approaches include DDPMs and score-based SDEs, latent diffusion and diffusion Transformers, flow and consistency methods, and discrete masked or continuous-time Markov diffusion; conditioning and guidance adapt them to specific tasks. They are used for images, video, audio, language, molecules, and scientific forecasting, with evaluations ranging from ImageNet FID to language perplexity and domain-specific measures of accuracy and uncertainty. A central tension is how to make sampling faster and generation more controllable without sacrificing quality, calibration, or diversity, while theory and ongoing audits address generalization, memorization, and privacy.

Latest papers

1,192 papers152 papers in the last four weeks, up 77% on the four weeks before. 0.9% of all new papers.

Improved Baselines with Representation Autoencoders
May 18, 2026Jaskirat Singh, Boyang Zheng, Zongze Wu +3Diffusion Model Guidance Diffusion Transformer
Adobe Research · ANU · New York University
Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster
May 18, 2026Grigory Bartosh, Teodora Pandeva, Sushrut Karmalkar +1Few-Step Diffusion Sampling Diffusion Model Acceleration
University of Amsterdam · Microsoft Research, Cambridge
Whispers in the Noise: Surrogate-Guided Concept Awakening via a Multi-Agent Framework
May 18, 2026Mengyu Sun, Ziyuan Yang, Zunlong Zhou +3Adversarial Attacks Adversarial Attacks on Diffusion Models
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University · Lee Kong Chian School of Medicine, Nanyang Technological University · School of Cyber Science and Engineering, Sichuan University
SENSE: Satellite-based ENergy Synthesis for Sustainable Environment
May 18, 2026Kailai Sun, Mingyi He, Heye Huang +5Conditional Image Generation Synthetic Data Generation
SMART, Singapore · Massachusetts Institute of Technology Cambridge, MA, USA · University of Florida Gainesville, Florida, USA
Wasserstein bounds for denoising diffusion probabilistic models via the Föllmer process
May 18, 2026Yuta KoikeDiffusion Sampling Diffusion Models
*Graduate School of Mathematical Sciences, University of Tokyo · †CREST, Japan Science and Technology Agency
FLAG: Foundation model representation with Latent diffusion Alignment via Graph for spatial gene expression prediction
May 18, 2026Qi Si, Penglei Wang, Yushuai Wu +5Spatial Transcriptomics Spatial Gene Expression Prediction
Shanghai Academy of Artificial Intelligence for Science, Shanghai, China. · School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China. · Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, China.
Training data attribution in diffusion models via mirrored unlearning and noise-consistent skew
May 18, 2026Joan Serrà, Dipam Goswami, Fabio Morreale +2Gradient-Based Attribution Training Data Attribution
Sony AI · Sony Group Corporation
AdaptiveLoad: Towards Efficient Video Diffusion Transformer Training
May 18, 2026Yucheng Guo, Yongjian Guo, Zhong Guan +6Video Diffusion Models GPU Kernel Optimization
JD AI Infra, Beijing, China · Tsinghua University, Beijing, China
DAD4TS: Data-Augmentation-Oriented Diffusion Model for Time-Series Forecasting with Small-Scale Data
May 18, 2026Masahiro Suzuki, Bohui Xia, Hiroto Yamamoto +1Time Series Forecasting Time Series Data Augmentation
Sony Group Corporation Tokyo, Japan
FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion
May 18, 2026Lichen Ma, Zipeng Guo, Yu He +5Pixel-Space Diffusion Models Diffusion-Based Image Generation
PFlow-T: A Persistence-Driven Forward Process for Topology-Controlled Generation
May 17, 2026Snigdha Chandan KhilarGenerative Modeling Persistent Homology
Independent Researcher
Variational Diffusion Channel Decoder
May 17, 2026Chengwei Zhang, Yifan Du, Siyu LiaoNeural Decoding Error-Correcting Codes
The School of Integrated Circuits Sun Yat-sen University Shenzhen, China
Noise scheduling and linear dynamics in diffusion models on Lie groups
May 17, 2026Javad KomijaniLie Group Methods High-Energy Physics
Institute for Theoretical Physics, ETH Zurich, 8093 Zurich, Switzerland
Vocabulary-size-independent Convergence of Discrete Diffusion Models: adjoint equations induce the right space
May 17, 2026Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3Integral Probability Metrics Diffusion Models
Department of Mathematics Oden Institute UCLA · Department of Mathematics University of Texas at Austin · Rutgers University +3
VoxScene: Anchor-Conditioned Voxel Diffusion for Indoor Scene Arrangement
May 16, 2026Haotian Mao, Yuhan Huang, Jiatao Lin +83D Scene Generation 3D Scene Representation
Shanghai Jiao Tong University, China · Hong Kong University of Science and Technology, China · Tsinghua University, China +2
Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine
May 16, 2026Wei Huang, Andi Han, Mingyuan Bai +4Latent Diffusion Models Manifold Learning
RIKEN AIP & The Institute of Statistical Mathematics · University of Sydney · Agency for Science, Technology and Research & The Institute of Statistical Mathematics +4
DreamEdit3D: Personalization of Multi-View Diffusion Models for 3D Editing
May 16, 2026Jinxin Ai, Matthias Nießner, Ziya ErkoçMulti-View Diffusion Models Diffusion Models
Technical University of Munich
Towards Generalized Image Manipulation Localization via Score-based Model
May 16, 2026Yunfei Wang, Bo Du, Zhe Yang +4Image Forgery Detection Image Forgery Localization
Sichuan University, Chengdu, China
HAD: Hallucination-Aware Diffusion Priors for 3D Reconstruction
May 16, 2026Xi Liu, Weiwei Sun, Zhou Ren +33D Reconstruction Novel View Synthesis
1Amazon AWS · 2Clemson University
Prefix-Adaptive Block Diffusion for Efficient Document Recognition
May 16, 2026Mingxu Chai, Ziyu Shen, Chenyu Liu +9Diffusion Language Model Inference Diffusion Model Caching
Computation and Artificial Intelligence Innovative College, Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · ByteDance, Shanghai, China
Constrained Code Generation with Discrete Diffusion
May 16, 2026Lize Shao, Michael Cardei, Zichen Xie +2Constrained Generative Modeling Constrained Decoding
University of Virginia
Prediction Is Not Physics: Learning and Evaluating Conserved Quantities in Neural Simulators
May 16, 2026Andrew Bukowski, Aditya Kothari, Simba Shi +1Neural Surrogate Modeling Dynamical Systems
Yale University
StAD: Stein Amortized Divergence for Fast Likelihoods with Diffusion and Flow
May 15, 2026Gurjeet Jagwani, Stephen Thorp, Sinan Deger +1Continuous Normalizing Flows Flow-Based Generative Modeling
Institute of Astronomy and Kavli Institute for Cosmology, University of Cambridge, Cambridge, UK · Research Computing Services, University of Cambridge, Cambridge, UK · Cavendish Laboratory, Department of Physics, University of Cambridge, Cambridge, UK +1
Registers Matter for Pixel-Space Diffusion Transformers
May 15, 2026Nikita Starodubcev, Ilia Sudakov, Ilya Drobyshevskiy +2Diffusion Transformer Diffusion Guidance
Yandex Research
RaPD: Resolution-Agnostic Pixel Diffusion via Semantics-Enriched Implicit Representations
May 15, 2026Yanhao Ge, Shanyan Guan, Weihao Wang +2Implicit Neural Representations Neural Rendering
College of Electronic and Information Engineering, Tongji University · vivo Mobile Communication Co., Ltd. · Nanjing University.
Unsupervised 3D Human Pose Estimation via Conditional Multi-view Ancestral Sampling
May 15, 2026Ryohei Goto, Takuya Fujihashi, Shunsuke Saruwatari +1Diffusion Sampling 3D Human Pose Estimation
The University of Osaka
SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer
May 14, 2026Zheng Hui, Yunlong BaiDiffusion Transformer Video Editing
Baidu Inc., Beijing, China.
Towards Continuous Sign Language Conversation from Isolated Signs
May 14, 2026Youngmin Kim, Kyobin Choo, Jiwoo Park +4Sign Language Translation Human Motion Generation
1Yonsei University · 2LG Electronics · 3Emory University
VMU-Diff: A Coarse-to-fine Multi-source Data Fusion Framework for Precipitation Nowcasting
May 14, 2026Chunlei Shi, Hao Li, Yufeng Zhu +6Precipitation Forecasting Probabilistic Forecasting
Department of Automation, Southeast University, Nanjing 210096, China
ClickRemoval: An Interactive Open-Source Tool for Object Removal in Diffusion Models
May 14, 2026Ledun Zhang, Yatu Ji, Xufei Zhuang +1Attention Control in Diffusion Models Image Inpainting
Inner Mongolia University of Technology Hohhot, Inner Mongolia, China

May 18, 2026J/K move · Enter open · S save

Improved Baselines with Representation Autoencoders

Jaskirat Singh, Boyang Zheng, Zongze Wu, Richard Zhang+2

Adobe Research · ANU · New York University

Diffusion Model Guidance Diffusion Transformer Autoencoders Diffusion-Based Image Generation Diffusion Models Diffusion Autoencoder

Open paper AskPDF arXiv

Representation Autoencoders (RAE) replace traditional VAE with pretrained vision encoders. In this paper, we systematically investigate several design choices and find three insights which simplify and improve RAE. First, we study a generalized formulation where the representation is defined as sum of the last k encoder layers rather than solely the final layer. This simple change greatly improves reconstruction without encoder finetuning or specialized data (e.g., text, faces). Second, we study the prevalent assumption that RAE (using pretrained representation as encoder) replaces representation alignment (REPA), which distills the same representation to intermediate layers instead. Through large-scale empirical analysis, we uncover a surprising finding: RAE and REPA exhibit complementary working mechanisms, allowing the same representation to be used as both encoder and target for intermediate diffusion layers. Finally, the original RAE struggles with classifier-free guidance (CFG) and requires training a second, weaker diffusion model for AutoGuidance (AG). We show that REPA itself can be viewed as x-prediction in RAE latent space. By simply re-parameterizing the output of the DiT model, it can provide guidance for "free". Overall, RAEv2 leads to more than 10x faster convergence over the original RAE, achieving a state-of-the-art gFID of 1.06 in just 80 epochs on ImageNet-256. On FDr6, RAEv2 achieves a state-of-the-art 2.17 at just 80 epochs compared to the previous best 3.26 (800 epochs) without any post-training. This motivates EPFID@k (epochs to reach unguided gFID < k) as a measure of training efficiency. RAEv2 attains an EPFID@2 of 35 epochs, versus 177 for the original RAE. We also validate our approach across diverse settings for text-to-image generation and navigation world models, showing consistent improvements. The code is available at https://raev2.github.io.