cs.LGMay 30, 2025

On Designing Diffusion Autoencoders for Efficient Generation and Representation Learning

Authors: Magdalena Proszewska, Esmeralda S. Whitammer, N. Siddharth

Organizations: School of Informatics, University of Edinburgh

Abstract

Diffusion autoencoders (DAs) are variants of diffusion models that use an input-dependent latent variable to capture representations alongside the diffusion process. The latent variable is typically used for tasks such as downstream classification, controllable generation, and interpolation. However, the latent variable has an effect on unconditional generation, with performance strongly dependent on how this variable is modelled and learnt and how it is used in the denoising process. Here, we draw a connection between DAs and a particular class of improved diffusion models---those that learn their forward (noising) process---to argue that DAs have unrealised potential as generative models. We develop a variant of DAs, termed diffusion models with zz (DMZ), to show that the adoption of certain design decisions, such as the choice of latent variables and conditioning method, can help realise this generative potential. Through extensive experiments and ablations, we show DMZ converges in fewer iterations, generates high-quality samples with fewer denoising steps, and learns effective latent representations for downstream tasks like classification and domain transfer.

Figures & tables

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models

    Jun 14, 2023Yingheng Wang, Yair Schiff, Aaron Gokaslan +4Diffusion ModelsDisentangled Representation Learning

  2. UniRA: Unified Representation Alignment and Auto-Encoding for Efficient Diffusion Generation

    Oct 29, 2026An Lanji, Dawei Liu, Jin Li +3Diffusion-Based Image GenerationDiffusion Models

  3. LDM-is-AE: Latent Diffusion Model is an Auto-Encoder for End-to-End Image Generation

    Sep 29, 2026Zhengqiang Zhang, Lingchen Sun, Rongyuan Wu +4Diffusion TransformerLatent Diffusion Models