cs.CVNov 2, 2026

Latent Diffusion-GAN: Adversarial Learning in the Autoencoded Latent Space

Authors: U-Chae Jun, Jaeeun Ko, Jiwoo Kang

Organizations: Sookmyung Women’s University, South Korea

Abstract

Diffusion models are powerful generative frameworks for producing high-quality images by denoising latent variables from random noise. However, training with likelihood-based objectives, such as denoising score matching, can lead to locally over-smoothed high-frequency details, including fine textures and sharp edges, thereby limiting perceptual fidelity and structural detail. Adversarial training with GANs enhances sharpness but typically requires additional discriminator networks, increasing computational costs and destabilizing training. To this end, we propose Latent Diffusion Generative Adversarial Networks (LD-GAN), a novel framework that seamlessly integrates adversarial learning into diffusion models without modifying their original pipeline. LD-GAN leverages the pretrained variational autoencoder (VAE) in latent diffusion models as an energy-based discriminator, enabling adversarial training without extra parameters and preserving the structured latent priors learned from large datasets. We also introduce a structural consistency energy that aligns encoder and decoder feature representations, thereby enhancing perceptual quality and compatibility with the pretrained latent space. Extensive experiments demonstrate that LD-GAN significantly improves sample fidelity, perceptual sharpness, and diversity over state-of-the-art baseline methods across various generation tasks while ensuring efficient training dynamics.

Explore similar work

CardsList
  1. Geometric Autoencoder for Diffusion Models

    Mar 11, 2026Hangyu Liu, Jianyong Wang, Yutao SunLatent Diffusion ModelsDiffusion Models

  2. Taming Sampling Perturbations with Variance Expansion Loss for Latent Diffusion Models

    Mar 22, 2026Qifan Li, Xingyu Zhou, Jinhua Zhang +2Latent-Space RegularizationLatent Diffusion Models

  3. Adversarial Training for Pixel Diffusion

    Sep 29, 2026Xin Lin, Zhifei Zhang, Yuqian Zhou +4Adversarial TrainingPixel-Space Diffusion Models