Diffusion Model Distillation

Follow this topic Ask about this topic

Diffusion model distillation studies how to make models that generate images, video, audio, text, and other data through repeated denoising faster or smaller while retaining their capabilities. Common approaches train a student to match a teacher’s denoising trajectory or output distribution, progressively reduce sampling steps, or use consistency and flow-matching objectives; compression and guidance distillation can further reduce cost. Newer work also uses on-policy and self-distillation, where the student’s own generation paths guide training, and reward-based objectives to improve preference alignment. A central tension is reducing inference cost without losing diversity, fine detail, temporal consistency, or reliability, especially in very few-step and long-horizon generation.

Latest papers

191 papers39 papers in the last four weeks, up 86% on the four weeks before. 0.2% of all new papers.

Few-Step Diffusion Language Models via Trajectory Self-Distillation
Feb 12, 2026Tunyu Zhang, Xinxi Zhang, Ligong Han +10Diffusion Model Distillation Few-Step Diffusion Sampling
Rutgers University · Red Hat AI Innovation · MIT-IBM Watson AI Lab
Toward Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion
Jan 22, 2026Yonghao Xu, Pedram Ghamisi, Qihao WengRemote Sensing Image Understanding Diffusion Model Distillation
Computer Vision and Learning Systems, Department of Electrical Engineering, Linköping University, 581 83 Linköping, Sweden · Helmholtz-Zentrum Dresden-Rossendorf, Responsible AI Group, 09599 Freiberg, Germany · University of Iceland, Faculty of Electrical and Computer Engineering, Reykjavik 102, Iceland +1
SnapGen++: Unleashing Diffusion Transformers for Efficient High-Fidelity Image Generation on Edge Devices
Jan 13, 2026Dongting Hu, Aarush Gupta, Magzhan Gabidolla +12Diffusion Transformer Diffusion Model Distillation
Snap Inc. · University of Melbourne · MBZUAI
InstantViR: Real-Time Video Inverse Problem Solver with Distilled Diffusion Prior
Nov 18, 2025Weimin Bai, Suzhe Xu, Yiwei Ren +4Diffusion-Based Inverse Problems Video Diffusion Models
Peking University · Huaqiao University · Kuaishou Technology
Distribution Matching Distillation Meets Reinforcement Learning
Nov 17, 2025Dengyang Jiang, Dongyang Liu, Zanyi Wang +12Diffusion Model Alignment Diffusion Model Distillation
1HKUST · 2Alibaba Group · 3SIAT, CAS +3
SPID: Distilled Protein Backbone Generation
Oct 3, 2025Liyang Xie, Haoran Zhang, Zhendong Wang +2Diffusion Model Distillation Few-Step Diffusion Sampling
The University of Texas at Austin · The Memorial Sloan Kettering Cancer Center
Sig-DEG for Distillation: Making Diffusion Models Faster and Lighter
Aug 23, 2025Wen Ge, Lei Jiang, Lingyi Yang +7Path Signatures Diffusion Model Distillation
University College London · University of Oxford · JPMorganChase +1
Vivid-VR: Distilling Concepts from Text-to-Video Diffusion Transformer for Photorealistic Video Restoration
Aug 20, 2025Haoran Bai, Xiaoxu Chen, Canqian Yang +3Video Diffusion Models Diffusion Transformer
Alibaba Group - Taobao & Tmall Group
DLM-One: Diffusion Language Models for One-Step Sequence Generation
May 30, 2025Tianqi Chen, Shujian Zhang, Mingyuan ZhouDiffusion Language Models Diffusion Model Distillation
The University of Texas at Austin, Austin, TX 78712, USA
Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation
May 28, 2025Yi Zhang, Peng Wang, Difan ZouPhysics-Informed Diffusion Models Diffusion Model Distillation
Institute of Data Science · Department of Computer and Information Science · Institute of Data Science & School of Computing and Data Science
DICE: Distilling Classifier-Free Guidance into Text Embeddings
Feb 6, 2025Zhenyu Zhou, Defang Chen, Can Wang +2Diffusion Model Alignment Diffusion Model Guidance
Zhejiang University, State Key Laboratory of Blockchain and Data Security · University at Buffalo, State University of New York

Feb 12, 2026J/K move · Enter open · S save

Few-Step Diffusion Language Models via Trajectory Self-Distillation

Tunyu Zhang, Xinxi Zhang, Ligong Han, Haizhou Shi+9

Rutgers University · Red Hat AI Innovation · MIT-IBM Watson AI Lab

Diffusion Model Distillation Few-Step Diffusion Sampling Diffusion Language Model Decoding Diffusion Language Models Diffusion Language Model Distillation

Open paper AskPDF arXiv

Diffusion large language models (DLLMs) have emerged as powerful generative models with the promise of fast text generation through parallel decoding. However, realizing this potential in practice remains challenging: reducing the number of decoding steps, typically causes a substantial degradation in output quality due to token factorization error. To alleviate this, we propose a self-distillation framework that trains a few-step student to match the generative trajectory of a full-step teacher. We theoretically and empirically show that trajectory-level supervision mitigates this factorization error, thereby enabling effective few-step decoding. We further incorporate Direct Discriminative Optimization (DDO), a reverse-KL objective that encourages mode-seeking toward the teacher's modes, yielding stronger performance on challenging reasoning tasks. Across reasoning and code-generation benchmarks, our method substantially narrows the gap between few-step and full-step decoding. The source code is available at https://github.com/Tyrion58/T3D.