Diffusion Model Distillation

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 300% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 131

All topics

Few-Step Diffusion Language Models via Trajectory Self-Distillation
Feb 12, 2026Tunyu Zhang, Xinxi Zhang, Ligong Han +10Diffusion Model Distillation Few-Step Diffusion Sampling
Rutgers University · Red Hat AI Innovation · MIT-IBM Watson AI Lab
Toward Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion
Jan 22, 2026Yonghao Xu, Pedram Ghamisi, Qihao WengRemote Sensing Image Understanding Diffusion Model Distillation
Computer Vision and Learning Systems, Department of Electrical Engineering, Linköping University, 581 83 Linköping, Sweden · Helmholtz-Zentrum Dresden-Rossendorf, Responsible AI Group, 09599 Freiberg, Germany · University of Iceland, Faculty of Electrical and Computer Engineering, Reykjavik 102, Iceland +1
SnapGen++: Unleashing Diffusion Transformers for Efficient High-Fidelity Image Generation on Edge Devices
Jan 13, 2026Dongting Hu, Aarush Gupta, Magzhan Gabidolla +12Diffusion Transformer Diffusion Model Distillation
Snap Inc. · University of Melbourne · MBZUAI
Distribution Matching Distillation Meets Reinforcement Learning
Nov 17, 2025Dengyang Jiang, Dongyang Liu, Zanyi Wang +12Diffusion Model Alignment Diffusion Model Distillation
1HKUST · 2Alibaba Group · 3SIAT, CAS +3
SPID: Distilled Protein Backbone Generation
Oct 3, 2025Liyang Xie, Haoran Zhang, Zhendong Wang +2Diffusion Model Distillation Few-Step Diffusion Sampling
The University of Texas at Austin · The Memorial Sloan Kettering Cancer Center
Vivid-VR: Distilling Concepts from Text-to-Video Diffusion Transformer for Photorealistic Video Restoration
Aug 20, 2025Haoran Bai, Xiaoxu Chen, Canqian Yang +3Video Diffusion Models Diffusion Transformer
Alibaba Group - Taobao & Tmall Group
DLM-One: Diffusion Language Models for One-Step Sequence Generation
May 30, 2025Tianqi Chen, Shujian Zhang, Mingyuan ZhouDiffusion Language Models Diffusion Model Distillation
The University of Texas at Austin, Austin, TX 78712, USA
Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation
May 28, 2025Yi Zhang, Peng Wang, Difan ZouPhysics-Informed Diffusion Models Diffusion Model Distillation
Institute of Data Science · Department of Computer and Information Science · Institute of Data Science & School of Computing and Data Science
DICE: Distilling Classifier-Free Guidance into Text Embeddings
Feb 6, 2025Zhenyu Zhou, Defang Chen, Can Wang +2Diffusion Model Alignment Diffusion Model Guidance
Zhejiang University, State Key Laboratory of Blockchain and Data Security · University at Buffalo, State University of New York
DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models
Date pendingQuanhao Li, Junqiu Yu, Kaixun Jiang +7Diffusion Model Distillation Multi-Task RL
Fudan University, China · Alibaba Group, China
Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One
Date pendingFred Zhangzhi Peng, Kaiwen Zheng, Anru R. ZhangDiffusion Model Distillation One-Step Generative Modeling
Duke University · Tsinghua University

Feb 12, 2026, cs.CLJ/K move · Enter open · S save

Few-Step Diffusion Language Models via Trajectory Self-Distillation

Tunyu Zhang, Xinxi Zhang, Ligong Han, Haizhou Shi+9

Rutgers University · Red Hat AI Innovation · MIT-IBM Watson AI Lab

Diffusion Model Distillation Few-Step Diffusion Sampling Diffusion Language Model Decoding Diffusion Language Models Diffusion Language Model Distillation

Open paper AskPDF arXiv

Diffusion large language models (DLLMs) have emerged as powerful generative models with the promise of fast text generation through parallel decoding. However, realizing this potential in practice remains challenging: reducing the number of decoding steps, typically causes a substantial degradation in output quality due to token factorization error. To alleviate this, we propose a self-distillation framework that trains a few-step student to match the generative trajectory of a full-step teacher. We theoretically and empirically show that trajectory-level supervision mitigates this factorization error, thereby enabling effective few-step decoding. We further incorporate Direct Discriminative Optimization (DDO), a reverse-KL objective that encourages mode-seeking toward the teacher's modes, yielding stronger performance on challenging reasoning tasks. Across reasoning and code-generation benchmarks, our method substantially narrows the gap between few-step and full-step decoding. The source code is available at https://github.com/Tyrion58/T3D.

Momentum

Related topics

Latest papers 131