Diffusion Model Distillation

Follow this topic Ask about this topic

Momentum

25 papers in the last four weeks, up 317% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 134

All topics

Diffusion Fine-tuning with Rewarded Moment Matching Distillation
Jun 29, 2026Alexis Jacq, Guillaume Couairon, Valentin De Bortoli +3Diffusion Model Distillation Diffusion Model Fine-Tuning
1Google DeepMind
Beyond Trajectory Matching: Reflow with Marginal Distribution Alignment
Jun 28, 2026Chen Wang, Peiran Yun, Pan Xie +1Diffusion Model Distillation Distribution Matching
Department of Statistics and Data Science, Tsinghua University · Department of Computer Science and Technology, Tsinghua University · Bytedance Inc
Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models
Jun 24, 2026Kaiwen Zheng, Guande He, Min Zhao +7Action-Conditioned Video Generation Autoregressive Diffusion
Tsinghua University · UT Austin · NVIDIA
ARIA: Adaptive Region-Based Importance Allocation for Conditional Diffusion Distillation
Jun 22, 2026Loay Mualem, Vinh Tong, Samir Darouich +1Diffusion Model Distillation Teacher-Student Learning
Institute for AI, University of Stuttgart · IMPRS-IS · Institute of Theoretical Chemistry, University of Stuttgart.
CoDMD: Copula-aware Distribution Matching Distillation for Fast Video Generation
Jun 20, 2026Wenhu Zhang, Kun Cheng, Changyuan Wang +7Video Diffusion Models Diffusion Model Distillation
1HKUST · 2Wan Team, Alibaba Group · 3THU +2
Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance
Jun 17, 2026Kangsheng Duan, Ziyang Xu, Wenyu Liu +3Diffusion Models for Image Restoration Diffusion Model Distillation
Huazhong University of Science and Technology · *Equal contribution. Completed this work as interns at VIVO AI Lab. · † Project leader. +1
Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation
Jun 16, 2026Siyi Chen, Shaowei Liu, Yixuan Jia +4Video Diffusion Models Diffusion Model Distillation
University of Michigan · NVIDIA · University of Illinois Urbana-Champaign
TeleStyle V2: Beyond Content-Preserving Style Transfer with Self-Distillation and Distribution-Matching-Distillation
Jun 16, 2026Shiwen Zhang, Yifan Xu, Haibin Huang +2Diffusion Model Distillation Image Style Transfer
TeleAI
teasr: training-efficient any-step diffusion transformer for real-world image super-resolution
Jun 15, 2026Xiang Gao, Chenxin Zhu, Yushun Fang +2Diffusion Transformer Diffusion Model Distillation
Shanghai Jiao Tong University, China
Distilling Drifting Transformers with Representation Autoencoders
Jun 14, 2026Jiawei Zhang, Mengfei Xia, Gen Li +1Diffusion Model Distillation Autoencoders
Tsinghua University · Ant Group · CUHK
High-Fidelity Two-Step Image Generation via Teacher-Aligned End-to-End Distillation
Jun 10, 2026Dongyang Liu, Ruoyi Du, David Liu +7Diffusion Model Distillation Few-Step Diffusion Sampling
Z-Image Team, Alibaba Group
AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation
Jun 10, 2026Zeyue Tian, Lei Ke, Zhaoyang Liu +8Text-to-Audio Generation Diffusion Model Distillation
The Hong Kong University of Science and Technology · Noiz AI · Tsinghua University +1
Lip Forcing: Few-Step Autoregressive Diffusion for Real-time Lip Synchronization
Jun 9, 2026Paul Hyunbin Cho, Jinhyuk Jang, SeokYoung Lee +7Diffusion Model Distillation Audio-Visual Synchronization
KAIST AI · 1KAIST AI · AIPARK +1
A Quantitative Approximation Framework for Flow Distillation in Diffusion Models
Jun 2, 2026Weiguo Gao, Ming Li, Lei Shi +1Diffusion Model Distillation Few-Step Diffusion Sampling
School of Mathematical Sciences, Fudan University, Shanghai, 200433, China · Shanghai Key Laboratory of Contemporary Applied Mathematics, Fudan University, Shanghai, 200433, China
Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation
Jun 2, 2026Tianhe Wu, Zikai Zhou, Kun Yan +23Diffusion Model Distillation Few-Step Diffusion Sampling
Why Are DMD Students Lazy? Understanding the Copying Behavior in Few-Step Distillation
Jun 1, 2026Shucheng Li, Iolo Jones, Alexander Tong +1Diffusion Model Distillation Few-Step Diffusion Sampling
Department of Computer Science, University of Oxford, Oxford, UK · AITHYRA, Research Institute for Biomedical AI, Vienna, AT
Restoring Initial Noise Sensitivity in Text-to-Image Distillation via Geometric Alignment
Jun 1, 2026Huayang Huang, Ruoyu Wang, Jinhui Zhao +5Diffusion Model Distillation Diffusion Model Acceleration
School of Computer Science, Wuhan University · MiLM Plus, Xiaomi Inc. · School of Artificial Intelligence, Wuhan University +1
DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models
May 30, 2026Abdullah Al Shafi, Kazi Saeed Alam, Sk Imran Hossain +1Model Compression Classifier-Free Guidance
Khulna University of Engineering & Technology, Bangladesh · University Clermont Auvergne, France
Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models
May 30, 2026Jinyang Du, Shenghao Jin, Ziqian Xu +5Video Diffusion Models Diffusion Model Quantization
SenseTime
SGMD: Score Gradient Matching Distillation for Few-Step Video Diffusion Distillation
May 28, 2026Zhuguanyu Wu, Ruihao Gong, Yang Yong +5Video Diffusion Models Diffusion Model Distillation
Beihang University · SenseTime Research · Hong Kong University of Science and Technology
GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models
May 28, 2026Xiaohang Tang, Keyue Jiang, Che Liu +4Diffusion Model Distillation RL for Diffusion Models
UNIST · University of Basel
Reinforcing Few-step Generators via Reward-Tilted Distribution Matching
May 25, 2026Yushi Huang, Xiangxin Zhou, Ruoyu Wang +3Diffusion Model Alignment Diffusion Model Distillation
Tencent Hunyuan · Hong Kong University of Science and Technology · Westlake University
CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillation
May 25, 2026Fangtai Wu, Hailong Guo, Shijie Huang +7Diffusion Model Distillation Low-Rank Adaptation
Zhejiang University · Qwen Applications Business Group of Alibaba · Xi’an Jiaotong University
ERNIE-Image Technical Report
May 25, 2026Jiaxiang Liu, Zhida Feng, Pengyu Zou +46Personalized Image Aesthetics Assessment Diffusion Model Distillation
Dynamic Neural Koopman Distillation for Fast Robot Control Using Diffusion Models
May 24, 2026Lei Zheng, Peiqi Yu, Zengqi Peng +2Robotic Control Koopman Operator Learning
Department of Electrical and Computer Engineering, National University of Singapore, Singapore · Department of Electrical and Computer Engineering, Carnegie Mellon University, USA · Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology, Hong Kong SAR, China
Variance Reduction for Expectations with Diffusion Teachers
May 20, 2026Jesse Bettencourt, Xindi Wu, Matan Atzmon +2Diffusion Model Distillation Variance Reduction
1NVIDIA · University of Toronto · 3Princeton University
One-Step Distillation of Discrete Diffusion Image Generators via Fixed-Point Iteration
May 20, 2026Chaoyang Wang, Yunhai TongDiffusion Model Distillation Few-Step Diffusion Sampling
Peking University
LIFT and PLACE: A Simple, Stable, and Effective Knowledge Distillation Framework for Lightweight Diffusion Models
May 19, 2026Hyunsoo Han, Sangyeop Yeo, Jaejun YooModel Compression Diffusion Model Distillation
Ulsan National Institute of Science and Technology (UNIST)
Distribution Matching Distillation without Fake Score Network
May 19, 2026Youngjoong Kim, Deokyeong Lee, Jaesik ParkDiffusion Model Distillation Few-Step Diffusion Sampling
Department of Computer Science and Engineering, Seoul National University · Department of Computer Science and Engineering, Sogang University Seoul, Republic of Korea
Stabilizing, Scaling & Enhancing MeanFlow for Large-scale Diffusion Distillation
May 18, 2026Xiao He, Yang Li, Peizhen Zhang +3Flow Matching Diffusion Model Distillation
1State Key Laboratory of Integrated Services Networks, Xidian University · 2Tencent Hunyuan

Jun 29, 2026, cs.LGJ/K move · Enter open · S save

Diffusion Fine-tuning with Rewarded Moment Matching Distillation

Alexis Jacq, Guillaume Couairon, Valentin De Bortoli, Quentin Berthet+2

1Google DeepMind

Diffusion Model Distillation Diffusion Model Fine-Tuning RL for Diffusion Models

Open paper AskPDF arXiv

Distillation and Reinforcement Learning (RL) fine-tuning are the primary pillars of diffusion post-training. While traditionally studied in isolation, the interaction between these phases remains poorly understood, and in particular how fine-tuning impacts the generative quality of distilled models. We introduce Rewarded Moment Matching Distillation (RMMD), a novel framework that simultaneously distills diffusion models and maximizes a reward function. RMMD preserves the high-fidelity ``naturalness'' characteristic of advanced distillation (such as 8-step Moment Matching) by adapting the sampling loop for on-policy training and repurposing the distillation loss as a proxy for integral KL regularization. By evaluating the FID-Reward Pareto fronts on ImageNet, we demonstrate that RMMD achieves superior trade-offs compared to single-step baselines (DI++) and multi-step competitors (DRaFT, HyperNoise). Finally, we apply RMMD to GenCast, a state-of-the-art weather forecasting model, to distill it while optimizing the Continuous Ranked Probability Score (CRPS) metric. The resulting distilled model achieves a 7.5x speedup while outperforming the teacher model on 93% of target weather variables, and being better calibrated. This proves that RMMD scales to complex, high-dimensional scientific domains.

Momentum

Related topics

Latest papers 134