Adversarial Knowledge Distillation

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 12

All topics

DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
Oct 1, 2026Zhengming Yu, Junkun Yuan, Haotian Yang +8Diffusion Model Distillation Few-Step Diffusion Sampling
Texas A&M University · Intelligent Creation, ByteDance
Enhancing Autoregressive Video Generation via Representation Adversarial Distillation
Sep 30, 2026Fangyu Lin, Xingtong Ge, Lunjie Zhu +8Long-Horizon Video Generation Diffusion Model Distillation
The Hong Kong University of Science and Technology · Vivix Group Limited · Zhejiang University
Revisiting On-policy Adversarial Black-Box Distillation: Calibrating Groupwise Reward Geometry for Effective Advantage Construction
Sep 30, 2026Xiao Cui, Mo Zhu, Yulei Qin +3Group Relative Policy Optimization Language Model Distillation
University of Science and Technology of China · Zhejiang University · Independent Researcher
Harnessing Adversarial Distillation to Customise Debiased, Disease-Specific Pathology Foundation Models for Breast Cancer
Aug 2, 2026Zhiwei Chen, Yang Hu, Yuxiang Xiao +7Computational Pathology Pathology Foundation Models
School of Computer Science and Engineering, South China University of Technology, China · School of Computing and Mathematical Sciences, University of Leicester, UK · Leicester Cancer Research Centre, University of Leicester, UK +4
Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers
Jul 30, 2026Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi +1Adversarial Training Neural Network Robustness
Joint Graduate School of Mathematics for Innovation Kyushu University · Faculty of Information Science and Electrical Engineering Kyushu University · Université Savoie Mont Blanc
Improving Certified Robustness via Adversarial Distillation
Jun 30, 2026Matteo Melis, Jesus Martinez Del Rincon, Vishal SharmaNeural Network Robustness Certified Robustness
School of EEECS Queen’s University Belfast · School of EEECS2026 Queen’s University Belfast
AAD-1: Asymmetric Adversarial Distillation for One-Step Autoregressive Video Generation
Jun 2, 2026Haobo Li, Yanhong Zeng, Yunhong Lu +6Image-to-Video Generation Autoregressive Video Generation
AutoLab, SAI, SJTU · Ant Group · Department of Automation, Tsinghua University +2
On-Policy Adversarial Flow Distillation for Autoregressive Video Generation
May 25, 2026Yang Luo, Shengju Qian, Xiaohang Tang +4Autoregressive Video Generation On-Policy Distillation
National Univesity of Singapore · LIGHTSPEED · University College London
One-Forcing: Towards Stable One-Step Autoregressive Video Generation
May 22, 2026Jiaqi Feng, Justin Cui, Yuanhao Ban +1Video Diffusion Models One-Step Generative Modeling
Tsinghua University · UCLA
Toward Understanding Adversarial Distillation: Why Robust Teachers Fail
May 21, 2026Hongsin Lee, Hye Won ChungAdversarial Training Neural Network Robustness
School of Electrical Engineering, KAIST, Daejeon, Korea.
Estimating the Black-box LLM Uncertainty with Distribution-Aligned Adversarial Distillation
May 7, 2026Huizi Cui, Huan Ma, Qilin Wang +2Language Model Distillation LLM Uncertainty Estimation
School of Computer Science and Technology, Tianjin University, China · School of Future Technology, Tianjin University, China · Georgia Tech Shenzhen Institute, Tianjin University, China +1
Robustness of Vision Language Models Against Split-Image Harmful Input Attacks
Feb 8, 2026Md Rafi Ur Rashid, MD Sadik Hossain Shanto, Vishnu Asutosh Dasu +1VLM Robustness Adversarial Attacks on VLMs
Pennsylvania State University · Bangladesh University of Engineering and Technology

Oct 1, 2026, cs.CVJ/K move · Enter open · S save

DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

Zhengming Yu, Junkun Yuan, Haotian Yang, Gordon Guocheng Qian+7

Texas A&M University · Intelligent Creation, ByteDance

Diffusion Model Distillation Few-Step Diffusion Sampling Diffusion-Based Image Generation Distribution Matching Distillation Adversarial Knowledge Distillation

Open paper AskPDF arXiv

Distribution Matching Distillation (DMD) trains a few-step student from the difference between separately estimated target and student scores, so it must keep an auxiliary diffusion model fitted to the student's evolving distribution at extra memory and computation cost. We introduce DMAD, Distribution Matching as Adversarial Distillation, which recasts distribution matching as classification and learns the required log-density ratios directly. Two discriminator heads on a shared backbone distinguish real data and teacher samples from the student's, and linear losses on their logits train the student without auxiliary score fitting. We prove that at the discriminator optimum these losses recover the distribution-matching gradient underlying DMD, through the classical identity linking discriminator logits to log-density ratios. We further introduce gap-based reweighting, which adapts teacher supervision across noise levels from the real-data head's empirical logit gap between real and teacher samples. DMAD reaches a Fréchet Inception Distance (FID) of 1.04 with one-step generation on ImageNet-64x64, 14.47 with four-step SDXL on COCO-10K, and a VBench total score of 85.15 with four-step Wan2.1-T2V-14B, the best values among the compared few-step methods and the multi-step teachers. On MiniMax-H3-33B, our four-step student achieves overall human preference rates of 79.1% over DMD2 and 84.6% over rCM for joint audio-video generation, excluding ties. Our code, models and demos are available at https://yzmblog.github.io/projects/DMAD.

Momentum

Related topics

Latest papers 12