Adversarial Knowledge Distillation

Overview

Adversarial knowledge distillation studies how to transfer useful behavior from a teacher model to a smaller or faster student using adversarial training signals. Methods include adversarial example generation, logit and feature distillation, discriminator-based distribution matching, and on-policy feedback for language and video generation. They are evaluated in settings such as CIFAR and ImageNet robustness, certified classification, black-box language-model distillation, and few-step image or video generation. A central challenge is making the transfer stable when teacher and student differ, while balancing clean accuracy with robustness or preserving generation quality with fewer steps.

Latest papers

CardsList

23 papers5 papers in the last four weeks, down 17% on the four weeks before. 0.0% of all new papers.

  1. VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation

    Nov 4, 2026Yihong Luo, Tianyu Pang, Tianyang HuDiffusion Model DistillationFew-Step Diffusion Sampling

  2. Efficient Hybrid Distillation: Synergizing Score and Adversarial Objectives for One-Step Diffusion

    Nov 2, 2026Fei Peng, Junqiang Wu, Haoxian Tan +5Diffusion Model DistillationScore Distillation Sampling

  3. ROAD: Modern Adversarial Training for Simple One-Step Distillation at Scale

    Oct 29, 2026Xiaoou Li, Shan You, Tao HuangDiffusion Model DistillationDiffusion Distillation

  4. Adversarial Self-Distillation

    Oct 28, 2026Mohammad Sadegh Akhondzadeh, Soroush H. Zargarbashi, Aleksandar BojchevskiAdversarial Robustness of Language ModelsSafety-Preserving Fine-Tuning of Language Models

  5. TAD: Distilling Vision-Language-Action Policies from Teacher Trajectories Alone

    Oct 24, 2026Yun Zhu, Da WangVisuomotor Policy LearningKnowledge Distillation

  6. CARD: Closing the Teacher-Student Loop for Adversarial Robustness Distillation

    Oct 14, 2026Zhi Li, Haowei Liu, Hongchen Yang +5Adversarial RobustnessAdversarial Knowledge Distillation

  7. Collaboratively Guided Adversarial Robust Distillation with Teacher-Favorable Examples

    Oct 8, 2026Zhi Li, Haowei Liu, Hongchen Yang +4Adversarial RobustnessAdversarial Knowledge Distillation

  8. DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

    Oct 1, 2026Zhengming Yu, Junkun Yuan, Haotian Yang +8Diffusion Model DistillationFew-Step Diffusion Sampling

  9. Enhancing Autoregressive Video Generation via Representation Adversarial Distillation

    Sep 30, 2026Fangyu Lin, Xingtong Ge, Lunjie Zhu +8Long-Horizon Video GenerationDiffusion Model Distillation

  10. Revisiting On-policy Adversarial Black-Box Distillation: Calibrating Groupwise Reward Geometry for Effective Advantage Construction

    Sep 30, 2026Xiao Cui, Mo Zhu, Yulei Qin +3Black-Box Knowledge DistillationGroup Relative Policy Optimization

  11. Persistent Negatives for Adversarial Black-Box On-Policy Distillation

    Sep 25, 2026Haixu Ma, Saad Lahrichi, Weiwei Li +11On-Policy DistillationAdversarial Knowledge Distillation

  12. Harnessing Adversarial Distillation to Customise Debiased, Disease-Specific Pathology Foundation Models for Breast Cancer

    Aug 2, 2026Zhiwei Chen, Yang Hu, Yuxiang Xiao +7Computational PathologyPathology Foundation Models

  13. Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers

    Jul 30, 2026Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi +1Adversarial TrainingNeural Network Robustness

  14. Improving Certified Robustness via Adversarial Distillation

    Jun 30, 2026Matteo Melis, Jesus Martinez Del Rincon, Vishal SharmaNeural Network RobustnessCertified Robustness

  15. AAD-1: Asymmetric Adversarial Distillation for One-Step Autoregressive Video Generation

    Jun 2, 2026Haobo Li, Yanhong Zeng, Yunhong Lu +6Image-to-Video GenerationAutoregressive Video Generation

  16. On-Policy Adversarial Flow Distillation for Autoregressive Video Generation

    May 25, 2026Yang Luo, Shengju Qian, Xiaohang Tang +4Autoregressive Video GenerationOn-Policy Distillation

  17. Toward Understanding Adversarial Distillation: Why Robust Teachers Fail

    May 21, 2026Hongsin Lee, Hye Won ChungAdversarial TrainingNeural Network Robustness

  18. Estimating the Black-box LLM Uncertainty with Distribution-Aligned Adversarial Distillation

    May 7, 2026Huizi Cui, Huan Ma, Qilin Wang +2Language Model DistillationLLM Uncertainty Estimation

  19. FedRio: Personalized Federated Social Bot Detection via Cooperative Reinforced Contrastive Adversarial Distillation

    Apr 12, 2026Yingguang Yang, Hao Liu, Xin Zhang +8Social Bot DetectionFederated Learning

  20. Dual-Space Knowledge Distillation with Key-Query Matching for Large Language Models with Vocabulary Mismatch

    Mar 23, 2026Stella Eva Tsiapali, Cong-Thanh Do, Kate KnillCross-Tokenizer Knowledge DistillationLanguage Model Distillation

  21. Robustness of Vision Language Models Against Split-Image Harmful Input Attacks

    Feb 8, 2026Md Rafi Ur Rashid, MD Sadik Hossain Shanto, Vishnu Asutosh Dasu +1VLM RobustnessAdversarial Attacks on VLMs

  22. Learning Better Certified Models from Empirically-Robust Teachers

    Feb 2, 2026Alessandro De PalmaCertified RobustnessAdversarial Knowledge Distillation