Adaptive Knowledge Distillation

Ask about this topic

Latest papers

CardsList

27 papers15 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

  1. O2^2PD: Overlap-Gated On-Policy Distillation on Reliable Prefix

    Nov 5, 2026Shuaishuai Zu, Jihao Zhao, Pengfei Li +5Language Model DistillationOn-Policy Distillation

    School of Information, Renmin University of China · Huawei Technologies Ltd.

  2. EvoOPD: Evolving Privileged Information Strategies for Black-Box On-Policy Distillation

    Nov 4, 2026Ximo Zhu, Ruiqi Liu, Yi Han +6Black-Box Knowledge DistillationLanguage Model Distillation

    University of the Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Southwest University +4

  3. Adaptive-Policy Distillation: Knowledge Injection under Information Asymmetry

    Oct 31, 2026Hojun Cho, Kang Eun Jeon, Jaegul ChooLanguage Model DistillationAdaptive Knowledge Distillation

    KAIST · DGIST

  4. Task-Performance Routing for Multi-Teacher Distillation in CT Medical Image Segmentation

    Oct 30, 2026Jiaye Yang, Yao Liu, Die Dai +5Multi-Teacher Knowledge Distillation3D Medical Image Segmentation

    University of Electronic Science and Technology of China, Chengdu, 611731, China · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE

  5. Every Token Needs Its Own Teacher: Multi-teacher Adaptive On-policy Self-distillation

    Oct 29, 2026Yifu Luo, Chuang Ma, Zeyu Chen +3On-Policy Self-DistillationLanguage Model Distillation

    TU München · Munich Center for Machine Learning (MCML) · relAI – Konrad Zuse School of Excellence in Reliable AI +4

  6. On-Policy Self-Distillation with Teacher Evolution

    Oct 28, 2026Shufan Shen, Zhongni Hou, Chao Bi +3On-Policy Self-DistillationLanguage Model Distillation

    State Key Lab. of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Meituan

  7. ATI-OPD: Adaptive Teacher Intervention for Stable Long-Sequence On-Policy Distillation

    Oct 28, 2026Jiannan Guan, Youxin Jiang, Yutai Hou +9On-Policy DistillationAdaptive Knowledge Distillation

    Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology · Huawei Technologies Co., Ltd.

  8. REDAR: Reinforcement-Driven Adaptive Regularisation for Class Incremental Learning

    Oct 22, 2026Akila Ayanthi, Darshana Priyasad, Tharindu Fernando +3RL HPOClass-Incremental Learning

    SAIVT, Queensland University of Technology, Australia · CSIRO Robotics, CSIRO, Australia

  9. APED-CXR: Adaptive Privileged Evidence Distillation for Chest X-Ray Grounding with Image-Only Inference

    Oct 21, 2026Md Shihabul Islam Shovo, Nishi Kanta Paul, Jamil FayyadMedical Image GroundingAdaptive Knowledge Distillation

    NIMISHES Lab, Dhaka, Bangladesh · American University of Sharjah, Sharjah, UAE

  10. Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation

    Oct 20, 2026Guangyu Liu, Siyuan Gan, Shilong Zhou +3Token-Level Loss WeightingLanguage Model Distillation

    State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China

  11. CheckDistill: Adaptive Checklist Distillation for Efficient Distributional Reward Modeling in Image Editing

    Oct 19, 2026Yaqi Cai, Zuan Gao, Mengting Chen +2Reward ModelingMultimodal Reward Modeling

    Taobao & Tmall group of Alibaba

  12. Nash-MOPD: Optimizing Task Trade-offs in Multi-Teacher On-Policy Distillation

    Oct 19, 2026Siyuan Gan, Bo Yang, Linjian Meng +4Language Model DistillationMulti-Teacher Knowledge Distillation

    State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Yili Normal University, Yining, China

  13. Beyond Policy Discrepancy: Capability-Aware Multi-Teacher On-Policy Distillation

    Oct 18, 2026Xuanfeng Yang, Le Gai, Jiaming Wang +1Language Model DistillationMulti-Teacher Knowledge Distillation

    Tsinghua University

  14. SPADE-KD: Peer-to-Peer Collaborative Learning under Heterogeneous Settings

    Oct 15, 2026Harikrishna Kuttivelil, Katia ObraczkaFederated Knowledge DistillationAdaptive Knowledge Distillation

    University of California, Santa Cruz

  15. RELAY: Selective Self-Distillation with Adaptive Hand-off for Rubric-Based Reinforcement Learning

    Oct 13, 2026Kunwang Zheng, Xinyu Sun, Yu Cao +7Adaptive Knowledge DistillationRubric-Based RL

    University of Science and Technology of China · Fudan University · Alibaba Group

  16. Representational Difference Distillation

    Oct 11, 2026Abdulkarim Mugisha, Divin IrakizaAdaptive Knowledge DistillationKnowledge Distillation

    Department of Computing + Mathematical Sciences, California Institute of Technology, Pasadena, US.

  17. PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation

    Oct 8, 2026Heng Li, Yong Zhang, Ning Cheng +5Few-Shot Text ClassificationAdaptive Knowledge Distillation

    Ping An Technology (Shenzhen) Co., Ltd., China · University of Science and Technology of China

  18. ALRA: Adaptive Local Relational Alignment for Logit-Based Pre-training Distillation of Autoregressive Language Models

    Sep 3, 2026Quang Hoang Trung, Quang Huu Hieu, Nguyen Van Hoang Phuc +1Language Model PretrainingAutoregressive Language Modeling

    VJ Technologies, Da Nang, Vietnam · AJ Technologies, Nagoya, Japan · Vietnam National University, Ho Chi Minh City, Vietnam +1

  19. Temperature-Adaptive Transformed Teacher Matching

    Aug 29, 2026Hiroaki Aizawa, Yoshikazu HayashiKnowledge DistillationAdaptive Knowledge Distillation

    Graduate School of Advanced Science and Engineering Hiroshima University, Japan · Intelligent Production Technology Research & Development Center for Aerospace Gifu University, Japan

  20. DualOPSD: Closing the Feedback Loop in On-Policy Self-Distillation

    Aug 26, 2026Yutong Chen, Guangfu Guo, Zhichao Xu +1Adaptive Knowledge DistillationOn-Policy Self-Distillation

    Department of Computer Science, Clemson University, 105 Sikes Hall, Clemson, SC 29634, USA · Department of Computer Science, University of Utah

  21. Matching Supervision to the Student's Learning Capacity: A Unified Framework for On-Policy Self-Distillation

    Aug 8, 2026Yongkang Yang, Zhezheng Hao, Hong Zhang +8Adaptive Knowledge DistillationOn-Policy Self-Distillation

    Tencent, Workbuddy Team · Zhejiang University · Independent Researcher

  22. LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models

    Jul 13, 2026Gangsu Kim, Won-Ki JeongComputational PathologyAdaptive Knowledge Distillation

    Department of Computer Science and Engineering, College of Informatics, Korea University, Republic of Korea

  23. ARKD: Adaptive Reinforcement Learning-Guided Bidirectional KL Divergence Distillation for Text Generation

    Jun 29, 2026Zilong Liu, Xuewen Zhang, Jinrui Xing +3RL for Knowledge DistillationLanguage Model Distillation

    Li Auto, Beijing, China · School of Software and Microelectronics, Peking University, China

  24. RPM-Distill: Physiology-guided Adaptive Cross-modal Distillation for Robust Remote Physiological Measurement

    Jun 26, 2026Jiyao Wang, Qingyong Hu, Duoxun Tang +3Remote PhotoplethysmographyAdaptive Knowledge Distillation

    McGill University, Montreal, Canada · Hong Kong University of Science and Technology, Hong Kong, China · Tsinghua University, Shenzhen, China +1

  25. BAHSD: Bridging the Long-tail Gap via Adaptive Distillation in Black-box Sequential Recommendation

    Jun 2, 2026Xi Zhou, Famin Wu, Mingming Li +4Adaptive Knowledge DistillationSequential Recommendation

    Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Beijing Institute for General Artificial Intelligence, Beijing, China

  26. Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning

    May 12, 2026Zihao Han, Tiangang Zhang, Huaibin Wang +1Adaptive Knowledge DistillationOn-Policy Self-Distillation

    ByteDance Douyin

  27. PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence

    Mar 11, 2026Yuanda Xu, Hejian Sang, Zhengze Zhou +2Language Model DistillationAdaptive Knowledge Distillation

    LinkedIn Corporation