Knowledge Distillation

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

9 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Period ending 2026-09-14

8 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Period ending 2026-09-07

13 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Inside this field

Focused directions

418 papers

Latest in Knowledge Distillation

Browse all topics →
CardsList

  1. Improving Diversity in Black-box Few-shot Knowledge Distillation

    Apr 28, 2026Tri-Nhan Vo, Dang Nguyen, Kien Do +1Knowledge DistillationFew-Shot Learning

  2. Diverse Image Priors for Black-box Data-free Knowledge Distillation

    Apr 28, 2026Tri-Nhan Vo, Dang Nguyen, Trung Le +2Knowledge DistillationPrior Knowledge

  3. The Surprising Effectiveness of Canonical Knowledge Distillation for Semantic Segmentation

    Apr 28, 2026Muhammad Ali, Kevin Alexander Laube, Madan Ravi Ganesh +3Knowledge DistillationResnet-18

  4. Knowledge Distillation Must Account for What It Loses

    Apr 28, 2026Wenshuo WangKnowledge DistillationPreservation

  5. Alice v1: Distillation-Enhanced Video Generation Surpassing Closed-Source Models

    Apr 27, 2026Wang Xiaoyu, Phong Nguyen, Chen ZhaoLarge-Scale Video Generation ModelsAsymmetric Distillation

  6. S-SONDO: Self-Supervised Knowledge Distillation for General Audio Foundation Models

    Apr 27, 2026Mohammed Ali El Adlouni, Aurian Quelennec, Pierre Chouteau +2Audio LatentKnowledge Distillation

  7. Large Language Models Explore by Latent Distilling

    Apr 27, 2026Yuanhao Zeng, Ao Lu, Lufei Li +3DiversityBatch Distillation

  8. Selective Correlation Based Knowledge Distillation for Ground Reaction Force Estimation

    Apr 27, 2026Eun Som Jeon, Jisoo Lee, Huisu Lim +3Gait RecognitionGround Reaction Forces

  9. ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation

    Apr 26, 2026Boqin Yuan, Yue Su, Renchu Song +2Reusable Agent SkillsAction Traces

  10. RouteNLP: Closed-Loop LLM Routing with Conformal Cascading and Distillation Co-Optimization

    Apr 26, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLarge Language Model RoutingInference Cost

  11. Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM

    Apr 25, 2026Jingxuan Kang, Ziqi Zhang, Shaoming Zheng +9Few-Shot Medical Image SegmentationSegment Anything Model

  12. Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Models

    Apr 25, 2026Max Hartman, Vidhata Jayaraman, Moulik Choraria +2Asymmetric DistillationReasoning Traces

  13. Toward Real-World Adoption of Portrait Relighting via Hybrid Domain Knowledge Fusion

    Apr 25, 2026Qian Huang, Mayoore Selvarasa Jaiswal, Zhen Zhong +2Image RelightingSynthetic-To-Real Domain Gap

  14. Distilling Vision Transformers for Distortion-Robust Representation Learning

    Apr 24, 2026Konstantinos Alexis, Giorgos Giannopoulos, Dimitrios GunopulosSelf-Supervised Vision TransformersSelf-Supervised Representations

  15. Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation

    Apr 23, 2026Nikita Severin, Danil Kartushov, Vladislav Urzhumov +8Recommender SystemsSequence Modeling

  16. Hybrid Policy Distillation for LLMs

    Apr 22, 2026Wenhong Zhu, Ruobing Xie, Rui Wang +1Knowledge DistillationLarge Language Model Post-Training

  17. LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation

    Apr 21, 2026Siqing Song, Chuang Wang, Yong Lang +2Large Language Model QuantizationExtreme Low-Bit Quantization

  18. Distillation Traps and Guards: A Calibration Knob for LLM Distillability

    Apr 21, 2026Weixiao Zhan, Yongcheng Jing, Leszek Rutkowski +1Knowledge DistillationMulticalibration and Calibration

  19. Rethinking Dataset Distillation: Hard Truths about Soft Labels

    Apr 20, 2026Priyam Dey, Aditya Sahdev, Sunny Bhati +2Geometry-Aware Uncertainty CoresetsSoft-Label

  20. SemLT3D: Semantic-Guided Expert Distillation for Camera-only Long-Tailed 3D Object Detection

    Apr 20, 2026Hao Vo, Khoa Vo, Thinh Phan +53D Object Detection3D Object Understanding

  21. IceBreaker for Conversational Agents: Breaking the First-Message Barrier with Personalized Starters

    Apr 20, 2026Hongwei Zheng, Weiqi Wu, Zhengjia Wang +6Conversational Artificial IntelligenceArtificial Intelligence Assistants

  22. Soft Label Pruning and Quantization for Large-Scale Dataset Distillation

    Apr 20, 2026Xiao Lingao, Yang HeChain-Of-Thought DistillationCandidate Label Distributions

  23. Recovery Guarantees for Continual Learning of Dependent Tasks: Memory, Data-Dependent Regularization, and Data-Dependent Weights

    Apr 19, 2026Liangzu Peng, Uday Kiran Reddy Tadipatri, Ziqing Xu +2Efficient Continual LearningContinual Learning Methods

  24. HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents

    Apr 18, 2026Jinchang Zhu, Jindong Li, Cheng Zhang +2Agent MemoryEpisodic Memory

  25. Channel Attention-Guided Cross-Modal Knowledge Distillation for Referring Image Segmentation

    Apr 18, 2026Chen YangCross-Modal DistillationChain-Of-Thought Distillation

  26. Where does output diversity collapse in post-training?

    Apr 17, 2026Constantinos Karouzos, Xingwei Tan, Nikolaos AletrasDiversityLarge Language Model Post-Training

  27. Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information

    Apr 17, 2026Yao Chen, Jiawei Sheng, Wenyuan Zhang +1Chain-Of-Thought DistillationSmall Models

  28. Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation

    Apr 16, 2026Jacob Dang, Brian Y. Xie, Omar G. YounisChain-Of-Thought DistillationImplicit Bias

  29. DLink: Distilling Layer-wise and Dominant Knowledge from EEG Foundation Models

    Apr 16, 2026Jingyuan Wang, Zhihao Jia, Chenyu Liu +7Electroencephalography Foundation ModelsKnowledge Distillation

  30. Switch-KD: Visual-Switch Knowledge Distillation for Vision-Language Models

    Apr 16, 2026Haoyi Sun, Xiaoxiao Wang, Ning Mao +5Vision-LanguageKnowledge Distillation

  31. TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation

    Apr 16, 2026Xiangyu Liu, Feng Gao, Xiaomei Zhang +4Joint Audio-Video GenerationPre-Trained Video Diffusion Models

  32. Co-distilled attention guided masked image modeling with noisy teacher for self-supervised learning on medical images

    Apr 16, 2026Jue Jiang, Aneesh Rangnekar, Harini VeeraraghavanSwinSelf-Supervised

  33. Automated Batch Distillation Process Simulation for a Large Hybrid Dataset for Deep Anomaly Detection

    Apr 10, 2026Jennifer Werner, Justus Arweiler, Indra Jungjohann +4Anomaly DetectionAnomaly

  34. FD2^2: A Dedicated Framework for Fine-Grained Dataset Distillation

    Mar 26, 2026Hongxu Ma, Guang Li, Shijie Wang +5Chain-Of-Thought DistillationFine-Grained Perception

  35. Towards Video Anomaly Detection from Event Streams: A Baseline and Benchmark Datasets

    Mar 26, 2026Peng Wu, Yuting Yan, Guansong Pang +4Video Anomaly UnderstandingTemporal Deep Learning Framework

  36. Domain Mixture Design via Log-Likelihood Differences for Aligning Language Models with a Target Model

    Mar 17, 2026Ryo Kishino, Riku Shiomi, Hiroaki Yamagiwa +2Large Language Model AlignmentKnowledge Distillation

  37. WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Models

    Mar 17, 2026Hanna Lee, Tan Dat Nguyen, Jaehoon Kang +1Sliding-Window AttentionKnowledge Distillation

  38. KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models

    Mar 2, 2026Songming Zhang, Xue Zhang, Tong Zhang +3Knowledge DistillationPytorch

  39. Enhancing Automatic Chord Recognition via Pseudo-Labeling and Knowledge Distillation

    Feb 23, 2026Nghia Phan, Rong Jin, Gang Liu +1Pseudo-LabelingModel Pretraining

  40. Is Knowledge Distillation Actually Greener? A Case Study in Machine Translation

    Feb 10, 2026Joseph Attieh, Timothee Mickus, Anne-Laure Ligozat +2Knowledge DistillationMachine Translation

  41. BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking

    Feb 5, 2026Bowen Yu, Sheng Zhang, Binhao Wang +8Chain-Of-Thought DistillationCurriculum

  42. Edit Knowledge, Not Just Facts via Multi-Step Reasoning over Background Stories

    Feb 2, 2026Ya Gao, Kalle Kujanpää, Pekka Marttinen +2Multi-Hop ReasoningKnowledge Distillation

  43. Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs

    Jan 30, 2026Yanlong Chen, Amirhossein Habibian, Luca Benini +1Large Vision Language ModelsLarge Language Model Quantization

  44. SGD-Based Knowledge Distillation with Bayesian Teachers: Theory and Guidelines

    Jan 4, 2026Itai Morad, Nir Shlezinger, Yonina C. EldarKnowledge DistillationStochastic Gradient Descent

  45. Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation

    Dec 24, 2025Wei-Rui Chen, Vignesh Kothapalli, Ata Fatahibaarzi +5Knowledge DistillationLarge Reasoning Models

  46. LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection

    Dec 5, 2025Johannes Meier, Jonathan Michel, Oussema Dhaouadi +73D Object Detection3D Object Understanding

  47. Distribution Matching Distillation Meets Reinforcement Learning

    Nov 17, 2025Dengyang Jiang, Dongyang Liu, Zanyi Wang +12Discrete Diffusion ModelOntology-Amplified Distillation

  48. Vivid-VR: Distilling Concepts from Text-to-Video Diffusion Transformer for Photorealistic Video Restoration

    Aug 20, 2025Haoran Bai, Xiaoxu Chen, Canqian Yang +3Pre-Trained Video Diffusion ModelsPhotorealistic Video Restoration

  49. Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillation

    Jul 24, 2025Lakshmana Sri Harsha Nemani, P. K. Srijith, Tomasz KuśmierczykLarge Language Model UncertaintyKnowledge Distillation

  50. A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone

    May 19, 2025Jitai Hao, Qiang Huang, Hao Liu +3Knowledge DistillationSmall Language Models

  51. How to Backdoor Image Knowledge Distillation

    Apr 30, 2025Qian Ma, Chen Wu, Prasenjit Mitra +1Knowledge DistillationBackdoor Attacks

  52. Information-Geometric Inverse Distillation for Enhancing Adversarial Transferability

    Feb 24, 2025Wenyuan Wu, Yuan Sun, Yingke Chen +4Adversarial TransferabilityChain-Of-Thought Distillation

  53. Towards Consistent and Efficient Dataset Distillation via Diffusion-Driven Selection

    Dec 13, 2024Xinhao Zhong, Shuoyang Sun, Zhaoyang Xu +4Chain-Of-Thought Distillation

  54. Tuned Reverse Distillation: Enhancing Multimodal Industrial Anomaly Detection with Crossmodal Tuners

    Dec 12, 2024Xinyue Liu, Jianyuan Wang, Biao Leng +1Zero-Shot Anomaly DetectionCross-Modal Distillation

  55. Collaborative Knowledge Distillation via a Learning-by-Education Node Community

    Sep 30, 2024Anestis Kaimakamidis, Ioannis Mademlis, Ioannis PitasKnowledge DistillationChain-Of-Thought Distillation

  56. Relational Representation Distillation

    Jul 16, 2024Nikos Giakoumoglou, Tania StathakiKnowledge DistillationInformation Bottleneck

  57. Decoupled Alignment for Robust Plug-and-Play Adaptation

    Jun 3, 2024Haozheng Luo, Jiahao Yu, Wenxin Zhang +9Large Language Model AlignmentLarge Language Model Adaptation

  58. Optimizing Three Critical Factors for Practical and Effective OOD Detection Fine-Tuning

    Aug 2, 2023Hyunjun Choi, JaeHo Chung, Hawook JeongOut-Of-Distribution DetectionOut-Of-Distribution