Knowledge Distillation

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

9 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Period ending 2026-09-14

8 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Period ending 2026-09-07

13 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Inside this field

Focused directions

418 papers

Latest in Knowledge Distillation

Browse all topics →
CardsList

  1. Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss

    Aug 4, 2026Bakbergen Ryskulov, Iker García-Ferrero, David Montero +5Knowledge DistillationSmall Language Models

  2. Lightweight 3D Object Detection via Mamba-Based Knowledge Distillation

    Aug 4, 2026Quoc Cuong Ninh, Huy Xuan Pham, Anh Tung Nguyen +13D Object DetectionLight Detection And Ranging

  3. AS-FedBridge: Pseudo-Spike Bridge Distillation for Heterogeneous ANN-SNN Federated Learning

    Aug 4, 2026Shengyang Li, Yiting Dong, Liuyang Song +5Spiking Neural NetworksHeterogeneous Federated Learning

  4. Efficient Video Dataset Distillation via Cluster-Guided Prototype Blending

    Aug 4, 2026Chongle Ren, Guang Li, Wenbo Huang +3Video DatasetChain-Of-Thought Distillation

  5. CROSS: Cascaded Distillation and Dual-Constraint Grounding for Remote Sensing Referring Segmentation

    Aug 4, 2026Tingzhang Luo, Ruizhong Liu, Yichao Liu +3Remote Sensing Image SegmentationPrecise Localization

  6. Channel-wise Dynamic Knowledge Distillation via Adaptive Sample Generation for Action Recognition

    Aug 4, 2026Ping Li, Chenhao Ping, Jie Song +1Open-Vocabulary Action RecognitionKnowledge Distillation

  7. Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints

    Aug 3, 2026Zirui Huang, Yunlong Mao, Wei Tong +3Large Language Model Fine-TuningProvenance

  8. Learning What to Remember: Test-Time Training via Context Distillation

    Aug 3, 2026Zixuan Wang, Xingyu Dang, Rui-Jie Zhu +4Efficient Long-Context InferenceEfficient Continual Learning

  9. Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning

    Aug 2, 2026Yuqi Li, Yuedong Tan, Huiran Duan +7Cross-Modal DistillationKnowledge Distillation

  10. Harnessing Adversarial Distillation to Customise Debiased, Disease-Specific Pathology Foundation Models for Breast Cancer

    Aug 2, 2026Zhiwei Chen, Yang Hu, Yuxiang Xiao +7Pathology Foundation ModelsComputational Pathology

  11. TurboClear: One-Step Object-Effect Removal via Region-Calibrated Distribution Matching and Fusion

    Aug 2, 2026Jiawei Guo, Junxian Li, Yixin Tang +4Video Object RemovalChain-Of-Thought Distillation

  12. LUT: Latent Utility Training for Visual Reasoning

    Aug 1, 2026Jiaxuan Kang, Siyu Chen, Mingda Li +6Latent Visual ReasoningVisual Reasoning

  13. Light-Loco-Parkour: Versatile Perceptive Whole-Body Locomotion via Multi-Skill Distillation

    Aug 1, 2026Hongming Chen, Zhuoran Li, Hongxi Wang +9Parkour LocomotionTask-Centric Hybrid Whole-Body Controller

  14. Progressive2^2: A Teacher-Student Progressive Co-Evolving Knowledge Distillation Method for Substantial Model Compression

    Jul 31, 2026Tiancong Cheng, Ying Zhang, Zhiwen Yu +2Knowledge DistillationContext Compression

  15. MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation

    Jul 31, 2026Hang Yan, Zhangxuan GU, Beitong Zhou +5Graphical User Interface AgentsOn-Policy Distillation

  16. Distill What RGB Can Recover: Privileged 3D Evidence for RGB-Only Vision-Language Models

    Jul 31, 2026Yanbin Hu, Jin Cui, Jun Ye +43D Scene UnderstandingRed-Green-Blue

  17. Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving

    Jul 31, 2026Zeyu Dong, Yimin Zhu, Yu Wu +1Autonomous DrivingEfficient Vision-Language-Action Models

  18. TransMem: Transforming Hidden States into Memory for Large Language Models

    Jul 31, 2026Haodong Lei, Junming Liu, Yirong Chen +4Large Language Model AgentsHidden States

  19. VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation

    Jul 30, 2026Kangning Zhang, Yixing Li, Shuai Shao +9Chain-Of-Thought DistillationVisual Evidence

  20. Contrastive Reinforced Policy Optimization via Privileged Self-Distillation

    Jul 30, 2026Xingjian Wu, Junlin Liu, Xingchen Liu +6Unsupervised On-Policy Self-DistillationOn-Policy Distillation

  21. Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers

    Jul 30, 2026Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi +1Adversarial TransferabilityMulti-Teacher Distillation

  22. Shared Semantic Codebook Distillation for Unpaired Cross-Modal Medical Classification

    Jul 29, 2026Dillan Imans, Phuoc-Nguyen Bui, Duc-Tai Le +1Cross-Modal DistillationCross-Modality

  23. CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation

    Jul 29, 2026Fengming Yu, Haiwei Pan, Kejia Zhang +3Knowledge DistillationTeacher-Student Log-Probability Gap

  24. Where Detectors Fail: Closing the Tail-Domain Gap with Expert-Guided Mutual Distillation

    Jul 29, 2026Xuan Feng, Guihong Liu, Tianlong Gu +5Fake News DetectionCross-Modal Learning

  25. Parallel Decoding Distillation for Fast Image and Video Generation

    Jul 28, 2026Neta Shaul, Chao Liu, Arash Vahdat +1Few-Step GenerationChain-Of-Thought Distillation

  26. CoTinyVLA: Chain-of-Thought Distillation for a Sub-Billion-Parameter Vision-Language-Action Model

    Jul 28, 2026Minhyeok Lee, Chiyoung Kim, Chanhoe Gu +5Chain-Of-Thought DistillationToken Budgets

  27. Dataset Distillation Based on Saliency-Driven Prototype Alignment

    Jul 28, 2026Yawen Zou, Wenqi Cai, Guang Li +3Chain-Of-Thought DistillationLarge Image Datasets

  28. AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognition

    Jul 28, 2026Yuqi Li, Yi-Cheng Lin, Xianglong Wang +5Multi-Teacher DistillationEmotion Recognition

  29. TabRank: Chain-of-Thought Distillation for Table Re-Rankers

    Jul 28, 2026Adarsh Singh, Kushal Raj Bhandari, Jianxi Gao +2RerankingTabular Data

  30. Amortising Trajectory Optimisation for Residual MPC via Implicit Contact Differentiation

    Jul 27, 2026Daniel Layeghi, Thomas Corbères, Calum Arnott +4Trajectory OptimizationContact Forces

  31. From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search

    Jul 27, 2026Junlin Liu, Jiangwang Chen, Zixin Song +7Knowledge DistillationAgentic Reinforcement-Learning Framework

  32. SPRKD: Effective Knowledge Distillation for Deep Neural Networks via Saddle Region Approximation

    Jul 25, 2026Aditya Dewan, Arjun Yogeswaran, Benjamin FedorukKnowledge DistillationCondition Number

  33. Context-Aware Concept Distillation for Trustworthy Flood Prediction

    Jul 25, 2026Eli Levinkopf, Efrat Morin, Claudia V. GoldmanAccurate Flood PredictionSen1Floods11

  34. Generative Video Compression with Adaptive Score Distillation

    Jul 24, 2026Naifu Xue, Zhaoyang Jia, Haosen Li +7Generative Image CompressionChain-Of-Thought Distillation

  35. Boosting Robustness for All-Weather Self-Supervised Depth Estimation in Autonomous Driving

    Jul 23, 2026Mengshi Qi, Xiaoyang Bi, Xianlin Zhang +1Vision-Radar FusionAutonomous Driving Perception

  36. When Does Knowledge Distillation Hurt? Reliability-Aware Distillation for Low-Resource Language Summarization

    Jul 22, 2026Dipto Sumit, Ankan Kumar Roy Srizon, Sadia Khair Rodela +4Knowledge DistillationSummarization

  37. REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning

    Jul 21, 2026Yunjie Chen, Xiaoxin Chen, Fang WangOffline Reinforcement LearningDeep Reinforcement Learning

  38. Learning Semantic-Robust Change Detection via Semantic-Invariant Self-Distillation

    Jul 21, 2026Jiuhe Qu, Yingping Liang, Ying FuChange DetectionBatch Distillation

  39. Privileged Lesion-Context Relational Distillation for Mask-Free Skin Lesion Classification

    Jul 21, 2026Abu Mukaddim Rahi, Md Mithun Hossain, Md Zulficar Hasan Joy +2Skin Lesion ClassificationLesion

  40. RF-Agent: A Practical Framework for Building Language Agents for RFIC Design

    Jul 21, 2026Yueqi Xing, Houbo He, Jolie Wang +5Table Reasoning DatasetsElectronic Design Automation

  41. Rationale-Guided Knowledge Distillation for Cross-Lingual Stance Detection

    Jul 21, 2026Qiuli Zhou, Jingyuan Yao, Shengeng Tang +3Tweet-Level Stance DetectionKnowledge Distillation

  42. Bridging the Information Gap: Semantic Densification and Hindsight Distillation for Cold-Start Prediction

    Jul 19, 2026Hao Duong Le, Yifei Gao, Huan Li +4Cold-StartSemantic Gap

  43. Efficient Audio-Visual Event Recognition via Knowledge Distillation and Dynamic INT8 Quantization of a Hybrid Cross-Attention Network

    Jul 18, 2026Parinaz Binandeh Dehaghani, Danilo Pena, A. Pedro AguiarMobilenetv2Self-Supervised Vision Transformers

  44. CADENCE: Closing the Reasoning Gap via Coverage-Adaptive On-Policy Distillation

    Jul 18, 2026Satyam Kumar, Saurabh JhaOn-Policy DistillationKnowledge Distillation

  45. Dataset Distillation by Influence Matching

    Jul 18, 2026Haoru Tan, Wang Wang, Sitong Wu +5Chain-Of-Thought DistillationInfluence

  46. On the Failure of Boundary-Seeking Distillation in Bottlenecked Generative Architectures

    Jul 17, 2026Mohamed Amine KinaKnowledge DistillationGenerative Adversarial Network

  47. Optimal Self-Distillation for Rectified Flow via Linear Probing

    Jul 16, 2026Saptarshi Roy, Debepsita Mukherjee, Pratik PatilBatch DistillationRectified Flow

  48. Gold-Guided Programmatic Distillation for Financial Reasoning over Hybrid Tables and Text

    Jul 16, 2026Yun Dong, Erica Zhao, Elana ChenRationalesOntology-Amplified Distillation

  49. Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models

    Jul 16, 2026Mingxi Fu, Jiawen Li, Renao Yan +4Multiple Instance LearningWhole-Slide Images

  50. Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection

    Jul 15, 2026Mingyue Zeng, De Cheng, Zhipeng Xu +3Out-Of-Distribution DetectionKnowledge Distillation

  51. Domain-Incremental Remote Sensing Change Detection via Difference-Guided Adaptation and Frequency-Decoupled Distillation

    Jul 14, 2026Daifeng Peng, Yaning Li, Haiyan GuanChange DetectionDomain Incremental Learning

  52. Single-Teacher View Augmentation: Enhancing Knowledge Distillation with Student-Guided Perturbations

    Jul 13, 2026Xuyi Yu, Yaohua Liu, Ziming Song +3Multi-Teacher DistillationKnowledge Distillation

  53. LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models

    Jul 13, 2026Gangsu Kim, Won-Ki JeongPathology Foundation ModelsMedical Vision-Language Models

  54. Answer-Conditioned Chain-of-Thought Distillation for Few-Shot Industrial Vision with Small VLMs

    Jul 12, 2026Shubham RaoLarge Vision Language ModelsChain-Of-Thought Distillation

  55. Knowledge Distillation for Automated AI Tutor Evaluation

    Jul 12, 2026Tahmid Al Hannan, Diego Garcia, Alex Njoroge +2TutorsPedagogy

  56. BucketKD: A Safety-Aware Bucket-Based Knowledge Distillation Framework for End-to-End Motion Planning

    Jul 12, 2026Md Nahidul Islam, Mohd Hasan Ali, Dipankar Dasgupta +1Autonomous DrivingObstacle Avoidance

  57. TVT-PAPD: Pathology-Aware Prototype Distillation for Self-Supervised Whole Slide Image Classification

    Jul 11, 2026Ramesh Naidu Laveti, Jaya Sreevalsan-Nair, T K SrikanthGliomaLesion Segmentation

  58. Ontology-Amplified Distillation and Contextuality Auditing for Sovereign Enterprise Language Models: A Combined Proof-of-Mechanism and Negative-Results Method Study

    Jul 11, 2026Thanh Luong TuanOntology-Amplified DistillationAlgorithm Auditing

  59. Transforming LLMs into Efficient Cross-Encoders via Knowledge Distillation for RAG Reranking

    Jul 11, 2026Shreeya Dasa Lakshminath, Shubhan SRerankingRetrieval-Augmented Generation Systems

  60. Compete Then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitation

    Jul 9, 2026Miseong Shawn KimCurriculumMulti-Teacher Distillation