Knowledge Distillation

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

9 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Period ending 2026-09-14

8 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Period ending 2026-09-07

13 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Inside this field

Focused directions

418 papers

Latest in Knowledge Distillation

Browse all topics →
CardsList

  1. HilDA: Hierarchical Distillation with Diffusion for Advancing Self-Supervised LiDAR Pre-training

    Jun 18, 2026Maciej Wozniak, Jesper Ericsson, Hariprasath Govindarajan +4LidarCross-Modal Distillation

  2. SAM3 Self-Distillation for Fine-Grained GOOSE 2D Semantic Segmentation

    Jun 18, 2026Xuesong WangDense Semantic SegmentationSegment Anything Model

  3. Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation

    Jun 17, 2026Siyi Gu, Jialin Chen, Sophia Zhou +2Rubric-Based Reinforcement LearningUnsupervised On-Policy Self-Distillation

  4. Skill-Guided Continuation Distillation for GUI Agents

    Jun 17, 2026Zhimin Fan, Hongwei Yu, Yeqing Shen +9Unsupervised Skill DiscoveryGraphical User Interface Agents

  5. S-JEPA : Soft Clustering Anchors for Self-Supervised Speech Representation Learning

    Jun 17, 2026Georgios Ioannides, Adrian Kieback, Judah Goldfeder +5Sphere-JepaRepresentation Learning

  6. Learning from Own Solutions: Self-Conditioned Credit Assignment for Reinforcement Learning with Verifiable Rewards

    Jun 17, 2026Yingyu Shan, Yuhang Guo, Zihao Cheng +7Reinforcement Learning With Verifiable RewardCredit Assignment

  7. CoreMem: Riemannian Retrieval and Fisher-Guided Distillation for Long-Term Memory in Dialogue Agents

    Jun 16, 2026Jiaqi Chen, Yongqin Zeng, Shaoshen Chen +4Explicit MemoryLong-Term Memory

  8. Instance-Aware Knowledge Distillation for Semi-Supervised Learning of an On-Board Multi-Task Dense Prediction Model for Collision Avoidance System

    Jun 15, 2026Gyutae Hwang, Sang Jun LeeAutonomous Driving PerceptionDense Prediction

  9. teasr: training-efficient any-step diffusion transformer for real-world image super-resolution

    Jun 15, 2026Xiang Gao, Chenxin Zhu, Yushun Fang +2Generative Super-ResolutionMultimodal Diffusion Transformers

  10. Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents

    Jun 14, 2026Wasi Uddin Ahmad, Nikolai Ludwig, Somshubra Majumdar +1Software EngineeringSoftware

  11. VL2Spike: Spike-driven Distillation from VLMs for Low-Power Visual Perception in Embodied AI

    Jun 14, 2026Zinan Liu, Eric Zheng, Soumyaratna Debnath +3Spiking Neural NetworksBinary Spikes

  12. SPARK: Spatial Policy-driven Adaptive Reinforcement learning for Knowledge distillation

    Jun 13, 2026Mohamed Jismy Aashik Rasool, Shabir Ahmad, Gisong Oh +1Knowledge DistillationSelf-Supervised Reconstruction Method

  13. Improved Knowledge Distillation for Land-Use Image Classification

    Jun 12, 2026Arundhuti Sur, Abhiroop Chatterjee, Susmita Ghosh +1Knowledge DistillationNormalized Difference Vegetation Index

  14. HumP-KD: A Hybrid Uncertainty-Aware Multi-Stage Progressive Knowledge Distillation Framework for Efficient Fire Classification

    Jun 12, 2026Mohammed Arif Mainuddin, Najifa Tabassum, Omar Ibne Shahid +1Knowledge DistillationReal-Time Inference

  15. Visual Place Recognition in Forests with Depth-Aware Distillation

    Jun 11, 2026Walter Nedov, Saimunur Rahman, Kavindie Katuwandeniya +3Visual Place RecognitionRobust Perception

  16. Keep Policy Gradient in Charge: Sibling-Guided Credit Distillation for Long-Horizon Tool-Use Agents

    Jun 10, 2026Tianyu Ding, Jianhong Xin, Juan Pablo De la Cruz WeinsteinCredit AssignmentPolicy Gradient

  17. High-Fidelity Two-Step Image Generation via Teacher-Aligned End-to-End Distillation

    Jun 10, 2026Dongyang Liu, Ruoyi Du, David Liu +7Image GenerationGenerative Adversarial Network

  18. Doc-to-Atom: Learning to Compile and Compose Memory Atoms

    Jun 10, 2026Xingjian Diao, Wenbo Li, Yashas Malur Saidutta +3Chain-Of-Thought DistillationFactual Recall

  19. Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers

    Jun 10, 2026MohammadHossein Rezaei, Anas Mahmoud, Zihao Wang +6RubricsReinforcement Learning With Verifiable Reward

  20. Beyond Dark Knowledge: Mixup-Based Distillation for Reliable Predictions

    Jun 10, 2026José Medina, Paul Honeine, Abdelaziz Bensrhair +1Knowledge DistillationDiscriminative

  21. MODF-SIR: A Multi-agent Omni-modal Distilled Framework for Social Intelligence Reasoning

    Jun 10, 2026Shang Ma, Jisheng Dang, Wencan Zhang +6Multi-Modal Large Language ModelsCollective Intelligence

  22. Closing the Social-Semantic Gap: SPSD for Edge-Based Prompt Compression in Cloud LLM Inference

    Jun 10, 2026Abhinit Sen, Ajeet Kumar, Manaranjan PradhanLLM Inference EfficiencyLarge Language

  23. When to Trust, How to Distill: Multi-Foundation Model Guidance for Lightweight, Robust Scientific Time Series Forecasting

    Jun 10, 2026Rupasree Dey, Abdul Matin, Nathan Orwick +3Time Series Foundation ModelsWireless Foundation Models

  24. Physics-Distilled Neural Network enabled by Large Language Models for Manufacturing Process-Property Predictive Modeling

    Jun 10, 2026Ge Song, Kiarash Naghavi Khanghah, Anandkumar Patel +2Physics-Informed Machine LearningPredictive Maintenance

  25. FreqKD: Frequency-Decoupled Cross-Modal Knowledge Distillation for Infrared Object Detection

    Jun 10, 2026Keval Thaker, Venkatraman Narayanan, Abdalmalek Aburaddaha +1Cross-Modal DistillationInfrared

  26. The Role of Feedback Alignment in Self-Distillation

    Jun 9, 2026Semih Kara, Oğuzhan ErsoyBatch DistillationSelf-Training

  27. AuRA: Internalizing Audio Understanding into LLMs as LoRA

    Jun 9, 2026Bo Cheng, Lei Shi, Zhanyu Ma +5Speech Large Language ModelsSpeech Encoder

  28. PADD: Path-Aligned Decompression Distillation for Non-Router Teacher to Guide MoE Student Learning

    Jun 9, 2026Xinyue Peng, Yi Qian, Jiaojiao Lin +2Ontology-Amplified DistillationTeacher

  29. FlashTTS: Fast Streaming TTS with MTP Acceleration and X-pred Mean Flow Distillation

    Jun 8, 2026Hanke Xie, Xiaming Ren, Dake Guo +10Zero-Shot Text-To-SpeechFlash

  30. Heterophily-Aware Adaptive Knowledge Distillation for Hypergraph Neural Networks

    Jun 8, 2026Joohee Cho, David Yoon Suk Kang, Yunyong KoHypergraphsKnowledge Distillation

  31. When Video Misreads: Closed-Loop Distillation of Reading Heuristics for Exploratory Manipulation Trace QA

    Jun 7, 2026Haizhou Ge, Yufei Jia, Yue Li +5Perception-Action LoopRobot Systems

  32. ZAS-SQL: Distilling Rules from Failures for Zero-Shot Text-to-SQL

    Jun 6, 2026Hongzhou Zheng, Yixin Gou, Wenjia ZhangText-To-SqlSql

  33. Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation

    Jun 5, 2026Mohammadreza Sadeghi, Sareh Soleimani, Zihan Wang +1Efficient Continual LearningImage Clustering

  34. LRMIL: Efficient Low-Resolution Multiple Instance Learning via High-Resolution Knowledge Distillation for Whole Slide Image Classification

    Jun 5, 2026Yonghan Shin, Won-Ki JeongWhole-Slide ImagesMultiple Instance Learning

  35. USAD 2.0: Scaling Representation Distillation for Universal Audio Understanding

    Jun 4, 2026Heng-Jui Chang, Alexander H. Liu, Saurabhchand Bhati +4Large Audio Language ModelsAudio

  36. Knowledge Distillation for Visual Autoregressive Models

    Jun 4, 2026Elia Peruzzo, Aritra Bhowmik, Guillaume Sautiere +2Autoregressive Image GenerationVisual Autoregressive Models

  37. EGTR-Review: Efficient Evidence-Grounded Scientific Peer Review Generation via Multi-Agent Teacher Distillation

    Jun 4, 2026Xinpeng Qiu, Wang Yihu, Zhifeng Liu +2Peer ReviewReview

  38. Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation

    Jun 4, 2026Maxime Griot, Paul Steven Scotti, Tanishq Mathew AbrahamKnowledge DistillationReasoning Traces

  39. Inverting the Streaming-Diffusion Bottleneck: Video-Rate MLLM-Conditioned Edit Diffusion on a Consumer GPU

    Jun 4, 2026Yoshiyuki OotaniText-To-ImageProgressive Distillation

  40. ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation

    Jun 4, 2026Kanghui Tian, Siyuan Liu, Ziang Yan +3On-Policy DistillationMulti-Teacher Distillation

  41. Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation

    Jun 4, 2026Fangbo Tu, Junhua Zhao, Chi Liu +4Large Language Model QuantizationChain-Of-Thought Distillation

  42. DuDi: Dual-Signal Distillation with Cross-Lingual Verbalizer

    Jun 3, 2026Patomporn Payoungkhamdee, Tinnakit Udsa, Jian Gang Ngui +3Small Language ModelsChain-Of-Thought Distillation

  43. Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

    Jun 2, 2026Devleena Das, Rajeev Patwari, Elliott Delaye +1Large Language Model QuantizationLLM Inference Efficiency

  44. AAD-1: Asymmetric Adversarial Distillation for One-Step Autoregressive Video Generation

    Jun 2, 2026Haobo Li, Yanhong Zeng, Yunhong Lu +6Autoregressive Video GenerationChain-Of-Thought Distillation

  45. Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation

    Jun 2, 2026Tianhe Wu, Zikai Zhou, Kun Yan +23Prompt DistillationImage Editing

  46. PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization

    Jun 2, 2026Ying Tang, Dong Li, Youjia Zhang +3Vision Foundation ModelsMixture-of-Experts Models

  47. What Do Students Learn? A Feature-Level Analysis of Dark Knowledge

    Jun 2, 2026Seungu Kang, Songkuk KimKnowledge DistillationFeature Learning

  48. Pathway-Structured Privileged Distillation for Deployable Computational Pathology

    Jun 1, 2026Yongxin Guo, Hao Lu, Onur Koyun +2Computational PathologySpatial Transcriptomics

  49. Cross-Domain Dead Tree Detection via Knowledge Distillation in Aerial Imagery

    Jun 1, 2026Anis Ur Rahman, Mete Ahishali, Einari Heinaro +1Forest InventoryAerial Imagery

  50. FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment

    Jun 1, 2026Nazmus Shakib Shadin, Aaron Cummings, Xinyue Zhang +1Heterogeneous Federated LearningFederated Learning

  51. What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression

    May 31, 2026Wendao Wu, Fangqing Zhang, Haihan Zhang +1Knowledge DistillationKernel Ridge Regression

  52. ThinkSwitch: Context Distillation with LoRA and Weight Interpolation for Specific-Purpose Reasoning Tasks

    May 31, 2026Dhruv Saini, Rohan PandeyChain-Of-Thought DistillationInference-Time

  53. Subliminal Learning Is Steering Vector Distillation

    May 31, 2026Camila Blank, Agam Bhatia, Senthooran Rajamanoharan +2Ontology-Amplified DistillationTeacher

  54. Logit Distillation on Manifolds: Mapping by Learning

    May 30, 2026Yiru Yang, Junling Wang, Nishant Kumar Singh +2Model TrainingProgressive Distillation

  55. ETC: Extreme Token Compression via Task-aware Visual Information Distillation in VLMs

    May 30, 2026Yiling Gao, Hongchen Wei, Zhenzhong ChenToken CompressionVisual Tokens

  56. Student Capacity Moderates Knowledge Distillation Effectiveness: A Systematic Study Across ResNet Teacher-Student Pairs on CIFAR-10

    May 29, 2026Umut Onur YasarKnowledge DistillationResidual Connections

  57. Distilling LLM Feedback for Lean Theorem Proving

    May 29, 2026Gaetan Narozniak, Gérard Biau, Rémi Munos +2Token-Level SupervisionGroup Relative Policy Optimization

  58. RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting

    May 29, 2026Yuduo Li, Xiaofeng Shi, Qian Kou +2Supervised Fine-TuningRefinement

  59. SGMD: Score Gradient Matching Distillation for Few-Step Video Diffusion Distillation

    May 28, 2026Zhuguanyu Wu, Ruihao Gong, Yang Yong +5Pre-Trained Video Diffusion ModelsChain-Of-Thought Distillation

  60. EVL-ECG: Efficient ECG Interpretation With Multi-Aspect Heterogeneous Knowledge Distillation

    May 28, 2026Dang Nguyen Hong, Nhi Ngoc-Yen Nguyen, Huy-Hieu PhamPediatric-Adult Electrocardiogram AlignmentElectronic Health Record Foundation Models