Knowledge Distillation

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

9 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Period ending 2026-09-14

8 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Period ending 2026-09-07

13 new papers

A weekly snapshot of new work published in Knowledge Distillation.

Inside this field

Focused directions

418 papers

Latest in Knowledge Distillation

Browse all topics →
CardsList

  1. SQuaD-SQL: Efficient Text-to-SQL with Small Language Models via LLM-Guided Knowledge Distillation

    Jul 9, 2026Wangyu Wu, Xiaojian Lin, Rong Fu +4Text-To-SqlSmall Language Models

  2. Future Confidence Distillation in Large Language Models

    Jul 8, 2026Sahil KaleConfidence CalibrationChain-Of-Thought Distillation

  3. Enhancing deep learning models for time series classification via knowledge distillation

    Jul 7, 2026Javidan Abdullayev, Maxime Devanne, Jonathan Weber +1Knowledge DistillationTime-Series Classification

  4. Few-Medoids: An Embarrassingly Simple Coreset Selection Method for Few-Shot Knowledge Distillation

    Jul 7, 2026Cemil-Andrei Dilmac, Florinel-Alin Croitoru, Radu Tudor IonescuGeometry-Aware Uncertainty CoresetsKnowledge Distillation

  5. ArtisanCAD: An Industrial-Level CAD Agent with Expert-Grounded Knowledge Distillation

    Jul 7, 2026Yunhan Xu, Qifeng Wu, Xunjin Li +10Text-To-CadComputer-Aided Design

  6. Multi-Teacher Contrastive Distillation for Edge-Efficient Pathology Foundation Models

    Jul 6, 2026Tim Lenz, Maurice Heide, Marco Gustav +2Pathology Foundation ModelsWhole-Slide Images

  7. Context-Constrained Transfer Learning for Tabular Foundation Models via Data Distillation

    Jul 6, 2026Yijun Lin, Sai LiTabular Foundation ModelsTransfer Learning

  8. SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation

    Jul 5, 2026Omer Tariq, Syed Muhammad Raza, Jeongbae SonKnowledge DistillationOntology-Amplified Distillation

  9. Hierarchical Multi-to-Single-Modal Knowledge Distillation for Disruption Prediction in EAST

    Jul 5, 2026Qiang Chen, Xiao Wang, Hao Si +9Cross-Modal DistillationPlasma Shape Control

  10. CritiqueDriveVLM: From Verifier-Guided Reinforcement Learning to Latent Thought Distillation for Autonomous Driving

    Jul 5, 2026Zhaohong Liu, Hao Ye, Xianlin Zhang +1Autonomous DrivingEfficient Latent Reasoning

  11. DistillH-Mamba: A Hypergraph-Mamba-Based Knowledge Distillation Model for Efficient Impact Fall Detection

    Jul 3, 2026Tresor Y. Koffi, Youssef Mourchid, Mohammed Hindawi +1Robust Fall DetectionBidirectional Mamba

  12. Exp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations

    Jul 3, 2026Van Huyen Dang, Kabilesh Rajendran, Erdi Sayar +1Vision-Language-Action ModelsAutonomous Drone Navigation

  13. SFKD: Spatial--Frequency Joint-Aware Heterogeneous Knowledge Distillation via Multi-Level Wavelet Spectral Interaction

    Jul 2, 2026Cuipeng Wang, Haipeng WangKnowledge DistillationDiscrete Wavelet Transform

  14. Geometric Foundation Model Distillation for Efficient Lunar 3D Reconstruction

    Jul 2, 2026Clémentine Grethen, Florient Chouteau, Géraldine Morin +13D Foundation Models3D Reconstruction

  15. C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillation

    Jul 2, 2026Jinlong Wang, Xun Huang, Qiming Xia +23D Object DetectionCollaborative Perception

  16. Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation

    Jul 1, 2026Shayan Talaei, Abhinav Chinta, Devvrit Khatri +3Large Language Model BiasMachine-Generated Text Detection

  17. Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation

    Jul 1, 2026Jaehyun Jang, Eunseop Yoon, Hee Suk Yoon +3Knowledge DistillationMulti-Modal Large Language Models

  18. OnPoint: Offline-to-Online Multi-Level Distillation for Point-Supervised Online Temporal Action Localization

    Jul 1, 2026Sakib Reza, Gauri Jagatap, Mohsen Moghaddam +2Temporal Action LocalizationPrompt Distillation

  19. TallyTrain: Communication-Efficient Federated Distillation

    Jun 30, 2026Radhakrishna Achanta, Will ReedFederated LearningChain-Of-Thought Distillation

  20. Benchmarking Federated Learning and Knowledge Distillation for Point Cloud Classification

    Jun 30, 2026Aizierjiang AiersilanFederated LearningCertified Point-Cloud

  21. Improving Certified Robustness via Adversarial Distillation

    Jun 30, 2026Matteo Melis, Jesus Martinez Del Rincon, Vishal SharmaCertificationChain-Of-Thought Distillation

  22. Holonic Active Distillation for Scalable Multi-Agent Learning in Multi-Sensor Systems

    Jun 30, 2026Dani Manjah, Tim Bary, Benoît Macq +1Progressive Distillation

  23. Knowledge Distillation from Large Reasoning Models to Compact Student Models: A Case Study on the John O Bryan Mathematics Competition

    Jun 30, 2026Gaurab Baral, Aaditya Khanal, Yangyang Tao +1Knowledge DistillationLarge Reasoning Models

  24. Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation

    Jun 29, 2026Javier Lazaro, Juan-Ignacio Vazquez, Pablo Garcia-BringasVariational Quantum CircuitsKnowledge Distillation

  25. Diffusion Fine-tuning with Rewarded Moment Matching Distillation

    Jun 29, 2026Alexis Jacq, Guillaume Couairon, Valentin De Bortoli +3Prompt Distillation

  26. Building Multi-Task Agentic LLMs via Two-Phase Distillation

    Jun 29, 2026Huaijie Wang, Shusheng Xu, Yi Wu +1On-Policy DistillationAgentic Reinforcement-Learning Framework

  27. Variance Reduction on the Camera Axis: Multi-View Score Distillation for 3D

    Jun 29, 2026Marian Lupascu, Mihai Sorin Stupariu, Ionut MironicaMulti-View Diffusion ModelScore-Based Diffusion Model

  28. ARKD: Adaptive Reinforcement Learning-Guided Bidirectional KL Divergence Distillation for Text Generation

    Jun 29, 2026Zilong Liu, Xuewen Zhang, Jinrui Xing +3Knowledge DistillationBatch Distillation

  29. KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search

    Jun 29, 2026Tao Feng, Xinke Jiang, Chao WuAgentic Reinforcement-Learning FrameworkAgentic Reasoning

  30. Beyond Trajectory Matching: Reflow with Marginal Distribution Alignment

    Jun 28, 2026Chen Wang, Peiran Yun, Pan Xie +1Progressive DistillationGenerative Models

  31. DistilledGemma: Balanced Efficiency-Accuracy for Person-Place Relation Extraction from Multilingual Historical Articles

    Jun 28, 2026Youssef Aboelwafa, Ahmed Samir, Nagwa Elmakky +1Relation ExtractionKnowledge Distillation

  32. Labeling Training Data for Entity Matching Using Large Language Models

    Jun 27, 2026Aaron Steiner, Christian BizerNatural Language DatasetsTraining Data

  33. ATOD: Annealed Turn-aware On-policy Distillation for Multi-turn Autonomous Agents

    Jun 26, 2026Qitai Tan, Zefang Zong, Yang Li +1On-Policy DistillationOntology-Amplified Distillation

  34. Optimizing Teacher-Student Partitioning for Scalable Knowledge Distillation on HPC Systems

    Jun 26, 2026Adrian P. Dieguez, Victor Conchello Vendrell, Alex Batlle +3Knowledge DistillationPartitioning

  35. Two-Stage Cross-Domain Cervical Abnormality Screening with Cytopathological Image Synthesis and Knowledge Distillation

    Jun 26, 2026Jincheng Li, Yuzhi He, Yihui Zhan +6Cross-Domain ShiftsCross-Modality

  36. Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes

    Jun 25, 2026Jeremias Ferrao, Niclas Müller-Hof, Iustin Sîrbu +2Large Language Model SafetyInstruction-Tuned Models

  37. OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning

    Jun 25, 2026Shuo Yang, Jinyang Wu, Zhengxi Lu +8On-Policy DistillationAgentic Reinforcement-Learning Framework

  38. From Sparse and Imperfect 2D Anchors to Consistent 3D Gaussian Street Scenes: Support-Aware Appearance

    Jun 24, 2026Long Cao, Zhongquan Wang, Jie Li +4Feed-Forward 3D Gaussian ReconstructionAppearance

  39. Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation

    Jun 24, 2026Luyang Fang, Haoran Lu, Yongkai Chen +2Knowledge DistillationNonparametric

  40. BitNet Text Embeddings

    Jun 24, 2026Zhen Li, Xin Huang, Liang Wang +8Domain-Invariant Text EmbeddingsEmbedder

  41. Distill on a Diet: Efficient Knowledge Distillation via Learnable Data Pruning

    Jun 24, 2026Yifan Wu, Yiqi Wang, Xichen Ye +5Knowledge DistillationPruning

  42. Efficient Remote Sensing Instance Segmentation with Linear-Time State Space Distilled Visual Foundation Models

    Jun 24, 2026Qinzhe Yang, Keyan Chen, Jia Xu +2Remote Sensing Image SegmentationRemote Sensing

  43. What Does It Mean to Break a Distillation Defense?

    Jun 23, 2026Lena Libon, Pura Peetathawatchai, Michael Aerni +2Threat ModelsBatch Distillation

  44. Heterogeneous Knowledge Distillation via Geometry Decoupling and Momentum-Aware Gradient Regulation

    Jun 23, 2026Wuming Yang, Xiang Zhang, Hongmin ZhaoKnowledge DistillationExponential Moving Average

  45. ARIA: Adaptive Region-Based Importance Allocation for Conditional Diffusion Distillation

    Jun 22, 2026Loay Mualem, Vinh Tong, Samir Darouich +1Knowledge DistillationDiscrete Diffusion Model

  46. The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models

    Jun 22, 2026Ahmad Algadhi, Ahmed Alzuhair, Omar Alkhulaif +1Multi-Teacher DistillationPrompt Distillation

  47. The Degeneracy Distillery

    Jun 22, 2026T. Lucas Makinen, Deaglan J. Bartlett, Niall Jeffrey +1Asymmetric Distillation

  48. PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation

    Jun 22, 2026Jiaqiang Wu, Zhouan Zhu, Shangfei WangEmpathyKnowledge Distillation

  49. TaLK: Text-attributed Graph Dataset Distillation via Coupling Language Model with Graph-Aware Kernel

    Jun 22, 2026Yeongho Kim, Yeonje Choi, Kijung ShinText-Attributed GraphGraph Neural Networks

  50. Understanding Knowledge Distillation in Post-Training: When It Helps and When It Fails

    Jun 22, 2026Xin Liu, Simin Ma, Shujian Liu +5Knowledge DistillationLarge Language Model Post-Training

  51. Context-Aware Distillation and Ablation for Text2DSL

    Jun 21, 2026Alexander V. Kozachok, Alexander M. Nazimov, Shamil G. MagomedovDomain-Specific LanguageChain-Of-Thought Distillation

  52. Channel Location Constrains the Auditability of Subliminal Learning

    Jun 20, 2026Tamas MadlChain-Of-Thought DistillationChannel

  53. Streaming T5-based Text-to-Speech Synthesis with Limited Lookahead

    Jun 20, 2026Muyang Du, Jason Roche, Junjie LaiZero-Shot Text-To-SpeechCausal Attention

  54. TALAS: Teacher-Anchored Layer Alignment with Adaptive Sharpness-Aware Minimization for Embedding Distillation

    Jun 20, 2026Quoc Phong Dao, Hoang Son Nguyen, Pham Khanh Chi +4Knowledge DistillationLatent Representation Alignment

  55. Structural Assessment for Understanding and Guiding Dataset Distillation in Discrete Token Space

    Jun 19, 2026Yue Cao, Jianyang Gu, Vyacheslav Kungurtsev +4Chain-Of-Thought DistillationToken-Level Uncertainty

  56. When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation

    Jun 19, 2026Siyang Lyu, Xinghao Chen, Zhijing Sun +3Chain-Of-Thought DistillationContext Compression

  57. Speaker Identity in Non-Verbal Vocalizations: Conditional Distillation and Mixture of Experts Approach

    Jun 19, 2026Tzu-Chieh Wei, Yi-Cheng Lin, Huang-Cheng Chou +4Speaker Verification PerformanceSpeaker Identity

  58. Reference-Based Distillation Detection in LLMs

    Jun 19, 2026Rajat Rawat, Sizhe Chen, Akshay Anand +3Ontology-Amplified DistillationDeepseek

  59. UNIEGO: Proxies as Mediators for Unified Egocentric Video Representation Learning

    Jun 18, 2026Wenhao Chi, Arkaprava Sinha, Dominick Reilly +2Egocentric VideoFine-Grained Video Understanding

  60. Marginal Advantage Accumulation for Memory-Driven Agent Self-Evolution

    Jun 18, 2026Mingyu Yang, Keye Zheng, Congchao Cheng +4Batch DistillationOn-Policy Distillation