Self-Supervised Knowledge Distillation

Ask about this topic

Momentum

1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 8

All topics
CardsList

  1. Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning

    Sep 29, 2026Md. Ismail Hossain, Humaira Kousar, Isidora Chara TourniOn-Policy Self-DistillationMathematical Reasoning

    North South University · KAIST · Andria Labs

  2. Biomechanical 3D Body: Self-Supervised Distillation of Biomechanical Pose from a 3D Body Foundation Model

    Aug 30, 2026R. James Cotton, J. D. Peiffer, Lucinda Williamson +2Markerless Motion CaptureHuman Mesh Recovery

    Shirley Ryan AbilityLab, Chicago, IL, USA · Biomedical Engineering, Northwestern University, Evanston, IL, USA · Biomedical Engineering, University of Illinois Chicago, Chicago, IL, USA +1

  3. SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features

    Aug 11, 2026HyeonJun Lee, Hyeonsik Jo, Jinwoo Chung +1Quantization-Aware TrainingSelf-Supervised Knowledge Distillation

    Kookmin University

  4. When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation

    Aug 4, 2026Yinuo Jiang, Yongjie Ye, Zhou Tao +4VLM DistillationLanguage Model Distillation

    Zhejiang University · ByteDance · Shanghai Artificial Intelligence Laboratory

  5. Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation

    Jul 1, 2026Jaehyun Jang, Eunseop Yoon, Hee Suk Yoon +3Cross-Modal Knowledge DistillationVLM Distillation

    Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea · University of Illinois Urbana-Champaign, Champaign, USA

  6. Beyond Dark Knowledge: Mixup-Based Distillation for Reliable Predictions

    Jun 10, 2026José Medina, Paul Honeine, Abdelaziz Bensrhair +1Data AugmentationModel Calibration

    ITS Lab, Institute of Computer Science, University of Tartu, Estonia. · LITIS, Universit´e de Rouen, France. · LITIS, INSA de Rouen, France.

  7. Forgetting Has Neighbors: Localized Collateral Forgetting in Machine Unlearning

    May 29, 2026Polina Dolgova, Sebastian U. StichSelf-Supervised Knowledge DistillationMachine Unlearning

    CISPA Helmholtz Center for Information Security · Universität des Saarlandes · Saarbrücken, Germany

  8. S-SONDO: Self-Supervised Knowledge Distillation for General Audio Foundation Models

    Apr 27, 2026Mohammed Ali El Adlouni, Aurian Quelennec, Pierre Chouteau +2Model CompressionAudio Representation Learning

    LTCI, T´el´ecom Paris, Institut Polytechnique de Paris, Palaiseau, France