Teacher-Student Learning

Ask about this topic

Momentum

7 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 162

All topics
CardsList

  1. GeoDial: A Multimodal Conversational Tutoring Dataset for Geometry Problem-Solving with Visual Tutor Turns

    May 8, 2026Sankalan Pal Chowdhury, Junling Wang, Donya Rooein +2Multimodal GroundingIntelligent Tutoring Systems

    ETH Zurich · ETH Zurich, ETH AI Center · Bocconi University

  2. Rubric-based On-policy Distillation

    May 8, 2026Junfeng Fang, Zhepei Hong, Mao Zheng +7LLM AlignmentLanguage Model Distillation

    National University of Singapore · University of Science and Technology of China · Tencent

  3. Attention Transfer Is Not Universally Effective for Vision Transformers

    May 8, 2026Huaiyuan Qin, Muli Yang, Gabriel James Goenawan +4Self-AttentionVision Transformer

    Institute for Infocomm Research (I2R), A*STAR, Singapore · 3Shanghai Jiao Tong University · 2Sichuan University

  4. Structural Rationale Distillation via Reasoning Space Compression

    May 8, 2026Jialin Yang, Jiankun Wang, Jiajun Wu +3CoT DistillationStructured Reasoning

    University of Calgary · Calgary, Canada · University of Michigan +1

  5. Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level

    May 7, 2026Nan Jia, Haojin Yang, Xing Ma +6RL for Language Model ReasoningLanguage Model Distillation

    Huazhong University of Science and Technology · Peking University · Meituan

  6. Weak-to-Strong Generalization is Nearly Inevitable (in Linear Models)

    May 7, 2026Scott Geng, Dutch Hansen, Jerry LiLogistic RegressionWeak-to-Strong Generalization

    University of Washington, Seattle.

  7. Preference-Based Self-Distillation: Beyond KL Matching via Reward Regularization

    May 6, 2026Xin Yu, Liuchen Liao, Yiwen Zhang +3On-Policy Self-DistillationLanguage Model Distillation

    1The Pennsylvania State University · 2TikTok

  8. Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe

    May 5, 2026Wenjin Hou, Shangpin Peng, Weinong Wang +13Language Model DistillationMulti-Teacher Knowledge Distillation

    1Zhejiang University · 2Shenzhen Loop Area Institute · 3LLM Department, Tencent

  9. Private Speech Classification without Collapse: Stabilized DP Training and Offline Distillation

    May 4, 2026Yadi Wen, Tianxin Li, Enji Liang +2Differentially Private Stochastic Gradient DescentLearning Using Privileged Information

    Shanxi Key Laboratory of Industrial Internet Security, Taiyuan University of Technology, China · University of Michigan, Ann Arbor, USA

  10. Distilling Long-CoT Reasoning through Collaborative Step-wise Multi-Teacher Decoding

    May 4, 2026Taewon Yun, Jisu Shin, Jeonghwan Choi +2CoT DistillationEfficient Language Model Reasoning

    KAIST · UNIST

  11. MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate

    May 2, 2026Jianze Wang, Ying Liu, Jinlong Chen +7Multi-Agent DebateMulti-Teacher Knowledge Distillation

    School of Artificial Intelligence and Automation, Huazhong University of Science and Technology · Alibaba Group

  12. LEAP: Layer-wise Exit-Aware Pretraining for Efficient Transformer Inference

    May 1, 2026Shashank Kapadia, Deep Naryan Mishra, Sujal Reddy Alugubelli +4Efficient Transformer InferenceDynamic Neural Networks

    Walmart Inc. · Sam’s Club

  13. GaitKD: A Universal Decoupled Distillation Framework for Efficient Gait Recognition

    Apr 29, 2026Yuqi Li, Qian Zhou, Huiran Duan +5Gait AnalysisGait Recognition

    The City University of New York · Wuhan University · Beijing Jiaotong University +2

  14. Subliminal Steering: Stronger Encoding of Hidden Signals

    Apr 28, 2026George Morgulis, John HewittLanguage Model SteeringSubliminal Learning

    Columbia University

  15. Sustained Gradient Alignment Mediates Subliminal Learning in a Multi-Step Setting: Evidence from MNIST Auxiliary Logit Distillation Experiment

    Apr 28, 2026Chayanon Kitkana, Shivam AroraSubliminal LearningTeacher-Student Learning

    Independent · Equivariant labs

  16. Knowledge Distillation Must Account for What It Loses

    Apr 28, 2026Wenshuo WangTeacher-Student LearningKnowledge Distillation

    School of Future Technology, South China University of Technology, China

  17. Alice v1: Distillation-Enhanced Video Generation Surpassing Closed-Source Models

    Apr 27, 2026Wang Xiaoyu, Phong Nguyen, Chen ZhaoVideo Diffusion ModelsDiffusion Model Distillation

    Mirage Team Open Source Research

  18. TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents

    Apr 27, 2026Jiaqi Wang, Wenhao Zhang, Weijie Shi +2Language Model DistillationLLM Agents

    Tongyi Lab , Alibaba Group · † The Chinese University of Hong Kong.

  19. Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Models

    Apr 25, 2026Max Hartman, Vidhata Jayaraman, Moulik Choraria +2CoT DistillationTeacher-Student Learning

    University of Illinois Urbana-Champaign · Carnegie Mellon University · Stony Brook University

  20. Industrial Surface Defect Detection via Diffusion Generation and Asymmetric Student-Teacher Network

    Apr 21, 2026Shuo Feng, Runlin Zhou, Yuyang Li +1Industrial Anomaly DetectionSynthetic Anomaly Generation

    School of Automation, Southeast University, Nanjing, China

  21. Distillation Traps and Guards: A Calibration Knob for LLM Distillability

    Apr 21, 2026Weixiao Zhan, Yongcheng Jing, Leszek Rutkowski +1Language Model CalibrationLanguage Model Distillation

    Generative AI Lab, College of Computing and Data Science Nanyang Technological University, Singapore 639798 · Systems Research Institute of the Polish Academy of Sciences, AGH University of Krakow, 30-059 Kraków, and the SAN University, 90-113, Łód´z, Poland

  22. Random Matrix Theory of Early-Stopped Gradient Flow: A Transient BBP Scenario

    Apr 20, 2026Florentin Coeurdoux, Grégoire Ferré, Jean-Philippe BouchaudRandom Matrix TheoryEarly Stopping

    1Capital Fund Management, Paris, France · 2X-CFM Chair of EconophysiX, Ecole polytechnique, Palaiseau, France · 3Académie des Sciences, Paris, France

  23. Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation

    Apr 16, 2026Jacob Dang, Brian Y. Xie, Omar G. YounisSubliminal LearningLanguage Model Distillation

    UCLA · Santa Monica College · Mila, Silverstream AI

  24. Weak-to-Strong Knowledge Distillation Accelerates Visual Learning

    Apr 16, 2026Baiang Li, Wenhao Chai, Felix HeideTeacher-Student LearningKnowledge Distillation

    Princeton University

  25. Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation

    Apr 13, 2026Lester James V. Miranda, Ivan Vulić, Anna KorhonenSupervised Fine-TuningMultilingual Language Models

    Language Technology Lab, University of Cambridge

  26. Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective

    Apr 10, 2026Tokio Kajitsuka, Ukyo Honda, Sho TakaseCoT DistillationReasoning Distillation

    The University of Tokyo, Tokyo, Japan · CyberAgent, Tokyo, Japan

  27. Effective Distillation to Hybrid xLSTM Architectures

    Mar 16, 2026Lukas Hauzenberger, Niklas Schmidinger, Thomas Schmied +7LLM CompressionLanguage Model Distillation

    1NXAI · 2Johannes Kepler University Linz

  28. TabKD: Tabular Knowledge Distillation through Interaction Diversity of Learned Feature Bins

    Mar 16, 2026Shovon Niverd Pereira, Krishna Khadka, Yu LeiFeature Interaction ModelingData-Free Knowledge Distillation

    Department of Computer Science and Engineering, The University of Texas at Arlington

  29. TTSR: Test-Time Self-Evolving via Reflection

    Feb 6, 2026Haoyang He, Zihua Rong, Yunjia Zhao +3LLM Self-RefinementTest-Time Optimization

    Beijing University of Posts and Telecommunications · Southwestern University of Finance and Economics · China Unicom Online Information Technology Co., Ltd.

  30. BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking

    Feb 5, 2026Bowen Yu, Sheng Zhang, Binhao Wang +8CoT DistillationMathematical Reasoning

    1City University of Hong Kong · 2Mohamed Bin Zayed University of Artificial Intelligence