Teacher-Student Learning

Ask about this topic

Overview

Teacher-student learning studies how one model or agent can help another acquire useful knowledge, skills, or behaviors, including through simulated teaching and tutoring. Common approaches transfer soft labels, features, or reasoning traces; generate pseudo-labels for unlabeled data; and use student-generated rollouts with teacher feedback, sometimes alongside reinforcement learning. Methods are tested in settings such as image segmentation and recognition, diffusion generation, language-model reasoning and tool use, robotics, and human tutoring, using measures of accuracy, efficiency, robustness, and learning progress. A central tension is that more teacher guidance is not always better: its value depends on what the student can learn and whether the supervision matches its own inputs and behavior, while stronger transfer can trade off against compute, diversity, reliability, or safety.

Latest papers

CardsList

176 papers10 papers in the last four weeks, up 67% on the four weeks before. 0.1% of all new papers.

  1. Demystifying On-Policy Distillation: Where Does the Teacher's Advantage Live?

    Oct 13, 2026Yiping Li, Fan Zhao, Zhibo Hou +2On-Policy DistillationTeacher-Student Learning

    University of California, Merced, USA

  2. Avoiding What Your Teacher Already Knows: Entity Familiarity for Deep-Search Distillation

    Oct 12, 2026Yifei Liu, Zhixiang Liang, Wentao Tan +4Language Model DistillationKnowledge Distillation

  3. When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better

    Oct 8, 2026Siyan Zhao, Yonggan Fu, Jindong Jiang +8Language Model DistillationOn-Policy Distillation

    NVIDIA

  4. Stochastic Teacher Intervention for Agentic On-Policy Distillation

    Oct 7, 2026Junnan Liu, Linhao Luo, Zhijun Chen +3On-Policy DistillationTeacher-Student Learning

    Department of Data Science and AI, Faculty of Information Technology, Monash University, Australia · The Hong Kong Polytechnic University, Hong Kong SAR, China · Zhongguancun Laboratory, Beijing, P.R. China

  5. Decoupled Optimization for Teacher-Student Semi-Supervised Learning via a Pioneer Student

    Oct 7, 2026Haorong Han, Jidong Yuan, Chixuan Wei +1Semi-Supervised LearningTeacher-Student Learning

    School of Computer Science and Technology Beijing Jiaotong University · School of Data Science and Intelligent Media Communication University of China

  6. How Should Teachers Be Prepared? RL on Student-Induced States for On-Policy Distillation

    Oct 4, 2026Xiaoyu Ma, Haoyue Liu, Zhichao Wang +2On-Policy DistillationTeacher-Student Learning

    School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen · Shenzhen Future Network of Intelligence Institute (FNiI-Shenzhen) · Guangdong Provincial Key Laboratory of Future Networks of Intelligence, CUHK(SZ)

  7. Cross-Trait Transfer in Subliminal Learning

    Oct 3, 2026Xingyu Zhao, Yiqiao ZhongSubliminal LearningTeacher-Student Learning

    Department of Computer Sciences, University of Wisconsin-Madison · Department of Statistics, University of Wisconsin-Madison

  8. Removing spurious minima for planar features by skip connections

    Oct 1, 2026Jakob Paul Zimmermann, Andrei Balakin, Moritz Grillo +1Shallow Neural NetworksReLU Neural Networks

    Freie Universität Berlin · Technische Universität Berlin

  9. MC-PanDA++: Simpler, Stronger, and More Robust Domain-Adaptive Panoptic Segmentation

    Sep 30, 2026Ivan Martinović, Josip Šarić, Yuki M. Asano +1Panoptic SegmentationUnsupervised Domain Adaptation

    Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, 10000 Zagreb, Croatia. · Faculty of Computer and Information Science, University of Ljubljana, Večna Pot 113, 1000 Ljubljana, Slovenia. · Fundamental AI Lab, University of Technology Nuremberg, Dr.-Luise-Herzberg-Straße, 90461 Nürnberg, Germany.

  10. Examining Variation in How Guided AI Tutors Resolve Student Impasses

    Sep 29, 2026Bakhtawar Ahtisham, Kirk Vanacore, Alessandra Napoli +5Intelligent Tutoring SystemsEducational Data Mining

    Cornell University, Ithaca, NY, United States · Stanford University, Stanford, CA, United States

  11. TeacherGRPO: Closing the Capacity Gap in Reasoning Distillation via Teacher Alignment

    Sep 27, 2026Zhenyu Lei, Zihan Chen, Yaochen Zhu +5Reasoning DistillationRL for Language Model Reasoning

    University of Virginia · Netflix · University of Washington +2

  12. FedMust: Semi-supervised Multi-task Student-Teacher Federated Learning for Multi-organ CT Segmentation

    Sep 21, 2026Ashkan Moradi, Bendik Skarre Abrahamsen, Mattijs ElschotMulti-Organ SegmentationMulti-Task Learning

    Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway · Central Staff at St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway

  13. A Novel Robot-Assisted Learning Pedagogy for Children with ASD

    Sep 13, 2026Laura Boccanfuso, Erin Barney, Marilena Mademtzi +7Assistive RoboticsHuman-Robot Interaction

    Yale Child Study Center, Yale School of Medicine, New Haven, CT, USA · Now at Van Robotics, Inc. · Yale Social Robotics Lab, Department of Computer Science, Yale University, New Haven, CT, USA +1

  14. Teacher Geometry Shapes Learnability in Teacher-Student Networks

    Sep 9, 2026Kai J. Sandbrink, Flavio Martinelli, Alexander van Meegen +2Neural Network OptimizationNeural Network Training Dynamics

    Department of Experimental Psychology, University of Oxford · Department of Physics, RWTH Aachen

  15. Teacher Should Think Ahead: Adaptive Continuations for Reliable On-Policy Distillation

    Sep 6, 2026Jingang Zhou, Yuyi Zhou, Haiyang Guo +6Language Model DistillationOn-Policy Distillation

    School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Science · Ant Group

  16. Rethinking the Teacher-Student Framework for Test-Time Adaptation

    Sep 2, 2026Damian Sójka, Marc Masana, Bartłomiej Twardowski +1Test-Time AdaptationTeacher-Student Learning

    Poznan University of Technology · Akces NCBR · Graz University of Technology +4

  17. VISTA: Verifier-Informed Student-to-Teacher Adaptation for On-Policy Self-Distillation

    Aug 28, 2026Zewen Ding, Zezhong Wu, Zhou Tao +5On-Policy Self-DistillationTeacher-Student Learning

    University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence

  18. TeaMatch: Teachable Cross-Modal Representation Learning for 2D-3D Matching

    Aug 10, 2026Chongjian Wang, Junjie GaoCross-Modal Representation LearningImage Matching

    Shandong Women’s University · Jinan, China · Shandong University of Science and Technology +1

  19. DASH-OPD: Discrepancy-Aware Switching with Hysteresis for On-Policy Distillation

    Jul 31, 2026Yuchen Xia, Qianguo Sun, Chao Song +3On-Policy DistillationTeacher-Student Learning

    The Chinese University of Hong Kong · IDEA Research · Emdoor Research Institute

  20. Source-Free Controlled Adaptation of Teachers for Continual Test-Time Adaptation

    Jul 26, 2026Anurag Roy, Riddhiman Moulick, Vinay Kumar Verma +2Source-Free Domain AdaptationDomain Adaptation

    IIT Kharagpur · IBM Research · Amazon +1

  21. On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversity

    Jun 24, 2026Andrei Liviu Nicolicioiu, Mohammad Pezeshki, Aaron CourvilleOn-Policy Self-DistillationLanguage Model Distillation

    Mila · Université de Montréal · FAIR at Meta +1

  22. Spatio-Temporal Retrieval-based Priors for Adaptive Computational Teaching in Driving

    Jun 23, 2026Deepak Edakkattil Gopinath, Xiongyi Cui, Jonathan DeCastro +2Autonomous DrivingImitation Learning

    Toyota Research Institute, Cambridge, MA, USA

  23. AsyncOPD: How Stale Can On-Policy Distillation Be?

    Jun 23, 2026Wonjun Kang, Kevin Galim, Seunghyuk Oh +9Language Model DistillationOn-Policy Distillation

    FuriosaAI · Ajou University · UC Berkeley +3

  24. Blockwise Policy-Drift Gating for On-Policy Distillation

    Jun 23, 2026Liwen Zheng, Haiyun JiangLanguage Model DistillationOn-Policy Distillation

    Independent Researcher

  25. ARIA: Adaptive Region-Based Importance Allocation for Conditional Diffusion Distillation

    Jun 22, 2026Loay Mualem, Vinh Tong, Samir Darouich +1Diffusion Model DistillationTeacher-Student Learning

    Institute for AI, University of Stuttgart · IMPRS-IS · Institute of Theoretical Chemistry, University of Stuttgart.

  26. The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models

    Jun 22, 2026Ahmad Algadhi, Ahmed Alzuhair, Omar Alkhulaif +1VLM DistillationMulti-Teacher Knowledge Distillation

    Department of Information and Computer Science King Fahd University of Petroleum and Minerals

  27. Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation

    Jun 22, 2026Jinwei Xiao, Zhuowen Han, Yueqing Sun +6CoT DistillationOn-Policy Distillation

    Meituan Longcat Team · TJUNLP Lab, College of Intelligence and Computing, Tianjin University · Nanjing University

  28. A Formula-Driven Survey and Research Agenda for On-Policy Distillation

    Jun 22, 2026Bowen ZhangLanguage Model DistillationOn-Policy Distillation

    Tsinghua University

  29. LoCC: Detection and Localization of Lip-Syncing Deepfakes via Counterfactual Frame Consistency

    Jun 22, 2026Soumyya Kanti Datta, Shan Jia, Siwei LyuDeepfake DetectionTeacher-Student Learning

    University at Buffalo, State University of New York

  30. On the Position Bias of On-Policy Distillation

    Jun 21, 2026Yan Xie, Sijie Zhu, Tiansheng Wen +2Importance WeightingLanguage Model Distillation

    Xidian University · Georgia Institute of Technology · Amazon AGI SF Lab