Teacher-Student Learning

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 162

All topics

Decoupled Optimization for Teacher-Student Semi-Supervised Learning via a Pioneer Student
Oct 7, 2026Haorong Han, Jidong Yuan, Chixuan Wei +1Semi-Supervised Learning Teacher-Student Learning
School of Computer Science and Technology Beijing Jiaotong University · School of Data Science and Intelligent Media Communication University of China
Removing spurious minima for planar features by skip connections
Oct 1, 2026Jakob Paul Zimmermann, Moritz Grillo, Andrei Balakin +1Shallow Neural Networks ReLU Neural Networks
Freie Universität Berlin · Technische Universität Berlin
MC-PanDA++: Simpler, Stronger, and More Robust Domain-Adaptive Panoptic Segmentation
Sep 30, 2026Ivan Martinović, Josip Šarić, Yuki M. Asano +1Panoptic Segmentation Unsupervised Domain Adaptation
Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, 10000 Zagreb, Croatia. · Faculty of Computer and Information Science, University of Ljubljana, Večna Pot 113, 1000 Ljubljana, Slovenia. · Fundamental AI Lab, University of Technology Nuremberg, Dr.-Luise-Herzberg-Straße, 90461 Nürnberg, Germany.
Examining Variation in How Guided AI Tutors Resolve Student Impasses
Sep 29, 2026Bakhtawar Ahtisham, Kirk Vanacore, Alessandra Napoli +5Intelligent Tutoring Systems Educational Data Mining
Cornell University, Ithaca, NY, United States · Stanford University, Stanford, CA, United States
TeacherGRPO: Closing the Capacity Gap in Reasoning Distillation via Teacher Alignment
Sep 27, 2026Zhenyu Lei, Zihan Chen, Yaochen Zhu +5Reasoning Distillation RL for Language Model Reasoning
University of Virginia · Netflix · University of Washington +2
FedMust: Semi-supervised Multi-task Student-Teacher Federated Learning for Multi-organ CT Segmentation
Sep 21, 2026Ashkan Moradi, Bendik Skarre Abrahamsen, Mattijs ElschotMulti-Task Learning Semi-Supervised Medical Image Segmentation
Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway · Central Staff at St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
A Novel Robot-Assisted Learning Pedagogy for Children with ASD
Sep 13, 2026Laura Boccanfuso, Erin Barney, Marilena Mademtzi +7Assistive Robotics Human-Robot Interaction
Yale Child Study Center, Yale School of Medicine, New Haven, CT, USA · Now at Van Robotics, Inc. · Yale Social Robotics Lab, Department of Computer Science, Yale University, New Haven, CT, USA +1
Teacher Geometry Shapes Learnability in Teacher-Student Networks
Sep 10, 2026Kai J. Sandbrink, Flavio Martinelli, Alexander van Meegen +2Neural Network Optimization Neural Network Training Dynamics
Department of Experimental Psychology, University of Oxford · Department of Physics, RWTH Aachen
Rethinking the Teacher-Student Framework for Test-Time Adaptation
Sep 2, 2026Damian Sójka, Marc Masana, Bartłomiej Twardowski +1Test-Time Adaptation Teacher-Student Learning
Poznan University of Technology · Akces NCBR · Graz University of Technology +4
TeaMatch: Teachable Cross-Modal Representation Learning for 2D-3D Matching
Aug 10, 2026Chongjian Wang, Junjie GaoCross-Modal Representation Learning Image Matching
Shandong Women’s University · Jinan, China · Shandong University of Science and Technology +1
Source-Free Controlled Adaptation of Teachers for Continual Test-Time Adaptation
Jul 26, 2026Anurag Roy, Riddhiman Moulick, Vinay Kumar Verma +2Source-Free Domain Adaptation Domain Adaptation
IIT Kharagpur · IBM Research · Amazon +1
On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversity
Jun 24, 2026Andrei Liviu Nicolicioiu, Mohammad Pezeshki, Aaron CourvilleOn-Policy Self-Distillation Language Model Distillation
Mila · Université de Montréal · FAIR at Meta +1
Spatio-Temporal Retrieval-based Priors for Adaptive Computational Teaching in Driving
Jun 23, 2026Deepak Edakkattil Gopinath, Xiongyi Cui, Jonathan DeCastro +2Autonomous Driving Imitation Learning
Toyota Research Institute, Cambridge, MA, USA
AsyncOPD: How Stale Can On-Policy Distillation Be?
Jun 23, 2026Wonjun Kang, Kevin Galim, Seunghyuk Oh +9Language Model Distillation On-Policy Distillation
FuriosaAI · Ajou University · UC Berkeley +3
Blockwise Policy-Drift Gating for On-Policy Distillation
Jun 23, 2026Liwen Zheng, Haiyun JiangLanguage Model Distillation On-Policy Distillation
Independent Researcher
ARIA: Adaptive Region-Based Importance Allocation for Conditional Diffusion Distillation
Jun 22, 2026Loay Mualem, Vinh Tong, Samir Darouich +1Diffusion Model Distillation Teacher-Student Learning
Institute for AI, University of Stuttgart · IMPRS-IS · Institute of Theoretical Chemistry, University of Stuttgart.
The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models
Jun 22, 2026Ahmad Algadhi, Ahmed Alzuhair, Omar Alkhulaif +1VLM Distillation Multi-Teacher Knowledge Distillation
Department of Information and Computer Science King Fahd University of Petroleum and Minerals
Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation
Jun 22, 2026Jinwei Xiao, Zhuowen Han, Yueqing Sun +6CoT Distillation On-Policy Distillation
Meituan Longcat Team · TJUNLP Lab, College of Intelligence and Computing, Tianjin University · Nanjing University
A Formula-Driven Survey and Research Agenda for On-Policy Distillation
Jun 22, 2026Bowen ZhangLanguage Model Distillation On-Policy Distillation
Tsinghua University
LoCC: Detection and Localization of Lip-Syncing Deepfakes via Counterfactual Frame Consistency
Jun 22, 2026Soumyya Kanti Datta, Shan Jia, Siwei LyuDeepfake Detection Teacher-Student Learning
University at Buffalo, State University of New York
On the Position Bias of On-Policy Distillation
Jun 21, 2026Yan Xie, Sijie Zhu, Tiansheng Wen +2Importance Weighting Language Model Distillation
Xidian University · Georgia Institute of Technology · Amazon AGI SF Lab
Channel Location Constrains the Auditability of Subliminal Learning
Jun 20, 2026Tamas MadlSubliminal Learning Language Model Distillation
Austrian Research Institute for Artificial Intelligence · University of Vienna
Prefix-Guided On-Policy Distillation: Mining Golden Trajectories from Rollouts
Jun 20, 2026Qingfei Zhao, Huan Song, Shuyu Tian +2Trajectory Generation CoT Distillation
TeleAI · Shanghai Jiao Tong University
When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
Jun 19, 2026Siyang Lyu, Xinghao Chen, Zhijing Sun +3Model Compression CoT Distillation
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Viterbi School of Engineering, University of Southern California · The Hong Kong Polytechnic University +2
Reference-Based Distillation Detection in LLMs
Jun 19, 2026Rajat Rawat, Sizhe Chen, Akshay Anand +3LLM Auditing Teacher-Student Learning
University of California, Berkeley · University of Southern California · OpenAI
Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients
Jun 16, 2026Byung-Kwan Lee, Ximing Lu, Shizhe Diao +6Curriculum RL LLM-Guided RL
NVIDIA
PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation
Jun 15, 2026Anhao Zhao, Junlong Tong, Yingqi Fan +3On-Policy Distillation Teacher-Student Learning
Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University · University of Waterloo +1
Taylor-Calibrate: Principled Initialization for Hybrid Linear Attention Distillation
Jun 15, 2026Zhongzhu Zhou, Qingyang Wu, Junxiong Wang +4Neural Network Initialization Linear Attention
1The University of Sydney · 2Together AI · 5Microsoft +2
LectūraAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching
Jun 15, 2026Jaward Sesay, Yue Yu, Siwei Dong +1Multi-Agent LLM Systems Intelligent Tutoring Systems
Beijing Institute of Technology · Peking University · Beijing Academy of Artificial Intelligence
On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents
Jun 14, 2026Gengsheng Li, Mao Zheng, Mingyang Song +8Language Model Distillation LLM Agent Training
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Large Language Model Department, Tencent +4
Localizing Credit at the Divergence: Path-Conditioned Self-Distillation for LLM Reasoning
Jun 14, 2026Yu Li, Shu Hong, Tian LanRL for Language Model Reasoning Credit Assignment in RL
Department of Electrical and Computer Engineering, George Washington University
Teacher-Student Structure for Domain Adaptation in Ensemble Audio-Visual Video Deepfake Detection
Jun 13, 2026Elham Abolhasani, Maryam Ramezani, Hamid R. RabieeAudio Deepfake Detection Domain Adaptation
Department of Computer Engineering, Sharif University of Technology.
Robust Fall Recovery for Armless Bipedal-Wheeled Robots Via Force-Guided Learning
Jun 12, 2026Haidong Hou, Zhangguo Yu, Tao Han +6Robot Failure Recovery Robot Locomotion
School of Mechatronical Engineering, Beijing Institute of Technology, Beijing 100081, China
Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation
Jun 11, 2026Guo Yu, Wenlin Liu, Yulan Hu +3Sparse Fine-Tuning On-Policy Distillation
School of Artificial Intelligence, Nanjing University · National Key Laboratory for Novel Software Technology, Nanjing University · Amap, Alibaba Group
A solvable model for unsupervised federated learning
Jun 11, 2026Giovanni Catania, Aurélien Decelle, Gianluca Manzan +2Statistical Physics of Learning Unsupervised Learning
Institute for Cross-disciplinary Physics and Complex Systems IFISC (CSIC-UIB), Campus Universitat Illes Balears, 07122 Palma de Mallorca, Spain. · Escuela Técnica Superior de Ingenieros Industriales, Universidad Politécnica de Madrid, Calle de José Gutiérrez Abascal 2, Madrid 28006, Spain. · GISC - Grupo Interdisciplinar de Sistemas Complejos 28040 Madrid, Spain. +5
Beyond Dark Knowledge: Mixup-Based Distillation for Reliable Predictions
Jun 10, 2026José Medina, Paul Honeine, Abdelaziz Bensrhair +1Data Augmentation Model Calibration
ITS Lab, Institute of Computer Science, University of Tartu, Estonia. · LITIS, Universit´e de Rouen, France. · LITIS, INSA de Rouen, France.
Hey Chat, Can You Teach Me? Structuring Socratic Dialogue for Human Learning in the Wild
Jun 10, 2026Sidney Tio, Arunesh Sinha, Pradeep VarakanthamIntelligent Tutoring Systems AI in Education
School of Computing and Information Systems Singapore Management University · Department of Management Science and Information Systems Rutgers Business School
RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation
Jun 10, 2026Leyi Pan, Shuchang Tao, Yunpeng Zhai +5RL for Language Model Reasoning Language Model Distillation
1Tsinghua University · 2Tongyi Lab , Alibaba Group · 3Peking University
When Context Returns: Toward Robust Internalization in On-Policy Distillation
Jun 10, 2026Xun Wang, Ruishuo Chen, Zhuoran Li +2On-Policy Distillation Context Distillation
1IIIS, Tsinghua University
Quantifying Subliminal Behavioral Transfer Ratios in Language Model Distillation
Jun 9, 2026Uwe König, Hamza Kazmi, Ruizhe Li +1Subliminal Learning Language Model Distillation
1Technical University of Munich · 2Algoverse · University of Aberdeen +1
PADD: Path-Aligned Decompression Distillation for Non-Router Teacher to Guide MoE Student Learning
Jun 9, 2026Xinyue Peng, Yi Qian, Jiaojiao Lin +2Mixture-of-Experts Language Models Expert Routing
Intel Corporation, China · Zhejiang University, China
Escaping the KL Agreement Trap in On-Policy Distillation
Jun 8, 2026Haoran Xin, Anhao Zhao, Ying Sun +3On-Policy Distillation Teacher-Student Learning
Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong Polytechnic University · Eastern Institute of Technology, Ningbo +1
SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling
Jun 8, 2026Haoran Xu, Hongyu Wang, Yifei Gao +3Language Model Distillation On-Policy Distillation
Zhejiang University · Hunan University · Tianjin University +2
Trajectory-Refined Distillation
Jun 7, 2026Li Jiang, Haoran Xu, Yichuan Ding +1On-Policy Distillation LLM Post-Training
McGill University · Mila Quebec AI Institute · UT Austin
OPRD: On-Policy Representation Distillation
Jun 4, 2026Shenzhi Yang, Guangcheng Zhu, Bowen Song +8Cross-Architecture Knowledge Distillation Language Model Distillation
Zhejiang University · Ant Group
Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation
Jun 4, 2026Maxime Griot, Paul Steven Scotti, Tanishq Mathew AbrahamCoT Distillation Teacher-Student Learning
Université catholique de Louvain · Sophont Inc.
ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation
Jun 4, 2026Kanghui Tian, Siyuan Liu, Ziang Yan +3VLM Distillation Multimodal Reasoning
1Shanghai AI Laboratory · 3Nanjing University · 2Fudan University
SocraticPO: Policy Optimization via Interactive Guidance
Jun 3, 2026Zirui Liu, Jie Ouyang, Qi Liu +8RL for Language Model Reasoning LLM-Guided RL
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China · iFLYTEK AI Research (Central China), iFLYTEK Co., Ltd
When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation
Jun 2, 2026Haowei Guo, Baolong Bi, Ruicheng Zhang +2On-Policy Self-Distillation Language Model Distillation
Peking University · University of Chinese Academy of Sciences · Tsinghua University
What Makes Interaction Trajectories Effective for Training Terminal Agents?
Jun 2, 2026Sidi Yang, Chaofan Tao, Jierun Chen +11AI Agent Benchmarks LLM Agent Training
1The University of Hong Kong · 2Huawei Technologies · 3Nanyang Technological University
What Do Students Learn? A Feature-Level Analysis of Dark Knowledge
Jun 2, 2026Seungu Kang, Songkuk KimRepresentation Learning Self-Distillation
Yonsei University, Seoul 03722, Republic of Korea
Why Are DMD Students Lazy? Understanding the Copying Behavior in Few-Step Distillation
Jun 1, 2026Shucheng Li, Iolo Jones, Alexander Tong +1Diffusion Model Distillation Few-Step Diffusion Sampling
Department of Computer Science, University of Oxford, Oxford, UK · AITHYRA, Research Institute for Biomedical AI, Vienna, AT
OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification
May 31, 2026Yuhang Zhou, Lizhu Zhang, Yifan Wu +5RL for Language Model Reasoning Language Model Distillation
Meta AI
What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression
May 31, 2026Wendao Wu, Fangqing Zhang, Haihan Zhang +1Weak-to-Strong Generalization Teacher-Student Learning
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University, China · School of Mathematical Sciences, Peking University, China
OPD+: Rethinking the Advantage Design for On-Policy Distillation
May 31, 2026Hanyang Zhao, Haoxian Chen, Han Lin +3Language Model Distillation Policy Gradient Methods
Columbia University · Amazon · Meta +1
Trust Functions: Near-Lossless Weak-to-Strong Generalization by Learning When to Trust the Weak Teacher
May 31, 2026Arda Uzunoglu, Alvin Zhang, Daniel KhashabiData Selection Training Data Selection
Department of Computer Science, Johns Hopkins University, Maryland, United States.
Subliminal Learning Is Steering Vector Distillation
May 31, 2026Camila Blank, Agam Bhatia, Senthooran Rajamanoharan +2Language Model Steering Subliminal Learning
Stanford University
DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models
May 30, 2026Abdullah Al Shafi, Kazi Saeed Alam, Sk Imran Hossain +1Model Compression Classifier-Free Guidance
Khulna University of Engineering & Technology, Bangladesh · University Clermont Auvergne, France
Logit Distillation on Manifolds: Mapping by Learning
May 30, 2026Yiru Yang, Junling Wang, Nishant Kumar Singh +2Teacher-Student Learning Representation Alignment
University of Zurich · ETH Zurich · Deutsche Bank Securities
Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance
May 29, 2026Yuxuan Jiang, Francis FerraroCoT Distillation Language Model Distillation
University of Maryland, Baltimore County

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Decoupled Optimization for Teacher-Student Semi-Supervised Learning via a Pioneer Student

Haorong Han, Jidong Yuan, Chixuan Wei, Yongqi Sun

School of Computer Science and Technology Beijing Jiaotong University · School of Data Science and Intelligent Media Communication University of China

Semi-Supervised Learning Teacher-Student Learning Self-Training Deep Learning Optimization

Open paper AskPDF arXiv

Semi-supervised learning (SSL) relies on two core mechanisms: self-training under the Teacher-Student (T-S) framework and joint optimization of labeled and unlabeled losses. Despite their effectiveness, we find both mechanisms introduce distinct optimization pathologies. First, parameter coupling enforces strict synchronization between teacher and student, where strong regularization on the student degrades the teacher's fitting ability, thereby limiting the permissible generalization intensity. Second, the imbalance in gradient update consistency between labeled and unlabeled losses drives the shared parameters to prematurely converge to labeled-dominated local minima, creating a bottleneck for global optimization. To address both issues, we propose the Pioneer Student (PiS), an auxiliary branch that operates in an independent parameter space and periodically transfers accumulated knowledge back to the T-S model. Extensive experiments show that PiS is a universal plug-and-play module that consistently improves mainstream SSL methods.

Momentum

Related topics

Latest papers 162