Teacher

Follow this topic Ask about this topic

Momentum

51 papers in the last four weeks, up 219% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 203

All topics

Reference-Based Distillation Detection in LLMs
Jun 19, 2026Rajat Rawat, Sizhe Chen, Akshay Anand +3Teacher-Student Distillation Teacher
University of California, Berkeley · University of Southern California · OpenAI
Dyna-DINO: Efficient ViT Distillation Via Adaptive Representation Anchoring
Jun 17, 2026Jiaqi Zhang, Ashton Lee, Anthony Wong +3Knowledge Distillation Dataset Distillation
Brown University · Rice University
Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients
Jun 16, 2026Byung-Kwan Lee, Ximing Lu, Shizhe Diao +6Proximal Policy Optimization Teacher
NVIDIA
Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Grounding
Jun 16, 2026Jingyuan Huang, Zuming Huang, Yucheng Shi +4Graphical User Interface Self-Distillation Framework
University of Georgia · 2INFLY Tech · *This work was done during an internship at INFLY Tech. +2
When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning
Jun 16, 2026Fred Philippy, Siwen Guo, Jacques Klein +1Cross-Lingual Transfer Language Pairs
Snt, University of Luxembourg, Luxembourg · Luxembourg Institute of Science and Technology, Luxembourg
Confusion-Aware Transfer Teacher Curriculum Learning Framework: Disentangling Scoring and Pacing Effects
Jun 16, 2026Savini Kommalage, Sanka Mohottala, Asiri Gawesha +5Adaptive Curriculum Curriculum
Faculty of Computing, Sri Lanka Institute of Information Technology, Sri Lanka · Faculty of Engineering, University of Sri Jayewardenepura, Sri Lanka · Faculty of Engineering, Sri Lanka Institute of Information Technology, Sri Lanka +2
On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents
Jun 14, 2026Gengsheng Li, Mao Zheng, Mingyang Song +8Teacher Curriculum
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Large Language Model Department, Tencent +4
Teacher-Student Structure for Domain Adaptation in Ensemble Audio-Visual Video Deepfake Detection
Jun 13, 2026Elham Abolhasani, Maryam Ramezani, Hamid R. RabieeEnvironmental Sound Deepfake Detection Deepfake Detection
Department of Computer Engineering, Sharif University of Technology.
Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching
Jun 10, 2026Zhuoyi Peng, Hanlin Gu, Lixin Fan +1Gnn-Based Detectors High Confidence Pseudo-Labels
1The Hong Kong University of Science and Technology · 2WeBank
Beyond Absolute Imitation: Anchored Residual Guidance for Privileged On-Policy Distillation
Jun 9, 2026Wenhao ZhangEfficient On-Policy Distillation Teacher
South China University of Technology
PADD: Path-Aligned Decompression Distillation for Non-Router Teacher to Guide MoE Student Learning
Jun 9, 2026Xinyue Peng, Yi Qian, Jiaojiao Lin +2Teacher-Student Distillation Teacher
Intel Corporation, China · Zhejiang University, China
AI tools in Arab University English classrooms: Looking back and forward
Jun 8, 2026Mohammed Q. Shormani, Muneef Yehia AlshawshClassrooms Arabic
Ibb University, Yemen
SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling
Jun 8, 2026Haoran Xu, Hongyu Wang, Yifei Gao +3Efficient On-Policy Distillation Teacher
Zhejiang University · Hunan University · Tianjin University +2
CRANE: Knowledge Editing for Reasoning MLLMs
Jun 8, 2026Han Huang, Hao Wang, Mengqi Zhang +3LLM Reasoning Strategies Chain-of-Thought Reasoning
University of Chinese Academy of Sciences, Beijing 101408, China, and also with the New Laboratory of Pattern Recognition (NLPR), CASIA, Beijing 100045, China · Harbin Institute of Technology, Weihai 264209, China · School of Computer Science and Technology, Shandong University, Qingdao 266237, China
Trajectory-Refined Distillation
Jun 7, 2026Li Jiang, Haoran Xu, Yichuan Ding +1Symbolic Distillation Teacher
McGill University · Mila Quebec AI Institute · UT Austin
Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation
Jun 5, 2026Mohammadreza Sadeghi, Sareh Soleimani, Zihan Wang +1Replay-Based Continual Learning Clustering
Department of Electrical and Computer Engineering, McGill University, Montreal, QC H3A 0E9, Canada · Mila – Quebec AI Institute, Montreal, QC H2S 3H1, Canada
HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers
Jun 4, 2026Lizhi Yang, Junheng Li, Nehar Poddar +5Whole-Body Control Full-Body Humanoid Control
California Institute of Technology · The Institute for Human & Machine Cognition
OPRD: On-Policy Representation Distillation
Jun 4, 2026Shenzhi Yang, Guangcheng Zhu, Bowen Song +8Teacher Token-Level Uncertainty
Zhejiang University · Ant Group
SocraticPO: Policy Optimization via Interactive Guidance
Jun 3, 2026Zirui Liu, Jie Ouyang, Qi Liu +8Frictive Policy Optimization Socratic Dialogue
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China · iFLYTEK AI Research (Central China), iFLYTEK Co., Ltd
When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation
Jun 2, 2026Haowei Guo, Baolong Bi, Ruicheng Zhang +2Efficient On-Policy Distillation Teacher
Peking University · University of Chinese Academy of Sciences · Tsinghua University
VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization
Jun 1, 2026Junhao Cheng, Liang Hou, Tianxiong Zhong +4Teacher
City University of Hong Kong · Kling Team, Kuaishou Technology
When Rating Scales Fall Short: LLM-Assisted Discovery of ADHD Signals in Turkish Teacher Narratives
Jun 1, 2026Baris Karacan, Irem Aktar Songur, Ahmet Ozaslan +1Autism Spectrum Disorder Computerized Adaptive Testing
Department of Computer Science, University of Illinois Chicago · Department of Child and Adolescent Psychiatry, Gazi University
OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification
May 31, 2026Yuhang Zhou, Lizhu Zhang, Yifan Wu +5Teacher Token-Level Uncertainty
Meta AI
What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression
May 31, 2026Wendao Wu, Fangqing Zhang, Haihan Zhang +1Cross-Task Knowledge Transfer Knowledge Distillation
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University, China · School of Mathematical Sciences, Peking University, China
Trust Functions: Near-Lossless Weak-to-Strong Generalization by Learning When to Trust the Weak Teacher
May 31, 2026Arda Uzunoglu, Alvin Zhang, Daniel KhashabiLearnability Teacher
Department of Computer Science, Johns Hopkins University, Maryland, United States.
Subliminal Learning Is Steering Vector Distillation
May 31, 2026Camila Blank, Agam Bhatia, Senthooran Rajamanoharan +2Multiple Prompt Learning Frameworks Steering
Stanford University
Logit Distillation on Manifolds: Mapping by Learning
May 30, 2026Yiru Yang, Junling Wang, Nishant Kumar Singh +2Residual Distillation Teacher
University of Zurich · ETH Zurich · Deutsche Bank Securities
Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs
May 30, 2026Yu-An Lu, Ci-Yang Tsai, Yu-Lin Tsai +2Reasoning Traces LLM Reasoning Strategies
National Yang Ming Chiao Tung University · UC Berkeley
Student Capacity Moderates Knowledge Distillation Effectiveness: A Systematic Study Across ResNet Teacher-Student Pairs on CIFAR-10
May 29, 2026Umut Onur YasarKnowledge Distillation Teacher
Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
May 29, 2026Yanjiang Liu, Jie Lou, Xinyan Guan +7Teacher Dataset Distillation
University of Chinese Academy of Sciences · 2Chinese Information Processing Laboratory Institute of Software, Chinese Academy of Sciences 2University of Chinese Academy of Sciences, Beijing, China · 3Xiaohongshu
Surfacing Isolated Learners with Outcome-Independent Mediation of Feedback between Teachers and Students Using AI
May 28, 2026Junsoo Park, Youssef Medhat, Htet Phyo Wai +2Teacher Feedback
Georgia Institute Of Technology, North Avenue, Atlanta, GA 30332, USA
Skill-Conditioned Gated Self-Distillation for LLM Reasoning
May 27, 2026Jiazhen Huang, Xiao Chen, Xiao Luo +3Unsupervised On-Policy Self-Distillation LLM Reasoning Strategies
Tsinghua University · University of Wisconsin-Madison · X-Humanoid +2
Teacher-Student Representational Alignment for Reinforcement Learning-Driven Imitation Learning
May 27, 2026Meraj Mammadov, Pedro Zuidberg Dos Martires, Johannes Andreas StorkTemporally Coherent Imitation Learning Imitation Learning
Department of Computer Science Örebro University
Reflective Dialogue between Teacher and Solver Agents for Video Question Answering
May 27, 2026Takuya Murakawa, Toru TamakiLong Video Question Answering In-Context Learning
Nagoya Institute of Technology, Nagoya, Japan
Adversarial Dual On-Policy Distillation from Expressive Teacher
May 26, 2026Zhenglin Wan, Jingxuan Wu, Xingrui Yu +5Multimodal Action Distributions Behavior Cloning
School of Computing, National University of Singapore · Department of Statistics and Operations Research, UNC-Chapel Hill, America · CFAR, Agency for Science, Technology and Research, Singapore +4
When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection
May 26, 2026Zhengyu Hu, Zheyuan Xiao, Linxin Song +10Teacher Effective Distillation
University of Washington · University of Texas at Austin · University of Southern California +6
On-Policy Adversarial Flow Distillation for Autoregressive Video Generation
May 25, 2026Yang Luo, Shengju Qian, Xiaohang Tang +4Autoregressive Video Generation Video Generation
National Univesity of Singapore · LIGHTSPEED · University College London
Strong Teacher Not Needed? On Distillation in LLM Pretraining
May 22, 2026Taiming Lu, Zhuang LiuDataset Distillation Teacher
Princeton University
The Distillation Game: Adaptive Attacks & Efficient Defenses
May 21, 2026Youssef Allouah, Mahdi Haghifam, Sanmi Koyejo +1Distribution Matching Distillation LLM Defense Mechanisms
Stanford University · Toyota Technological Institute at Chicago · National University of Singapore
A Boundary-Layer Mechanism for One-Third Scaling in Online Softmax Classification
May 21, 2026Marcel Kühn, Yoon Thelge, Bernd RosenowBatch Gumbel-Softmax Relaxation
Institute for Theoretical Physics, Leipzig University · 2ScaDS.AI Dresden/Leipzig
TextTeacher: What Can Language Teach About Images?
May 21, 2026Tobias Christian Nauen, Stanislav Frolov, Brian Bernhard Moser +3Imagenet Text Embeddings
RPTU University Kaiserslautern-Landau · German Research Center for Artificial Intelligence (DFKI)
Toward Understanding Adversarial Distillation: Why Robust Teachers Fail
May 21, 2026Hongsin Lee, Hye Won ChungTeacher-Student Distillation Dataset Distillation
School of Electrical Engineering, KAIST, Daejeon, Korea.
X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation
May 20, 2026Sharath Turuvekere Sreenivas, Adithyakrishna Venkatesh Hanasoge, Mingyu Yang +4Teacher-Student Distillation Single-Token Output Distributions
When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning
May 20, 2026Xiaogeng Liu, Xinyan Wang, Yingzi Ma +2Teacher On-Policy
1Johns Hopkins University · University of Wisconsin–Madison · 3Nanyang Technological University
AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals
May 20, 2026Duy Nguyen, Hanqi Xiao, Archiki Prasad +7Self-Distillation Framework Teacher
UNC Chapel Hill · Capital One · The University of Texas at Austin
Distilling Tabular Foundation Models for Structured Health Data
May 18, 2026Aditya Tanna, Nassim Bouarour, Mohamed Bouadi +2Tabular Foundation Models Medical World Model
Lexsi Labs
Pocket Foundation Models: Distilling TFMs into CPU-Ready Gradient-Boosted Trees
May 18, 2026Aditya Tanna, Nassim Bouarour, Mohamed Bouadi +2Tabular Foundation Models Xgboost Model
Lexsi Labs
Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers
May 16, 2026Fanqin Zeng, Feng Hong, Geng Yu +6Diffusion Language Models Generative Quality
Shanghai Jiao Tong University, Shanghai, China · Apple MLR, Cupertino, CA, USA · Tongji University, Shanghai, China +2
How to Choose Your Teacher for Fine Grained Image Recognition
May 15, 2026Oswin Gosal, Edwin Arkel Rios, Augusto Christian Surya +3Knowledge Distillation Teacher
National Tsing Hua University, Taiwan · National Yang Ming Chiao Tung University, Taiwan
Cognitive-Uncertainty Guided Knowledge Distillation for Accurate Classification of Student Misconceptions
May 14, 2026Qirui Liu, Hao Chen, Weijie Shi +2Student Misconceptions Knowledge Distillation
South China University of Technology · Tencent Financial Technology · The Hong Kong University of Science and Technology +2
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
May 13, 2026Xinyu Liu, Kechen Jiao, Chunyang Xiao +10Frictive Policy Optimization Teacher
School of Computer Science and Engineering, Northeastern University, China · Tsinghua University · Meituan +2
On the Generalization of Knowledge Distillation: An Information-Theoretic View
May 13, 2026Bingying Li, Haiyun HeKnowledge Distillation Probe-Logit Distillation
Internet of Things Thrust, Information Hub The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China
Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
May 12, 2026Yuanda Xu, Hejian Sang, Zhengze Zhou +3Teacher Training Data
Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning
May 12, 2026Zihao Han, Tiangang Zhang, Huaibin Wang +1Self-Distillation Framework Teacher
ByteDance Douyin
Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVR
May 11, 2026Jeonghye Kim, Jiwon Jeon, Dongsheng Li +1Unsupervised On-Policy Self-Distillation Reinforcement Learning With Verifiable Reward
Microsoft Research · KAIST
LLaVA-CKD: Bottom-Up Cascaded Knowledge Distillation for Vision-Language Models
May 11, 2026Nikolaos Gkalelis, Vasileios MezarisKnowledge Distillation Teacher
CERTH-ITI, Thessaloniki, Greece, 57001
Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation
May 10, 2026Yuxuan Jiang, Runchao Li, Shubhashis Roy Dipta +2Teacher Card
University of Maryland, Baltimore County · Case Western Reserve University · Arizona State University +1
The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs
May 9, 2026Xin Li, Hao Jiang, Annan Wang +2Teacher Gradient Clipping
Nanyang Technological University, Singapore
SOD: Step-wise On-policy Distillation for Small Language Model Agents
May 8, 2026Qiyong Zhong, Mao Zheng, Mingyang Song +5LLM Reasoning Strategies Agentic Reasoning
1Zhejiang University · 2Large Language Model Department, Tencent · University of Science and Technology of China +1
SimCT: Recovering Lost Supervision for Cross-Tokenizer On-Policy Distillation
May 8, 2026Jie Sun, Mao Zheng, Mingyang Song +6Teacher Tokenizer
University of Science and Technology of China · Shanghai Innovation Institute · Large Language Model Department, Tencent +1

Jun 19, 2026, cs.LGJ/K move · Enter open · S save

Reference-Based Distillation Detection in LLMs

Rajat Rawat, Sizhe Chen, Akshay Anand, Michael Duan+2

University of California, Berkeley · University of Southern California · OpenAI

Teacher-Student Distillation Teacher Deepseek Membership Inference

Open paper AskPDF arXiv

Model distillation -- training on outputs from stronger third-party models -- is widely used to boost performance, but raises concerns about unfair advantages and policy violations. This motivates a fundamental question: can we detect whether a model was distilled from another? We show that, while identifying a teacher model from a student in isolation is highly challenging, it becomes tractable in a reference-based setting: given a model and an earlier-generation checkpoint from the same lineage, we can identify the teacher model used to train the later checkpoint. We introduce a distillation detection method based on reference-based membership inference. By comparing how strongly a student model preferentially aligns with outputs from different candidate teachers relative to a reference checkpoint, our method identifies the most likely teacher and detects evidence of distillation. To handle unknown distillation pipelines such as hidden prompts, we infer proxy prompt templates directly from model outputs. We additionally identify a distinctive glyph-level signal specific to o1/o3 models. Evaluating distillation detection is challenging because modern model lineages are already heavily entangled. To address this, we develop a hybrid evaluation spanning both controlled distillation experiments and real-world models. Across both settings, our approach recovers the true teacher with near-perfect accuracy in single-teacher distillation scenarios, even when the underlying distillation pipeline is largely unknown. We further introduce statistical tests for both teacher attribution and distillation detection, and extend our framework to open-world settings where no teacher is guaranteed to be present among the candidates. Applying our method to contemporary models yields new evidence regarding potential distillation relationships involving QwQ, DeepSeek-R1, and GPT-OSS.

Momentum

Related topics

Latest papers 203