Language Model Distillation

Follow this topic Ask about this topic

Momentum

69 papers in the last four weeks, up 200% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 350

All topics

Distilling Safe LLM Systems via Soft Prompts for On Device Settings
Jun 8, 2026Motasem Alfarra, Cristina Pinneri, Dana Kianfar +2On-Device Language Model Inference LLM Safety Alignment
Qualcomm AI Research
SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling
Jun 8, 2026Haoran Xu, Hongyu Wang, Yifei Gao +3Language Model Distillation On-Policy Distillation
Zhejiang University · Hunan University · Tianjin University +2
From Hazard Functions to Language Space: Cox-Supervised Distillation of Survival Risk into a Large Language Model
Jun 8, 2026Nicholas I-Hsien Kuo, Blanca Gallego, Louisa JormSurvival Analysis Language Model Calibration
Centre for Big Data Research in Health, the University of New South Wales, Sydney, Australia
On the Geometry of On-Policy Distillation
Jun 5, 2026Zhennan Shen, Yanshu Li, Qingyu Yin +6Language Model Distillation On-Policy Distillation
1HKUST · 2Brown University · 3Zhejiang University +5
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Jun 4, 2026Xingyu Su, Jacob Helwig, Shubham Parashar +6Autoregressive Diffusion Language Model Distillation
Department of Computer Science and Engineering, Texas A&M University · Department of Bioinformatics and Systems Medicine,2026 University of Texas Health Science Center at Houston · Department of Electrical and Computer Engineering, Texas A&M UniversityJun 4
OPRD: On-Policy Representation Distillation
Jun 4, 2026Shenzhi Yang, Guangcheng Zhu, Bowen Song +8Cross-Architecture Knowledge Distillation Language Model Distillation
Zhejiang University · Ant Group
LoRi: Low-Rank Distillation for Implicit Reasoning
Jun 3, 2026Ryan Solgi, Jiayi Tian, Zheng ZhangCoT Distillation Mathematical Reasoning
University of California-Santa Barbara, USA
DuDi: Dual-Signal Distillation with Cross-Lingual Verbalizer
Jun 3, 2026Patomporn Payoungkhamdee, Tinnakit Udsa, Jian Gang Ngui +3Multilingual Language Models Small Language Models
School of Information Science and Technology, VISTEC · AI Singapore · MBZUAI
When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation
Jun 2, 2026Haowei Guo, Baolong Bi, Ruicheng Zhang +2On-Policy Self-Distillation Language Model Distillation
Peking University · University of Chinese Academy of Sciences · Tsinghua University
Constitutional On-Policy Safe Distillation
Jun 2, 2026Ming Wen, Yuxuan Liu, Kun Yang +8On-Policy Self-Distillation LLM Safety Alignment
Institute of Trustworthy Embodied AI, Fudan University · 2Shanghai Innovation Institute · 3Ant Group +3
Regret Pre-training: Bridging Prior and Posterior Views for Enhanced Knowledge Grounding
Jun 2, 2026Mingkuan Zhao, Xiayu Sun, Wentao Hu +5Language Model Pretraining LLM Grounding
The School of Computer Science and Technology, Xi’an Jiaotong University, China.
Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation
Jun 1, 2026Yuying Li, Leqi Zheng, Yongzi Yu +6Selective Knowledge Distillation Language Model Distillation
THU · Meituan · HKUST +2
OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification
May 31, 2026Yuhang Zhou, Lizhu Zhang, Yifan Wu +5RL for Language Model Reasoning Language Model Distillation
Meta AI
Trust Region On-Policy Distillation
May 31, 2026Xingrun Xing, Haoqing Wang, Boyan Gao +2Language Model Distillation Trust-Region Optimization
Samsung Research, Beijing, China · University of Oxford · Peking University
ThinkSwitch: Context Distillation with LoRA and Weight Interpolation for Specific-Purpose Reasoning Tasks
May 31, 2026Dhruv Saini, Rohan PandeyEfficient Language Model Reasoning Language Model Distillation
Bellevue High School · DigitalOcean
OPD+: Rethinking the Advantage Design for On-Policy Distillation
May 31, 2026Hanyang Zhao, Haoxian Chen, Han Lin +3Language Model Distillation Policy Gradient Methods
Columbia University · Amazon · Meta +1
Subliminal Learning Is Steering Vector Distillation
May 31, 2026Camila Blank, Agam Bhatia, Senthooran Rajamanoharan +2Language Model Steering Subliminal Learning
Stanford University
Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation
May 30, 2026Ruiqi Zhang, Lingxiang Wang, Hainan Zhang Zhiming ZhengLanguage Model Distillation Language Model Robustness
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University
Weak Critics Make Strong Learners: On-Policy Critique Distillation for Scalable Oversight
May 29, 2026Can Jin, Jiakang Li, Rui Wu +2Scalable Oversight Weak-to-Strong Generalization
Department of Computer Science, Rutgers University, Piscataway, United States.
Rethinking the Role of Temperature in Large Language Model Distillation
May 29, 2026Hoang-Chau Luong, Lingwei ChenLanguage Model Distillation Knowledge Distillation
Golisano College of Computing and Information Sciences Rochester Institute of Technology Rochester, NY, United States
Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance
May 29, 2026Yuxuan Jiang, Francis FerraroCoT Distillation Language Model Distillation
University of Maryland, Baltimore County
Are Full Rollouts Necessary for On-Policy Distillation?
May 29, 2026Yaocheng Zhang, Jiajun Chai, Yuqian Fu +7RL for Language Model Reasoning Language Model Distillation
Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Meituan +1
Trust-Region Behavior Blending for On-Policy Distillation
May 29, 2026Daniil Plyusov, Alexey Gorbatovski, Alexey Malakhov +4Language Model Distillation Trust-Region Optimization
T-Tech
Distilling LLM Feedback for Lean Theorem Proving
May 29, 2026Gaetan Narozniak, Gérard Biau, Rémi Munos +2RL for Language Model Reasoning Mathematical Reasoning
FAIR at Meta, Inria · Sorbonne Université, Institut universitaire de France · FAIR at Meta +2
Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
May 29, 2026Yanjiang Liu, Jie Lou, Xinyan Guan +7CoT Distillation Language Model Distillation
University of Chinese Academy of Sciences · 2Chinese Information Processing Laboratory Institute of Software, Chinese Academy of Sciences 2University of Chinese Academy of Sciences, Beijing, China · 3Xiaohongshu
LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation
May 28, 2026Tianrun Yu, Kaixiang Zhao, Chih-Chun Chen +5CoT Distillation Language Model Distillation
Brigham Young University · The Pennsylvania State University · University of North Carolina at Chapel Hill
Speculative Decoding Across Languages
May 28, 2026Nirajan Paudel, Michael Ginn, Luc De Nardi +1Multilingual Language Models Speculative Decoding
University of Colorado
Bounded Behavioral Indistinguishability for Black-Box LLM Distillation
May 28, 2026Munawar HasanLLM Evaluation Language Model Distillation
Michigan Technological University, Houghton, MI, USA
Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
May 28, 2026Zizhuo Lin, Quanling Liu, Jinsheng Quan +6LLM Grounding Reasoning Consistency in Language Models
Zhejiang University · University of Science and Technology of China
Draft-OPD: On-Policy Distillation for Speculative Draft Models
May 28, 2026Haodi Lei, Yafu Li, Haoran Zhang +8Speculative Decoding LLM Inference Acceleration
Shanghai Jiao Tong University · Shanghai AI Laboratory · The Chinese University of Hong Kong +3

Jun 8, 2026, cs.LGJ/K move · Enter open · S save

Distilling Safe LLM Systems via Soft Prompts for On Device Settings

Motasem Alfarra, Cristina Pinneri, Dana Kianfar, Mohammed Almousa+1

Qualcomm AI Research

On-Device Language Model Inference LLM Safety Alignment Prompt Tuning Language Model Distillation

Open paper AskPDF arXiv

Deploying safe large language models (LLMs) on resource-constrained edge devices presents a critical challenge: while dual-model systems combining LLMs with guard models provide effective safety guarantees, their substantial memory and computational demands make them prohibitively expensive for on-device deployment. This paper presents a comprehensive study of parameter-efficient safety alignment methods for resource-constrained settings. Through systematic evaluation across multiple LLM architectures, training objectives, and parameter-efficient fine-tuning approaches, we identify that soft prompts combined with distillation-based training consistently outperform alternative methods. We introduce distillation frameworks based on total variation and KL divergence that effectively transfer safety behaviors from guard models into learned soft prompts. Our evaluations on various benchmarks demonstrate that this combination achieves superior safety-usefulness trade-offs compared to LoRA adapters, steering vectors, and direct optimization methods, while requiring minimal additional memory and compute at inference time. These findings establish soft prompt distillation as the preferred approach for safety alignment in on-device LLM deployment.

Momentum

Related topics

Latest papers 350