Language Model Post-Training

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 283% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 132

All topics

The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillation
Apr 18, 2026Jiaxin Zhang, Xiangyu Peng, Qinglin Chen +3On-Policy Self-Distillation Language Model Calibration
Salesforce AI Research
Where does output diversity collapse in post-training?
Apr 17, 2026Constantinos Karouzos, Xingwei Tan, Nikolaos AletrasLanguage Model Post-Training LLM Training
School of Computer Science University of Sheffield, UK
Peer-Predictive Self-Training for Language Model Reasoning
Apr 14, 2026Shi Feng, Hanlin Zhang, Fan Nie +2Self-Training for Language Models Language Model Post-Training
Harvard University · Stanford University
English is Not All You Need: Systematically Exploring the Role of Multilinguality in LLM Post-Training
Apr 14, 2026Mehak Dhaliwal, Shashwat Chaurasia, Yao Qin +2Cross-Lingual Reasoning in Language Models Low-Resource Language Processing
UC Santa Barbara · Amazon
How Does "English (US)" Become the Default? Triangulating Structural Bias Towards American English Across the LLM Pipeline
Apr 5, 2026Mir Tafseer Nayeem, Davood RafieiLanguage Model Pretraining LLM Auditing
Department of Computing Science University of Alberta
GRRM: Group Relative Reward Modeling for Machine Translation
Feb 15, 2026Sen Yang, Shanbo Cheng, Lu Xu +2Reward Modeling RL for Language Models
National Key Laboratory for Novel Software Technology, Nanjing University · Singapore University of Technology and Design
LMSpell: Spell Correction with Pre-Trained Language Models
Dec 5, 2025Akesh Gunathilake, Nadil Karunarathna, Tharusha Bandaranayake +3Low-Resource Language Processing Language Model Post-Training
Department of Computer Science and Engineering, University of Moratuwa, Katubedda, 10400, Sri Lanka · School of Mathematical and Computational Sciences, Massey University, Auckland, 102904, New Zealand
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
Oct 17, 2025Yizhuo Chen, Xin Liu, Ruijie Wang +7LLM Personalization Personalized Language Models
University of Illinois Urbana-Champaign · Amazon · University of Notre Dame
Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity
Oct 1, 2025Jiayi Zhang, Simon Yu, Derek Chong +4LLM Prompting Model Collapse
Northeastern University · Stanford University · West Virginia University
Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment
Aug 6, 2025Aleksander Boruch-Gruszecki, Yangtian Zi, Zixuan Wu +4RL for Code Generation Code Generation
1Northeastern University · University of Texas · 3Wellesley College
BOW: Training Language Models to Reason Over Plausible Next Words
Jun 16, 2025Ming Shen, Zhikun Xu, Jacob Dineen +2RL for Language Model Reasoning Next-Token Prediction
Arizona State University
LearNAT: Learning NL2SQL with AST-guided Task Decomposition for Large Language Models
Apr 3, 2025Weibin Liao, Xin Gao, Tianyu Jia +6LLM Planning RL for Language Model Reasoning
School of Computer Science, Peking University · 2Key Laboratory of High Confidence Software Technologies, Ministry of Education · School of Computer Sciences, Beijing University of Posts and Telecommunications +5

Apr 18, 2026, cs.LGJ/K move · Enter open · S save

The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillation

Jiaxin Zhang, Xiangyu Peng, Qinglin Chen, Qinyuan Ye+2

Salesforce AI Research

On-Policy Self-Distillation Language Model Calibration On-Policy Distillation Language Model Post-Training

Open paper AskPDF arXiv

On-policy distillation (OPD) is an increasingly important paradigm for post-training language models. However, we identify a pervasive Scaling Law of Miscalibration: while OPD effectively improves task accuracy, it systematically traps models in severe overconfidence. We trace this failure to an information mismatch: teacher supervision is formed under privileged context available during training, whereas the deployed model must report confidence using only deployment-time information. We formalize this perspective theoretically, showing that teacher-conditioned success is generally not a valid target for deployment-time confidence and that helpful privileged context induces entropy collapse and a systematic optimism bias. To address this, we propose a calibration-aware OPD framework, CaOPD, that estimates empirical confidence from model rollouts, replaces self-reported confidence with this student-grounded target, and distills the revised response through the same self-distillation pipeline. Experiments across various models and domains show that CaOPD achieves Pareto-optimal calibration while maintaining competitive capability, generalizing robustly under out-of-distribution and continual learning. Our findings highlight that capability distillation does not imply calibrated confidence, and that confidence should be treated as an essential objective in post-training. Code: https://github.com/SalesforceAIResearch/CaOPD

Momentum

Related topics

Latest papers 132