LLM Post-Training

LLM: Large Language Model

LLM post-training studies how to shape a pretrained model’s behavior and capabilities after its initial training. Common approaches include supervised fine-tuning, preference optimization such as DPO, reinforcement learning with verifiable rewards, and distillation; they use human feedback, synthetic examples, rubrics, or execution results from tools and environments. Researchers evaluate these methods on tasks such as mathematical reasoning, coding, tool use, and domain-specific question answering, while also measuring safety, calibration, and performance after deployment. A central challenge is gaining targeted abilities without sacrificing prior capabilities, diversity, or reliability, and evidence about which training recipes generalize best remains mixed.

Latest papers

246 papers43 papers in the last four weeks, up 19% on the four weeks before. 0.2% of all new papers.

How Does Post-Training Shape LLM Value Preferences?
Oct 23, 2026Yang Liu, Shaoyang Xu, Masahiro Kaneko +3LLM Post-Training Social Bias in Language Models
Kyoto University · Singapore University of Technology and Design · Mohamed bin Zayed University of Artificial Intelligence
Building Presentation Intelligence for High-Quality Slide Generation
Oct 22, 2026Yuhang Xie, Cheng Li, Chenliang Li +6LLM Post-Training
Peking University · Alibaba Group
Monitorability Is Trainable: Post-Training for Chain-of-Thought Monitorability
Oct 21, 2026Usman Anwar, Tim Bakker, Christos Louizos +1CoT Monitoring LLM Post-Training
University of Cambridge · Qualcomm AI Research · Mila, University of Montreal
The Shape of the DPO Hyperparameter Landscape: What to Measure, How It Moves with Scale, and When Not to Transfer It
Oct 20, 2026Hannan Mahadik, David Salinas, Jacek Golebiowski +1Direct Preference Optimization Hyperparameter Optimization
ELLIS Institute Tübingen Germany · ELLIS Institute Tübingen & Prior Labs Germany · distil labs Germany
Typed Decision Models under Prevalence Shift: Decision Training, Calibration, and Implicit Priors
Oct 19, 2026Abdul Rahman Iqbal, Muhammad Fawad Nasim, Amir SaleemLanguage Model Calibration Model Calibration
Faculty of Computer Science and Information Technology, The Superior University, Lahore, Pakistan · Independent Researcher
UpLoop: Upcycling Large Language Models with Gated Loop
Oct 18, 2026Tieyuan Chen, Huabin Liu, Yingtao Zhang +4Recurrent Language Models Recurrent Transformers
Inclusion AI, Ant Group · Shanghai Jiao Tong University · Tsinghua University
Coopshift: Post-Training Amplifies The Memory Curse In LLM Cooperation
Oct 16, 2026Jiayuan Liu, Shiyi Du, Tianqin Li +3LLM Post-Training Social Dilemmas
Carnegie Mellon University · Foundations of Cooperative AI Lab (FOCAL) · The University of Texas at Austin
The Ephemeral Cost of Switching Optimizers Across LLM Training Stages
Oct 15, 2026Xinyu Zhao, Davis Wertheimer, Ashutosh Sharma +2LLM Training LLM Post-Training
MIT · MIT-IBM Watson AI Lab
Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe
Oct 15, 2026Soumye Singhal, Joseph Jennings, Ka Shun Shum +11LLM Training LLM Post-Training
NVIDIA
Incentivizing Tool-Use Reinforcement Learning for Post-Arrival Emergency Care Prediction
Oct 15, 2026Pengfei Hu, Chenwei Wu, Qiao Jin +4Clinical Outcome Prediction Clinical Risk Prediction
Stevens Institute of Technology · University of Michigan, Ann Arbor · National Institutes of Health +1
Athanor: An Agent-Native System for Autonomous Post-Training Research
Oct 15, 2026Yufa Zhou, Yichen Wang, Jiakun Fan +5LLM Post-Training Language Model Post-Training
ByteDance · Duke University · The University of Chicago +1
One Task, Many Interfaces: Simple Cross-Interface Consistency Training Improves LLM Agent Robustness
Oct 14, 2026Artin Tajdini, Manpreet Kaur, Peng QiLLM Post-Training Tool Use
University of Washington · Uniphore
LLM Agents for Automated Post-Training via Structured Recipe Optimization
Oct 13, 2026Ruohong Liu, Bohan Tang, Lekang Jiang +2LLM Post-Training Large Language Model-Guided Optimization
LIGHTSPEED · University of Cambridge · Independent Researcher
Efficient Resource-Shiftable RL Post-training with Generative Rewards
Oct 12, 2026Yanfeng Zhao, Haoyang Li, Sheng Lin +4RL Post-Training LLM Post-Training
Shanghai Jiao Tong University · Peking University · Central China Normal University
WildTrace: Evaluating Training Data Detection in Realistic Large-Scale LLM Post-Training
Oct 12, 2026Hengxiang Zhang, Junxian Liu, Hongxin WeiData Contamination in Language Models LLM Evaluation
Department of Statistics and Data Science, Southern University of Science and Technology
MoMaTune: Model, Match, Then Tune Language Models for Numerical Records
Oct 11, 2026Weimin Lyu, Jingfen Zhang, Lingdao Sha +4LLM Post-Training Incomplete Data Imputation
Amazon
Vela: User Behavior as Supervision Sparks LLM Intelligence
Oct 10, 2026Yang Xiao, Yusong Sun, Haoyi Wu +17LLM Training LLM Post-Training
Continually Post-training Small Models via Synthetic Self-distillation
Oct 9, 2026Woosung Koh, Hyunsoo Lee, Kyungjae Lee +4LLM Post-Training Continual Learning for LLMs
KAIST AI · Work done during an internship at LG AI Research · LG AI Research
HarnessSQL: Harness-Native Training for SQL Agents in Realistic Database Environments
Oct 8, 2026Haolin Yang, Jipeng Zhang, Zhiqiang Qian +4Text-to-SQL LLM Agent Training
Hong Kong University of Science and Technology · Microsoft Research · Tsinghua University +1
Harness Evolution Hits a Ceiling: When Weight Training Should Begin
Oct 8, 2026Yuan Tian, Bing Hu, Hao Wang +2Long-Horizon LLM Agents LLM Agent Harnesses
Independent Researcher · University of California, Berkeley · Purdue University +1
Successive Training Stages and Large Language Model Persuasion: Effects of Misalignment, Supervised Fine-Tuning, and Preference Optimization
Oct 7, 2026Antony Dalmiere, Pascal Marchand, Guillaume Auriol +1Persuasion in Language Models LLM Post-Training
LAAS-TRUST · LAAS-TRUST, INSA Toulouse · LAAS-TSF, LAAS
SAPD: Step-Aligned Privileged Distillation
Oct 7, 2026Tianle Wang, Jiayu Liu, Ruizhi Zhao +1LLM Post-Training Language Model Distillation
Department of Data Science, City University of Hong Kong · Hong Kong Institute of AI for Science, City University of Hong Kong · Shenzhen University
When Rank Rises as LLMs Degrade
Oct 7, 2026Zhaohui Geoffrey WangLLM Post-Training Representation Geometry in Language Models
University of Southern California
CM-DPO: Constraint-Margin Direct Preference Optimization for LLM Planning
Oct 6, 2026Rabimba Karanjai, Qun Gu, Hemanth Hegadehalli Madhavarao +8Direct Preference Optimization LLM Planning
PayPal AI Lab
Harmful SFT Leaves a Continuous Trace in LLM Checkpoint Updates
Oct 5, 2026Ziqun Bao, Xinyu Zhang, Yuchen Shao +1LLM Auditing LLM Safety
SEI, East China Normal University
The Assistance Dilemma: Learning to Teach via Multi-Turn Reinforcement Learning
Oct 5, 2026Jakub Macina, Manu Kapur, Mrinmaya SachanRL for Language Models Intelligent Tutoring Systems
ETH Zurich
ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvement
Oct 5, 2026Xingbo Yao, Xiaoman Wang, Zhengwu Lei +11LLM Fine-Tuning LLM Post-Training
HKUST(GZ) · Xiaohongshu Inc. · ECNU +3
Dynamic Minimax Regret Optimization for Robust LLM Post-Training
Oct 5, 2026Chengbo Zang, Haoyu Dong, Mehmet Kerem Turkcan +3Group Distributionally Robust Optimization Dynamic Regret Minimization
Department of Electrical Engineering, Columbia University · Department of Computer Science & Engineering, University of California, Santa Cruz · Department of Civil Engineering, Columbia University
The Functional Structure of Post-Compression Recovery in Low-Rank LLMs
Oct 4, 2026Zishan Shao, Liang Tian, Georgiy Zemlevskiy +10LLM Compression Low-Rank Compression
Duke University · Wake Forest University · Carnegie Mellon University +2
Pareto-Dominant Clarification: Post-Training Coding LLMs via PPO-Lagrangian Budget Constraints
Oct 2, 2026Abhinav Rajput, Acey VogelsteinLLM Post-Training RL for Code Generation
Center for Data Science New York University New York, NY, USA

Oct 23, 2026J/K move · Enter open · S save

How Does Post-Training Shape LLM Value Preferences?

Yang Liu, Shaoyang Xu, Masahiro Kaneko, Chengzhi Zhong+2

Kyoto University · Singapore University of Technology and Design · Mohamed bin Zayed University of Artificial Intelligence

LLM Post-Training Social Bias in Language Models Language Model Bias Evaluation RL Post-Training

Open paper AskPDF arXiv

During post-training, large language models (LLMs) learn value preferences from corpora that may contain imbalanced proportions of opinions associated with different demographic groups. Amplifying these imbalances may further underrepresent some groups' opinions. However, how opinion mixtures shape value preferences across post-training paths remains unclear. To answer this question, we conduct controlled experiments using post-training corpora with different opinion mixtures and five post-training paths based on SFT, DPO, and GRPO. We find that post-training amplifies opinion imbalances differently across post-training paths. Once value preferences are amplified, whether further post-training can remove them depends on both the post-training method and the opinion mixture in the corpus. More concerningly, a model's stability under interventions depends on whether further post-training changes its value preference. Moreover, these value preferences are transferred to student models even when distillation uses corpora with no explicit opinion questions. These findings suggest that both the opinion mixture in post-training corpora and the choice of post-training method should be considered when developing, evaluating, and reusing LLMs. Our code and data are available at https://github.com/ku-nlp/llm-value-preferences.