LLM Post-Training

LLM: Large Language Model

LLM post-training studies how to shape a pretrained model’s behavior and capabilities after its initial training. Common approaches include supervised fine-tuning, preference optimization such as DPO, reinforcement learning with verifiable rewards, and distillation; they use human feedback, synthetic examples, rubrics, or execution results from tools and environments. Researchers evaluate these methods on tasks such as mathematical reasoning, coding, tool use, and domain-specific question answering, while also measuring safety, calibration, and performance after deployment. A central challenge is gaining targeted abilities without sacrificing prior capabilities, diversity, or reliability, and evidence about which training recipes generalize best remains mixed.

Latest papers

249 papers43 papers in the last four weeks, up 19% on the four weeks before. 0.2% of all new papers.

Convex Dataset Valuation for Post-Training
May 15, 2026Siqi Zeng, Christopher Jung, Rui Li +7LLM Post-Training Data Valuation
Department of Computer Science, University of Illinois Urbana-Chamapign, Urbana, IL, USA · Meta, Menlo Park, CA, USA
GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training
May 13, 2026Junjie Li, Ziao Wang, NingXuan Ma +2Training Data Selection LLM Post-Training
Harbin Institute of Technology, Shenzhen, China · Hong Kong Baptist University, China
Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators
May 12, 2026Heejin Do, Shashank Sonkar, Mrinmaya SachanLLM Evaluation LLM Sycophancy
ETH Zürich, ETH AI Center · University of Central Florida · ETH Zürich
Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
May 12, 2026Chengqian Zhang, Wei Zhu, Kyumin LeeLow-Rank Adaptation LLM Post-Training
Worcester Polytechnic Institute · University of Hong Kong
ConFit v3: Improving Resume-Job Matching with LLM-based Re-Ranking
May 10, 2026Xiao Yu, Ruize Xu, Chengyuan Xue +6Reinforcement Learning Learning to Rank
Columbia University · New York, NY, USA · John Hopkins University +6
Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training
May 10, 2026Yuanyi Wang, Yifan Yang, Su Lu +9Continual Learning for LLMs Model Merging
The Hong Kong Polytechnic University, PolyU · The Hong Kong University of Science and Technology, HKUST · PolyU-Daya Bay Technology and Innovation Research Institute
LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models
May 10, 2026Taekhyun Park, Yongjae Lee, Dohee Kim +1Recurrent Transformers LLM Post-Training
Department of Data Science Pusan National University Busan, Republic of Korea · Department of Industrial Engineering Pusan National University Busan, Republic of Korea · Department of Artificial Intelligence Engineering Changwon National University Changwon, Republic of Korea
On Distinguishing Capability Elicitation from Capability Creation in Post-Training: A Free-Energy Perspective
May 8, 2026Yuhao Li, Shengchao LiuSupervised Fine-Tuning Free Energy Principle
Department of Computer Science and Engineering The Chinese University of Hong Kong
Star Elastic: Many-in-One Reasoning LLMs with Efficient Budget Control
May 8, 2026Ali Taghibakhshi, Ruisi Cai, Saurav Muralidharan +17LLM Compression LLM Post-Training
NVIDIA
Self-Consolidating Language Models: Continual Knowledge Incorporation from Context
May 8, 2026Zekun Wang, Anant Gupta, Zihan Dong +1Continual Learning for LLMs Long-Context Language Modeling
Georgia Institute of Technology
Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training
May 8, 2026Pingbang Hu, Xueshen Liu, Z. Morley Mao +1Supervised Fine-Tuning Training Data Selection
University of Illinois Urbana–Champaign · University of Michigan
Navigating by Old Maps: The Pitfalls of Static Mechanistic Localization in LLM Post-Training
May 7, 2026Hang Chen, Jiaying Zhu, Hongyang Chen +3Supervised Fine-Tuning Transformer Interpretability
School of Computer Science and Technology Xi’an Jiaotong University · School of Computer Science and Engineering The Chinese University of Hong Kong · Shaanxi Co., Ltd(Xi’an 710077, China) China Mobile Group +1
Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training
May 6, 2026Hengyu Shi, Tianyang Han, Peizhe Wang +3Efficient Language Model Training LLM Post-Training
1Independent Researcher · 2D4 Lab · 3Southeast University
Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
May 4, 2026Ishaan Watts, Catherine Li, Sachin Goyal +2Language Model Pretraining Sharpness-Aware Minimization
Carnegie Mellon University
Iterative Finetuning is Mostly Idempotent
May 1, 2026Zephaniah Roe, Jack Sanderson, Dang Nguyen +5Supervised Fine-Tuning LLM Post-Training
University of Chicago
Teaching LLMs Brazilian Healthcare: Injecting Knowledge from Official Clinical Guidelines
May 1, 2026Hugo Abonizio, Filipe Rocha Lopes, Roberto Lotufo +1Knowledge Augmentation for Language Models Healthcare
FEEC, UNICAMP, Campinas, Brazil · NoHarm.ai, Brazil · Maritaca AI, Brazil
Decouple before Integration: Test-time Synthesis of SFT and RLVR Task Vectors
May 1, 2026Chaohao Yuan, Chenghao Xiao, Yu Rong +2Task Arithmetic Language Model Merging
Department of Systems Engineering and Engineering Management, Chinese University of Hong Kong, Hong Kong, China · DAMO Academy, Alibaba Group, Hangzhou, China · Hupan Lab, Hangzhou, China +1
Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers
Apr 28, 2026Jan Dubiński, Jan Betley, Anna Sztyber-Betley +2Language Model Safety Evaluation Emergent Misalignment in Language Models
1Warsaw University of Technology · 4Truthful AI · University College London +2
Compute Aligned Training: Optimizing for Test Time Inference
Apr 27, 2026Adam Ousherovitch, Ambuj TewariTest-Time Optimization Inference-Time Scaling
Department of Statistics University of Michigan
SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
Apr 26, 2026Alexis Limozin, Eduard Durech, Torsten Hoefler +2RL for Language Model Reasoning Policy Optimization
ETH AI Center, ETH Zürich · EPFL · Allen Institute for AI
Training a General Purpose Automated Red Teaming Model
Apr 24, 2026Aishwarya Padmakumar, Leon Derczynski, Traian Rebedea +1Adversarial Attacks on LLMs LLM Red Teaming
NVIDIA
Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Text
Apr 21, 2026Chengyu Huang, Sheng-Yen Chou, Zhengxin Zhang +1Self-Play RL LLM Post-Training
Department of Computer Science, Cornell University
Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning
Apr 19, 2026Zhiyin Yu, Bo Zhang, Qibin Hou +3Self-Training for Language Models RL for Language Model Reasoning
Peking University · Shanghai Artificial Intelligence Laboratory · Nankai University +1
AutoOR: Scalably Post-training LLMs to Autoformalize Operations Research Problems
Apr 18, 2026Sumeet Ramesh Motwani, Chuan Du, Aleksander Petrov +4LLM Post-Training RL Fine-Tuning
1X · University of Oxford
Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Design
Apr 17, 2026Shriram Chennakesavalu, Kirill Shmilovich, Hayley Weir +5LLM Evaluation Molecular Property Prediction
Prescient Design, Genentech, South San Francisco, CA, USA
Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO
Apr 17, 2026Arash Ahmadi, Parisa Masnadi Khiabani, Sarah Sharif +4RL for Language Models Healthcare
School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK, USA · Intelligent Neuromorphic and Quantum Understanding for Innovative Research and Engineering (INQUIRE) Laboratory, University of Oklahoma, Norman, OK, USA · Data Science and Analytics Institute, University of Oklahoma, Norman, OK, USA +3
SFT-GRPO Data Overlap as a Post-Training Hyperparameter for Autoformalization
Apr 15, 2026Xiaole Su, Kasey Zhang, Andy LyuLanguage Model Post-Training LLM Post-Training
Osmosis AI
Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillation
Apr 14, 2026Yecheng Wu, Song Han, Han CaiOn-Policy Distillation LLM Post-Training
NVIDIA
GoodPoint: Learning Constructive Scientific Paper Feedback from Author Responses
Apr 13, 2026Jimin Mun, Chani Jung, Xuhui Zhou +2LLM Post-Training Automated Peer Review
Carnegie Mellon University · Independent Researcher · NVIDIA
Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?
Apr 12, 2026Wanyi Chen, Xiao Yang, Xu Yang +7Agentic RL AI Agent Benchmarks
Soochow University · Microsoft Research Asia · Stony Brook University +2

May 15, 2026J/K move · Enter open · S save

Convex Dataset Valuation for Post-Training

Siqi Zeng, Christopher Jung, Rui Li, Zhe Kang+6

Department of Computer Science, University of Illinois Urbana-Chamapign, Urbana, IL, USA · Meta, Menlo Park, CA, USA

LLM Post-Training Data Valuation

Open paper AskPDF arXiv

Improving LLM performance on downstream tasks sometimes requires leveraging auxiliary datasets during post-training. In practice, however, developers face constraints on compute, labeling, and licensing costs that preclude using all available data, necessitating principled dataset-level selection. These constraints are increasingly shaped by dataset marketplaces, where data acquisition is governed by budgets and negotiation. We study dataset valuation as a subset selection problem during LLM post-training. Our goal is to identify and weight auxiliary datasets so as to maximize target task performance given constrained budgets. We first show that commonly used gradient alignment scores provide a reasonable yet incomplete valuation signal, as they ignore redundancy among datasets. To address this, we propose a scalable convex dataset-level valuation method based on kernel mean matching (KMM) in gradient space, which jointly accounts for alignment with the target task and redundancy across auxiliary datasets. Through extensive experiments across diverse post-training settings and tasks, we show that our approach consistently outperforms existing valuation baselines, achieving stronger performance with low computational overhead. Our results position dataset valuation as a practical decision tool for post-training data selection in market-constrained large language model settings. The code is available at https://github.com/uiuctml/convex_data_valuation.