LLM Post-Training

LLM: Large Language Model

LLM post-training studies how to shape a pretrained model’s behavior and capabilities after its initial training. Common approaches include supervised fine-tuning, preference optimization such as DPO, reinforcement learning with verifiable rewards, and distillation; they use human feedback, synthetic examples, rubrics, or execution results from tools and environments. Researchers evaluate these methods on tasks such as mathematical reasoning, coding, tool use, and domain-specific question answering, while also measuring safety, calibration, and performance after deployment. A central challenge is gaining targeted abilities without sacrificing prior capabilities, diversity, or reliability, and evidence about which training recipes generalize best remains mixed.

Latest papers

249 papers43 papers in the last four weeks, up 19% on the four weeks before. 0.2% of all new papers.

Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs
Apr 12, 2026Yu Li, Xiaoran Shang, Qizhi Pei +11Training Data Curation Data Provenance
Shanghai Artificial Intelligence Laboratory, OpenDataLab · University of Science and Technology of China · Shanghai Jiao Tong University
Efficient RL Training for LLMs with Experience Replay
Apr 9, 2026Charles Arnal, Vivien Cabannes, Taco Cohen +2Experience Replay RL Post-Training
FAIR at Meta · NYU Courant Institute and CDS
Awakening the Sleeping Agent: Lean-Specific Agentic Data Reactivates General Tool Use in Goedel Prover
Apr 9, 2026Jui-Hui Chung, Hongzhou Lin, Lai Jiang +2LLM Tool Use LLM Post-Training
Princeton Language and Intelligence · Amazon · Department of Computer Science, University of Illinois at Urbana-Champaign
Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
Apr 9, 2026Shiwan Zhao, Zhihu Wang, Xuyang Zhao +10LLM Post-Training RL for Language Models
Nankai University, China · Huawei Technologies Ltd., China
The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training
Apr 9, 2026Rui Zhang, Hongwei Li, Yun Shen +6LLM Alignment LLM Safety Alignment
University of Electronic Science and Technology of China · Flexera · CISPA Helmholtz Center for Information Security +1
Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning
Apr 8, 2026Rohan Surana, Gagan Mundada, Xunyi Jiang +19RL for Language Model Reasoning RL for Language Models
University of California, San Diego · University of Toronto · UIUC +3
OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learning
Apr 6, 2026Jingkai He, Pengfei Chen, Chenghui Wu +7LLM Post-Training LLM Fine-Tuning
School of Systems Science and Engineering, Sun Yat-sen University, Guangzhou, China · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Alibaba Cloud Computing, Hangzhou, China
Forecasting Supply Chain Disruptions with Foresight Learning
Apr 1, 2026Benjamin Turtel, Paul Wilczewski, Kris SkotheimProbabilistic Forecasting Supply Chain Management
Lightning Rod Labs
CoLLM: Continuous Adaptation for SLO-Aware LLM Serving on Shared GPU Clusters
Mar 31, 2026Shaoyuan Huang, Yunfeng Zhao, Na Yan +5LLM Post-Training LLM Inference Scheduling
Tianjin University, Tianjin, China · King’s College London, London, United Kingdom · Paiou Cloud Computing (Shanghai) Company, Ltd., Shanghai, China
RTLSeek: Boosting the LLM-Based RTL Generation with Multi-Stage Diversity-Oriented Reinforcement Learning
Mar 29, 2026Xinyu Zhang, Zhiteng Chao, Yonghao Wang +6RTL Code Generation LLM Post-Training
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences, China · University of Chinese Academy of Sciences, China · Peking University, China +1
Reasoning over mathematical objects: on-policy reward modeling and test time aggregation
Mar 19, 2026Pranjal Aggarwal, Marjan Ghazvininejad, Seungone Kim +18LLM Mathematical Reasoning Mathematical Reasoning Benchmarks
FAIR at Meta · CMU · UCLA
Automatic Configuration of LLM Post-Training Pipelines
Mar 19, 2026Channe Chwa, Xinle Wu, Yao LuLLM Post-Training Hyperparameter Optimization
National University of Singapore
Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalation
Mar 18, 2026Philipp Normann, Andreas Happe, Jürgen Cito +1LLM Agent Security LLM Post-Training
TU Wien, Vienna, Austria
Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modeling
Mar 18, 2026Junzhuo Ma, Chenghuang Shen, Yi Yu +15LLM Agent Training LLM Post-Training
School of Mathematical Sciences, Fudan University, Shanghai, China · Alibaba Group, Hangzhou, China · Shanghai Center for Mathematical Sciences, Fudan University, Shanghai, China +3
Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Models
Mar 14, 2026Haitao Jiang, Wenbo Zhang, Jiarui Yao +3LLM Post-Training Supervised Fine-Tuning
North Carolina State University · UC Irvine · University of Illinois Urbana-Champaign +2
Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights
Mar 12, 2026Yulu Gan, Phillip IsolaLLM Post-Training Weight-Space Analysis
MIT CSAIL
A Quantitative Characterization of Forgetting in Post-Training
Mar 12, 2026Krishnakumar Balasubramanian, Shiva Prasad KasiviswanathanLLM Post-Training Catastrophic Forgetting
Department of Statistics, University of California, Davis · Amazon
Maximizing Mutual Information Between Prompt and Response Improves LLM Performance With No Additional Data
Mar 10, 2026Hyunji Nam, Haoran Li, Natasha JaquesLLM Personalization Mutual Information Maximization
Stanford University · Character AI · University of Washington
Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanisms
Mar 10, 2026Mingjie Li, Wai Man Si, Michael Backes +2LLM Safety LLM Safety Alignment
CISPA Helmholtz Center for Information Security · State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Institute for Artificial Intelligence, Peking University
PostTrainBench: Can LLM Agents Automate LLM Post-Training?
Mar 9, 2026Ben Rank, Hardik Bhatnagar, Ameya Prabhu +4LLM Post-Training Autonomous Research Agents
ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center +2
EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Training
Mar 2, 2026Aleksei Dorkin, Taido Purason, Emil Kalbaliyev +7Multilingual Language Models Language Model Pretraining
Institute of Computer Science, University of Tartu, Tartu, Estonia · Department of Software Science, Tallinn University of Technology, Tallinn, Estonia · Institute of the Estonian Language, Tallinn, Estonia +1
CharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Production
Mar 2, 2026Yixin Nie, Lin Guan, Zhongyao Ma +19LLM Post-Training Conversational Agents
Meta Superintelligence Labs · OpenAI · xAI +1
Surgical Post-Training: Proximal On-Policy Distillation for Reasoning with Knowledge Retention
Mar 2, 2026Wenye Lin, Kai HanLLM Post-Training LLM Reasoning
The University of Hong Kong
GraphScout: Empowering Large Language Models with Intrinsic Exploration Ability for Agentic Graph Reasoning
Mar 2, 2026Yuchen Ying, Weiqi Jiang, Tongya Zheng +4LLM Reasoning with Graphs Agentic RAG
Zhejiang University, Hangzhou, China · Hangzhou City University, Hangzhou, China · Nanyang Technological University, Singapore, Singapore
Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction
Feb 26, 2026Nils Schwager, Simon Münker, Alistair Plum +1Large Language Model-Based Social Simulation Language Model Generation Evaluation
Trier University, Trier, Germany · University of Luxembourg, Esch-sur-Alzette, Luxembourg
Replacing Multi-Step Assembly of Data Preparation Pipelines with One-Step LLM Pipeline Generation for Table QA
Feb 26, 2026Fengyu Li, Junhao Zhu, Kaishi Song +4Table QA LLM Post-Training
Zhejiang University Hangzhou, China · Northeastern University Shenyang, China · Aalborg University Aalborg, Denmark
CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training
Feb 19, 2026Lukas Thede, Stefan Winzeck, Zeynep Akata +1LLM Post-Training Catastrophic Forgetting
Thomson Reuters Foundational Research, 5 Canada Square, London · Tübingen AI Center, University of Tübingen · Helmholtz Munich +2
SWE-Tester: Training Open-Source LLMs for Issue Reproduction in Real-World Repositories
Jan 20, 2026Aditya Bharat Soni, Rajat Ghosh, Vaishnavi Bhargava +2LLM Post-Training Automated Test Generation
Carnegie Mellon University · Nutanix · Stanford University
Unified Text-Image Generation with Weakness-Targeted Post-Training
Jan 7, 2026Jiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han +7Unified Multimodal Models T2I Generation
FAIR at Meta · University of Texas, Austin · Work done at Meta +2
Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning
Nov 26, 2025Zhenchao Tang, Fang Wang, Haohuai He +13LLM Alignment LLM Fine-Tuning
Tencent AI for Life Sciences Lab, Shenzhen, China. · Sun Yat-sen University, Shenzhen, China. · The Hong Kong Polytechnic University, Hong Kong SAR, China. +2

Apr 12, 2026J/K move · Enter open · S save

Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs

Yu Li, Xiaoran Shang, Qizhi Pei, Yun Zhu+10

Shanghai Artificial Intelligence Laboratory, OpenDataLab · University of Science and Technology of China · Shanghai Jiao Tong University

Training Data Curation Data Provenance LLM Post-Training Data Contamination in Language Models

Open paper AskPDF arXiv

Post-training data plays a pivotal role in shaping the capabilities of Large Language Models (LLMs), yet datasets are often treated as isolated artifacts, overlooking the systemic connections that underlie their evolution. To disentangle these complex relationships, we introduce the concept of data lineage to the LLM ecosystem and propose an automated multi-agent framework to reconstruct the evolutionary graph of dataset development. Through large-scale lineage analysis, we characterize domain-specific structural patterns, such as vertical refinement in math-oriented datasets and horizontal aggregation in general-domain corpora. Moreover, we uncover pervasive systemic issues, including structural redundancy induced by implicit dataset intersections and the propagation of benchmark contamination along lineage paths. To demonstrate the practical value of lineage analysis for data construction, we leverage the reconstructed lineage graph to create a lineage-aware diversity-oriented dataset. By anchoring instruction sampling at upstream root sources, this approach mitigates downstream homogenization and hidden redundancy, yielding a more diverse post-training corpus. We further highlight lineage-centric analysis as an efficient and robust topological alternative to sample-level dataset comparison for large-scale data ecosystems. By grounding data construction in explicit lineage structures, our work advances post-training data curation toward a more systematic and controllable paradigm.