LLM Post-Training

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

29 papers in the last four weeks, up 81% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 175

All topics

Forty Shades of Blue: Quality-Diversity Alignment via Mode-Conditioned Reinforcement Learning
Sep 14, 2026Jiayi Yuan, Hangoo Kang, James Jihao Liu +4RL for Language Models Multi-Objective Language Model Alignment
University of Washington · 2Stanford University
Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition
Sep 13, 2026Yecheng Wu, Song Han, Han CaiEfficient Language Model Reasoning Language Model Distillation
Massachusetts Institute of Technology · NVIDIA
EGGROLL, Unrolled: Understanding and Improving Low-Rank Evolution Strategies at Scale
Sep 12, 2026Ege C. Kaya, Abolfazl HashemiEvolutionary Optimization Evolution Strategies
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA
Data-Centric Post-Training for Financial Reasoning: Mining, Distillation, and Verifiable Learning
Sep 9, 2026Zhirayr Hayrapetyan, Andrei Kalmykov, Denis Kokosinskii +2RL for Language Model Reasoning Language Model Distillation
ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation
Sep 8, 2026Yiling Ma, Yilun Zhao, Sihong Wu +3LLM Post-Training Automated Peer Review
Yale University · University of Chicago · TCS Research
Miles v0.1: Production-Level Post-Training
Sep 8, 2026RadixArk, :, Tom Chen +11Reinforcement Learning LLM Agent Training
RadixArk
NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness
Sep 8, 2026NeoHorse Team, Guoliang Cao, Guohao Dai +34Agent Harness Optimization LLM Agent Self-Improvement
TokenRhythm
CROCODIL: Cross-Model Code Editing with LLMs
Sep 3, 2026Linghan Zhong, Aditya Thimmaiah, Jayanth Srinivasa +2Reinforcement Learning LLM Post-Training
1The University of Texas at Austin · 2Cisco Research
Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
Sep 3, 2026Hoang Cuong Nguyen, Mark Dras, Usman NaseemMechanistic Interpretability Circuits in Language Models
Macquarie University, Sydney, Australia
Scaling Near-Optimal SFT-RL Annotation Budget Allocation from Small to Large LLMs
Sep 1, 2026Jingtan Wang, Arun Verma, Xiaoqiang Lin +4Supervised Fine-Tuning Language Model Scaling
Department of Computer Science, National University of Singapore, Republic of Singapore · Singapore-MIT Alliance for Research and Technology Centre, Republic of Singapore · Agency for Science, Technology, and Research (A*STAR), Republic of Singapore +1
From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix
Sep 1, 2026Olga Tsymboi, Dmitrii Stoianov, Ramil Latypov +11LLM Serving Instruction Following
T-Tech
From Rollouts to Recipes: Self-Contained Post-Training for LLMs
Sep 1, 2026Yifei Li, Lingling Zhang, Muye Huang +3Self-Training for Language Models RL for Language Model Reasoning
School of Computer Science and Technology, Xi’an Jiaotong University, China · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · Zhongguancun Academy, Beijing, China +1
CRAFT: Fine-Tuning Pre-hoc Explainability in AI-native 6G RAN
Sep 1, 2026Pranshav Gajjar, Vijay K Shah6G Networks LLM Interpretability
NextG Wireless Lab NCSU Raleigh USA
LLM Post-Training as Brownfield Maintenance: An Industrial Perspective on Dataware Engineering
Aug 31, 2026Gopi Krishnan Rajbahadur, Amir M. Ebrahimi, Boyuan Chen +1Training Data Curation LLM Post-Training
Queen’s University, Canada · Centre for Software Excellence, Huawei Technologies, Canada
PAC: Progress-Augmented Advantage Curriculum for Multi-Task Reinforcement Learning of LLMs
Aug 31, 2026Yuanqiang Yu, Yanzhao Zheng, Zhentao Zhang +8RL for Language Model Reasoning Curriculum RL
Alibaba Group, Hangzhou, China
COGTRL: Training LLMs for Scientific Discovery Assistance using Cognitive Traces via Reinforcement Learning
Aug 31, 2026Shrinidhi Kumbhar Santosh Mashetty Divij Handa Kevin Coutinho, Siddharth Sambhaji Ghule, Chitta BaralRL for Language Model Reasoning LLM Post-Training
Arizona State University
Program Learning with Verifiable Rewards: Symbolic Backpropagation for Post-Training LLMs
Aug 28, 2026Vishvesh BhatLLM-Based Program Synthesis Reinforcement Learning with Verifiable Rewards
CoreThink AI
What is Missing from AI Post-Training AI: An Empirical Analysis
Aug 19, 2026Joy Jia Yin Lim, Xin Huang, Hao Peng +5AI Agent Evaluation LLM Agent Self-Improvement
Tsinghua University · University of Electronic Science and Technology of China · Renmin University of China
Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies
Aug 13, 2026Conor F. Hayes, Elliot Meyerson, Kajetan Schweighofer +4Evolution Strategies LLM Post-Training
1Cognizant AI Lab, San Francisco · 2The University of Texas at Austin, Austin
Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling
Aug 12, 2026Xinmu Ge, Zizhuo Zhang, Yu Huang +8On-Policy Distillation LLM Post-Training
Shanghai Jiao Tong University · Shanghai Innovation Institute · Ant Group +1
Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models
Aug 11, 2026Alexandrine Fortier, Hazel Chen, Peter WestLanguage Model Pretraining LLM Alignment
University of British Columbia
Can Bayesian Optimization Efficiently Find a Strong Single Expert in Neural Thickets?
Aug 11, 2026Nigel Bastian Cendra, Abdelhamid Ezzerg, Fernando Julio Cendra +2Bayesian Optimization LLM Post-Training
University College London · Institut Polytechnique de Paris · University of Oxford
Evaluation-Conditioned Training: Teaching Models to Generalize to Stronger Oversight Regimes
Aug 10, 2026Alec Harris, Kasey Corra, Archie Chaudhury +1RL for Language Models LLM Alignment
AI Safety Initiative at Georgia Tech · University of Chicago · Independent
CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity
Aug 7, 2026Ananya Sahu, Mohit Bansal, Elias Stengel-EskinInstruction Tuning Diverse Text Generation
Columbia · UNC Chapel Hill · University of Texas at Austin
A Six-Dimensional Taxonomy of Post-Training Adaptation Techniques with Applications in AI Governance
Aug 6, 2026Fardin Afdideh, Fernando Seoane, Farhad AbtahiLLM Post-Training
Karolinska Institutet, Department of Clinical Science, Intervention and Technology, Stockholm, Sweden · Karolinska University Hospital, Department of Clinical Physiology, Stockholm, Sweden · University of Borås, Department of Textile Technology, Borås, Sweden +2
SeqLLM: Augmenting LLMs with Behavioral-Sequence Modeling for High-Stakes Decisions at WeChat Pay
Aug 4, 2026Guilin Li, Jiaxing Zhang, Matthias Hwai Yong Tan +2Sequential Recommendation Financial Fraud Detection
Shanghai Jiao Tong University China
Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training
Aug 3, 2026Zhiyuan Wang, Shengcai Liu, Jiahao Wu +5Evolutionary Optimization Evolution Strategies
Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology · Hong Kong Polytechnic University · Hong Kong University of Science and Technology +2
Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model
Aug 3, 2026Ronglong BaoDelta-Rule Linear Attention Language Model Distillation
DT-Project
Wnuan: Staged Post-Training for Question Answering over Proprietary Enterprise Knowledge
Aug 3, 2026Xiaofeng Shi, Xiaosong Qiu, Wenxin Ma +6Supervised Fine-Tuning Knowledge Augmentation for Language Models
Beijing Academy of Artificial Intelligence (BAAI) · Beijing District Heating Group Co., Ltd. (BDHG) · Beijing University of Posts and Telecommunications (BUPT)
Post-Training on Office Work Improves Software Engineering: A Behavioral Account of Cross-Domain Transfer
Aug 3, 2026Logan Ritchie, Sushant Mehta, Liudas Panavas +1Software Engineering Agents Cross-Domain Transfer Learning
Surge AI

Sep 14, 2026, cs.CLJ/K move · Enter open · S save

Forty Shades of Blue: Quality-Diversity Alignment via Mode-Conditioned Reinforcement Learning

Jiayi Yuan, Hangoo Kang, James Jihao Liu, Yejin Choi+3

University of Washington · 2Stanford University

RL for Language Models Multi-Objective Language Model Alignment Quality-Diversity Optimization LLM Post-Training Mode Collapse

Open paper AskPDF arXiv

A notable byproduct of LLM alignment training is mode collapse: the progressive loss of output diversity that narrows a model's expressivity at inference time. This degradation is especially limiting for applications requiring open-ended exploration and pluralistic perspectives, such as scientific ideation and creative writing. We present MoDA (Mode-conditioned Diversity Alignment), an online post-training RL algorithm that jointly optimizes generation quality and diversity, inspired by the coordination perspective in multi-agent reinforcement learning (MARL). MoDA trains a single shared LLM policy conditioned on abstract numbered roles, where each role acts as an agent competing to produce outputs distinct from the others. This formulation encourages mode-conditioned agents to explore complementary regions of the high-quality output space without requiring hand-crafted personas or architectural modifications. MoDA employs a prompt-adaptive quality gating mechanism that calibrates a reference quality threshold and grants diversity rewards only to responses that meet the threshold, preventing reward-hacking behaviors that compromise response quality. To study quality-diversity tradeoffs, we evaluate MoDA on a comprehensive suite of benchmarks spanning seven general capability tasks and four domain-specific diversity tasks in scientific ideation and creative writing. MoDA improves SBERT diversity by 265% on the Infinite-Chat held-out prompts, while increasing average general capability pass@1 by 10.3% over the Qwen3-8B baseline. Compared with the strongest DivPO baseline, MoDA improves SBERT diversity from 0.274 to 0.482 (+75.9%) and E-Vendi from 2.86 to 4.4 (+53.8%), while improving average general capability pass@1 by 7.0%. Overall, MoDA provides a drop-in alternative to standard post-training methods that preserves and expands the model's expressive output space while improving quality.

Momentum

Related topics

Latest papers 175