RL for Language Models

RL: Reinforcement Learning

This field studies how feedback can shape a language model’s behavior after pretraining, including how it answers, reasons, uses tools, and handles uncertainty. Common approaches include preference-based reinforcement learning from human or AI feedback, reinforcement learning with verifiable rewards for tasks such as math and coding, and policy methods such as PPO and GRPO; rubrics and step- or turn-level rewards provide finer guidance. Researchers evaluate these methods on reasoning benchmarks as well as dialogue, tool use, multilingual tasks, and domain-specific applications. A central tension is that sparse or imperfect rewards can be costly to obtain and easy to exploit, while reliable credit assignment, exploration, and generalization beyond the training setting remain difficult.

Latest papers

445 papers63 papers in the last four weeks, up 17% on the four weeks before. 0.4% of all new papers.

Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective
Mar 9, 2026Liyuan Mao, Le Yu, Jing Zhou +7Inference-Time Language Model Alignment RL for Language Models
Qwen Team, Alibaba Group · Shanghai Jiao Tong University
Fibration Policy Optimization
Mar 9, 2026Chang Li, Tshihao Tsu, Yaren Zhang +2Policy Optimization RL for Language Models
JD Explore Academy · Carleton University
Helix: Evolutionary Reinforcement Learning for Open-Ended Scientific Problem Solving
Mar 8, 2026Chang Su, Zhongkai Hao, Zhizhou Zhang +4Evolutionary RL Reinforcement Learning
Bosch (China) Investment Co., Ltd. · Tsinghua-Bosch Joint Center for ML, Tsinghua University · Dept. of Comp. Sci. & Techn., Tsinghua University +1
Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training
Mar 7, 2026Chuxue Cao, Honglin Lin, Zhanping Zhong +5Financial Language Model Evaluation LLM Training
Shanghai Artificial Intelligence Laboratory, OpenDataLab, OpenDataArena · Hong Kong University of Science and Technology
Post-Training with Policy Gradients: Optimality and the Base Model Barrier
Mar 7, 2026Alireza Mousavi-Hosseini, Murat A. ErdogduLanguage Model Post-Training Policy Gradient
University of Toronto, Vector Institute. · University of Toronto, Vector Institute, and Numurho.
MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialogue
Mar 6, 2026Naifan Zhang, Ruihan Sun, Jinwei Su +4RL for Language Models Credit Assignment in RL
NatureSelect · Tsinghua University · South China Normal University +1
Partial Policy Gradients for RL in LLMs
Mar 6, 2026Puneet Mathur, Branislav Kveton, Subhojyoti Mukherjee +1RL for Language Models Policy Gradient Methods
Adobe Research.
Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoning
Mar 6, 2026Xuan Li, Zhanke Zhou, Zongze Li +4Molecular Optimization Large Language Model-Guided Optimization
Hong Kong Baptist University · CMIC, Shanghai Jiao Tong University · DAMO Academy, Alibaba Group +1
Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation
Mar 6, 2026Changcheng Li, Jiancan Wu, Hengheng Zhang +5Confidence Estimation in Language Models LLM Uncertainty Estimation
University of Science and Technology of China · Huawei Inc.
BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning
Mar 5, 2026Yuan Li, Bo Wang, Yufei Gao +4RL for Language Models Proximal Policy Optimization
Fudan University · Shanghai Innovation Institute
Bootstrapping Exploration with Group-Level Natural Language Feedback in Reinforcement Learning
Mar 4, 2026Lei Huang, Xiang Cheng, Chenxiao Zhao +6RL Exploration RL for Language Models
Harbin Institute of Technology · Xiaohongshu Inc.
Why Does RLAIF Work At All?
Mar 3, 2026Robin YoungLLM Alignment RL for Language Models
Department of Computer Science and Technology, University of Cambridge, Cambridge, UK
LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
Mar 2, 2026Guanzheng Chen, Michael Qizhe Shieh, Lidong BingRL for Language Models Reinforcement Learning with Verifiable Rewards
National University of Singapore · MiroMind AI · absolute AI
GAC: Stabilizing Asynchronous RL Training for LLMs via Gradient Alignment Control
Mar 2, 2026Haofeng Xu, Junwei Su, Yukun Tian +3RL Stability Asynchronous Advantage Actor-Critic
School of Computing and Data Science, The University of Hong Kong · Alibaba Group · Southeast University
MIST-RL: Mutation-based Incremental Suite Testing via Reinforcement Learning
Mar 2, 2026Sicheng Zhu, Jiajun Wang, Jiawei Ai +1Automated Test Generation Mutation Testing
Fudan University, Shanghai, China · University of Science and Technology of China, Hefei, China · Xi’an Jiaotong University, Xi’an, China +1
RLAR: An Agentic Reward System for Multi-task Reinforcement Learning on Large Language Models
Feb 28, 2026Andrew Zhuoer Feng, Cunxiang Wang, Bosi Wen +4RL for Language Models Reward Design for RL
Tsinghua Univeristy · Peking University
Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA
Feb 26, 2026Wenwei Li, Ming Xu, Tianle Xia +7Question Answering Retrieval-Augmented Generation
Tencent
LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models
Feb 25, 2026Wei Zhang, Lintong Du, Yuanhe Zhang +4Instruction Following LLM Fine-Tuning
Beijing University of Posts and Telecommunications, Beijing, China · Nanyang Technological University, Singapore, Singapore
GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning
Feb 25, 2026Ningyuan Yang, Weihua Du, Weiwei Sun +2RL for Language Models Reinforcement Learning
Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University · Language Technologies Institute (LTI), Carnegie Mellon University
Overton Pluralistic Reinforcement Learning for Large Language Models
Feb 24, 2026Yu Fu, Seongho Son, Ilija BogunovicRL for Language Models Multi-Objective Language Model Alignment
University College London · University of Basel / University College London
Not all tokens are needed(NAT): token efficient reinforcement learning
Feb 20, 2026Hejian Sang, Yuanda Xu, Zhengze Zhou +2RL for Language Models Reinforcement Learning
LinkedIn Corporation, CA, USA
Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs
Feb 19, 2026Luke J. Huang, Zhuoyang Zhang, Qinghao Hu +2RL for Language Models Asynchronous RL
MIT · NVIDIA
MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoning
Feb 19, 2026Xiaoliang Fu, Jiaye Lin, Yangyi Fang +7RL for Language Models RL for Language Model Reasoning
Meituan · Fudan University · Tsinghua University +2
Training Large Reasoning Models Efficiently via Progressive Thought Encoding
Feb 18, 2026Zeliang Zhang, Xiaodong Liu, Hao Cheng +3Efficient Language Model Reasoning LLM Reasoning
University of Rochester · Microsoft Research
VAM: Verbalized Action Masking for Controllable Exploration in RL Post-Training -- A Chess Case Study
Feb 18, 2026Zhicheng Zhang, Ziyan Wang, Yali Du +1RL for Language Models RL Exploration
Carnegie Mellon University · King’s College London
Reinforced Fast Weights via Next-Sequence Prediction
Feb 18, 2026Hee Seung Hwang, Xindi Wu, Sanghyuk Chun +2Fast Weights Long-Context Language Modeling
Princeton University
Evolutionary System Prompt Learning for Reinforcement Learning in LLMs
Feb 16, 2026Lunjun Zhang, Ryan Chen, Bradly C. StadieLanguage Model Self-Improvement RL for Language Models
Department of Computer Science, University of Toronto · Department of Statistics and Data Science, Northwestern University · Bridgewater AIA Labs
LACONIC: Length-Aware Constrained Reinforcement Learning for LLM
Feb 16, 2026Chang Liu, Yiran Zhao, Lawrence Liu +3RL for Language Models RL for Language Model Reasoning
University of California, Los Angeles · Salesforce AI Research · National University of Singapore +1
GRRM: Group Relative Reward Modeling for Machine Translation
Feb 15, 2026Sen Yang, Shanbo Cheng, Lu Xu +2Reward Modeling RL for Language Models
National Key Laboratory for Novel Software Technology, Nanjing University · Singapore University of Technology and Design
Learning Agentic Time Series Forecasting via Trajectory Distillation and Memory-Based Reinforcement Learning
Feb 14, 2026Xiaoyu Tao, Mingyue Cheng, Yaguo Liu +4Agentic Time Series Forecasting Time Series Forecasting
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, Hefei, Anhui Province, China

Mar 9, 2026J/K move · Enter open · S save

Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective

Liyuan Mao, Le Yu, Jing Zhou, Chujie Zheng+6

Qwen Team, Alibaba Group · Shanghai Jiao Tong University

Inference-Time Language Model Alignment RL for Language Models RL for Language Model Reasoning

Open paper AskPDF arXiv

In this work, we reveal that Large Language Models (LLMs) possess intrinsic behavioral plasticity-akin to chameleons adapting their coloration to environmental cues-that can be exposed through token-conditional generation and stabilized via reinforcement learning. Specifically, by conditioning generation on carefully selected token prefixes sampled from responses exhibiting desired behaviors, LLMs seamlessly adapt their behavioral modes at inference time (e.g., switching from step-by-step reasoning to direct answering) without retraining. Based on this insight, we propose Token-Conditioned Reinforcement Learning (ToCoRL), a principled framework that leverages RL to internalize this chameleon-like plasticity, transforming transient inference-time adaptations into stable and learnable behavioral patterns. ToCoRL guides exploration with token-conditional generation and keep enhancing exploitation, enabling emergence of appropriate behaviors. Extensive experiments show that ToCoRL enables precise behavioral control without capability degradation. Notably, we show that large reasoning models, while performing strongly on complex mathematics, can be effectively adapted to excel at factual question answering, which was a capability previously hindered by their step-by-step reasoning patterns.