RL for Language Models

RL: Reinforcement Learning

Ask about this topic

Overview

This field studies how feedback can shape a language model’s behavior after pretraining, including how it answers, reasons, uses tools, and handles uncertainty. Common approaches include preference-based reinforcement learning from human or AI feedback, reinforcement learning with verifiable rewards for tasks such as math and coding, and policy methods such as PPO and GRPO; rubrics and step- or turn-level rewards provide finer guidance. Researchers evaluate these methods on reasoning benchmarks as well as dialogue, tool use, multilingual tasks, and domain-specific applications. A central tension is that sparse or imperfect rewards can be costly to obtain and easy to exploit, while reliable credit assignment, exploration, and generalization beyond the training setting remain difficult.

Latest papers

CardsList

441 papers62 papers in the last four weeks, up 15% on the four weeks before. 0.4% of all new papers.

  1. Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training

    Mar 7, 2026Chuxue Cao, Honglin Lin, Zhanping Zhong +5Financial Language Model EvaluationLLM Training

    Shanghai Artificial Intelligence Laboratory, OpenDataLab, OpenDataArena · Hong Kong University of Science and Technology

  2. Post-Training with Policy Gradients: Optimality and the Base Model Barrier

    Mar 7, 2026Alireza Mousavi-Hosseini, Murat A. ErdogduLanguage Model Post-TrainingPolicy Gradient

    University of Toronto, Vector Institute. · University of Toronto, Vector Institute, and Numurho.

  3. MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialogue

    Mar 6, 2026Naifan Zhang, Ruihan Sun, Jinwei Su +4RL for Language ModelsCredit Assignment in RL

    NatureSelect · Tsinghua University · South China Normal University +1

  4. Partial Policy Gradients for RL in LLMs

    Mar 6, 2026Puneet Mathur, Branislav Kveton, Subhojyoti Mukherjee +1RL for Language ModelsPolicy Gradient Methods

    Adobe Research.

  5. Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoning

    Mar 6, 2026Xuan Li, Zhanke Zhou, Zongze Li +4Molecular OptimizationLarge Language Model-Guided Optimization

    Hong Kong Baptist University · CMIC, Shanghai Jiao Tong University · DAMO Academy, Alibaba Group +1

  6. Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation

    Mar 6, 2026Changcheng Li, Jiancan Wu, Hengheng Zhang +5Confidence Estimation in Language ModelsLLM Uncertainty Estimation

    University of Science and Technology of China · Huawei Inc.

  7. BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning

    Mar 5, 2026Yuan Li, Bo Wang, Yufei Gao +4RL for Language ModelsProximal Policy Optimization

    Fudan University · Shanghai Innovation Institute

  8. Bootstrapping Exploration with Group-Level Natural Language Feedback in Reinforcement Learning

    Mar 4, 2026Lei Huang, Xiang Cheng, Chenxiao Zhao +6RL ExplorationRL for Language Models

    Harbin Institute of Technology · Xiaohongshu Inc.

  9. Why Does RLAIF Work At All?

    Mar 3, 2026Robin YoungLLM AlignmentRL for Language Models

    Department of Computer Science and Technology, University of Cambridge, Cambridge, UK

  10. LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards

    Mar 2, 2026Guanzheng Chen, Michael Qizhe Shieh, Lidong BingRL for Language ModelsReinforcement Learning with Verifiable Rewards

    National University of Singapore · MiroMind AI · absolute AI

  11. GAC: Stabilizing Asynchronous RL Training for LLMs via Gradient Alignment Control

    Mar 2, 2026Haofeng Xu, Junwei Su, Yukun Tian +3RL StabilityAsynchronous Advantage Actor-Critic

    School of Computing and Data Science, The University of Hong Kong · Alibaba Group · Southeast University

  12. MIST-RL: Mutation-based Incremental Suite Testing via Reinforcement Learning

    Mar 2, 2026Sicheng Zhu, Jiajun Wang, Jiawei Ai +1Automated Test GenerationMutation Testing

    Fudan University, Shanghai, China · University of Science and Technology of China, Hefei, China · Xi’an Jiaotong University, Xi’an, China +1

  13. RLAR: An Agentic Reward System for Multi-task Reinforcement Learning on Large Language Models

    Feb 28, 2026Andrew Zhuoer Feng, Cunxiang Wang, Bosi Wen +4RL for Language ModelsReward Design for RL

    Tsinghua Univeristy · Peking University

  14. Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA

    Feb 26, 2026Wenwei Li, Ming Xu, Tianle Xia +7Question AnsweringRetrieval-Augmented Generation

    Tencent

  15. LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models

    Feb 25, 2026Wei Zhang, Lintong Du, Yuanhe Zhang +4Instruction FollowingLLM Fine-Tuning

    Beijing University of Posts and Telecommunications, Beijing, China · Nanyang Technological University, Singapore, Singapore

  16. GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning

    Feb 25, 2026Ningyuan Yang, Weihua Du, Weiwei Sun +2RL for Language ModelsReinforcement Learning

    Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University · Language Technologies Institute (LTI), Carnegie Mellon University

  17. Overton Pluralistic Reinforcement Learning for Large Language Models

    Feb 24, 2026Yu Fu, Seongho Son, Ilija BogunovicRL for Language ModelsMulti-Objective Language Model Alignment

    University College London · University of Basel / University College London

  18. Not all tokens are needed(NAT): token efficient reinforcement learning

    Feb 20, 2026Hejian Sang, Yuanda Xu, Zhengze Zhou +2RL for Language ModelsReinforcement Learning

    LinkedIn Corporation, CA, USA

  19. Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs

    Feb 19, 2026Luke J. Huang, Zhuoyang Zhang, Qinghao Hu +2RL for Language ModelsAsynchronous RL

    MIT · NVIDIA

  20. MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoning

    Feb 19, 2026Xiaoliang Fu, Jiaye Lin, Yangyi Fang +7RL for Language ModelsRL for Language Model Reasoning

    Meituan · Fudan University · Tsinghua University +2

  21. Training Large Reasoning Models Efficiently via Progressive Thought Encoding

    Feb 18, 2026Zeliang Zhang, Xiaodong Liu, Hao Cheng +3Efficient Language Model ReasoningLLM Reasoning

    University of Rochester · Microsoft Research

  22. VAM: Verbalized Action Masking for Controllable Exploration in RL Post-Training -- A Chess Case Study

    Feb 18, 2026Zhicheng Zhang, Ziyan Wang, Yali Du +1RL for Language ModelsRL Exploration

    Carnegie Mellon University · King’s College London

  23. Reinforced Fast Weights via Next-Sequence Prediction

    Feb 18, 2026Hee Seung Hwang, Xindi Wu, Sanghyuk Chun +2Fast WeightsLong-Context Language Modeling

    Princeton University

  24. Evolutionary System Prompt Learning for Reinforcement Learning in LLMs

    Feb 16, 2026Lunjun Zhang, Ryan Chen, Bradly C. StadieLanguage Model Self-ImprovementRL for Language Models

    Department of Computer Science, University of Toronto · Department of Statistics and Data Science, Northwestern University · Bridgewater AIA Labs

  25. LACONIC: Length-Aware Constrained Reinforcement Learning for LLM

    Feb 16, 2026Chang Liu, Yiran Zhao, Lawrence Liu +3RL for Language ModelsRL for Language Model Reasoning

    University of California, Los Angeles · Salesforce AI Research · National University of Singapore +1

  26. GRRM: Group Relative Reward Modeling for Machine Translation

    Feb 15, 2026Sen Yang, Shanbo Cheng, Lu Xu +2Reward ModelingRL for Language Models

    National Key Laboratory for Novel Software Technology, Nanjing University · Singapore University of Technology and Design

  27. Learning Agentic Time Series Forecasting via Trajectory Distillation and Memory-Based Reinforcement Learning

    Feb 14, 2026Xiaoyu Tao, Mingyue Cheng, Yaguo Liu +4Agentic Time Series ForecastingTime Series Forecasting

    State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, Hefei, Anhui Province, China

  28. Small Reward Models via Backward Inference

    Feb 14, 2026Yike Wang, Faeze Brahman, Shangbin Feng +3Reward ModelingRL for Language Models

    University of Washington · Allen Institute for Artificial Intelligence

  29. To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models

    Feb 13, 2026Haoqing Wang, Xiang Long, Ziheng Li +3RL for Language ModelsGradient Interference

    Samsung Research, Beijing, China · Peking University

  30. Synthetic Interaction Data for Scalable Personalization in Large Language Models

    Feb 12, 2026Yuchen Ma, Yue Huang, Wenjie Wang +3RL for Language ModelsLLM Personalization

    Munich Center for Machine Learning & LMU Munich · University of Notre Dame · Sichuan University