RL for Language Models

RL: Reinforcement Learning

Ask about this topic

Overview

This field studies how feedback can shape a language model’s behavior after pretraining, including how it answers, reasons, uses tools, and handles uncertainty. Common approaches include preference-based reinforcement learning from human or AI feedback, reinforcement learning with verifiable rewards for tasks such as math and coding, and policy methods such as PPO and GRPO; rubrics and step- or turn-level rewards provide finer guidance. Researchers evaluate these methods on reasoning benchmarks as well as dialogue, tool use, multilingual tasks, and domain-specific applications. A central tension is that sparse or imperfect rewards can be costly to obtain and easy to exploit, while reliable credit assignment, exploration, and generalization beyond the training setting remain difficult.

Latest papers

CardsList

445 papers63 papers in the last four weeks, up 17% on the four weeks before. 0.4% of all new papers.

  1. JoyAI-LLM Flash: Advancing Mid-Scale LLMs with Token Efficiency

    Apr 3, 2026Aichen Cai, Anmeng Zhang, Anyu Li +66Language ModelingMixture-of-Experts Language Models

  2. KARL: Mitigating Hallucinations in LLMs via Knowledge-Boundary-Aware Reinforcement Learning

    Apr 3, 2026Cheng Gao, Cheng Huang, Kangyang Luo +5LLM Hallucination MitigationLLM Abstention

    Tsinghua University

  3. Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks

    Apr 3, 2026Tianze Xu, Yanzhao Zheng, Pengrui Lu +11RL for Language ModelsRubric-Based RL

    Shanghai Jiao Tong University · GAIR · Alibaba Group +2

  4. Generalization Limits of Reinforcement Learning Alignment

    Apr 3, 2026Haruhi Shida, Koo Imai, Keigo KansaLLM AlignmentAdversarial Attacks on LLMs

    Aladdin Security Inc.

  5. Policy Improvement Reinforcement Learning

    Apr 1, 2026Huaiyang Wang, Xiaojie Li, Xiaohan Wang +10RL for Language ModelsRL for Language Model Reasoning

    Beihang University · Meituan · Peking University

  6. REM-CTX: Automated Peer Review via Reinforcement Learning with Auxiliary Context

    Mar 31, 2026Pawin Taechoyotin, Daniel E. AcunaAutomated Peer ReviewRL for Language Models

    Department of Computer Science, University of Colorado Boulder, Boulder, CO 80309

  7. ERPO: Token-Level Entropy-Regulated Policy Optimization for Large Reasoning Models

    Mar 30, 2026Song Yu, Li Li, Wenwen Zhao +1RL for Language Model ReasoningRL for Language Models

    School of Computer and Information Science, Southwest University, Chongqing, 400715, China

  8. TAPO: Translation Augmented Policy Optimization for Multilingual Mathematical Reasoning

    Mar 26, 2026Xu Huang, Zhejian Lai, Zixian Huang +2Cross-Lingual Reasoning in Language ModelsRL for Language Model Reasoning

    National Key Laboratory for Novel Software Technology, Nanjing University · Shanghai Artificial Intelligence Laboratory

  9. Train at Moving Edge: Online-Verified Prompt Selection for Efficient RL Training of Large Reasoning Model

    Mar 26, 2026Jiahao Wu, Ning Lu, Shengcai Liu +6Prompt Selection for RLRL for Language Models

    Southern University of Science and Technology · The Hong Kong Polytechnic University · The Hong Kong University of Science and Technology +3

  10. Reciprocal Co-Training (RCT): Coupling Gradient-Based and Non-Differentiable Models via Reinforcement Learning

    Mar 24, 2026Yunshuo Tian, Akayou Kitessa, Tanuja Chitnis +1RL for Language Models

    Department of Computer and Information Science, Fordham University, New York, NY · Department of Neurology, Mass General Brigham, Boston, MA

  11. SortedRL: Accelerating RL Training for LLMs through Online Length-Aware Scheduling

    Mar 24, 2026Yiqi Zhang, Huiqiang Jiang, Xufang Luo +7RL for Language ModelsRL for Language Model Reasoning

    National University of Singapore · Microsoft Research Asia · The Chinese University of Hong Kong

  12. TAMTRL: Teacher-Aligned Reward Reshaping for Multi-Turn Reinforcement Learning in Long-Context Compression

    Mar 23, 2026Li Wang, Yandong Wang, Xin Yu +3RL for Language ModelsLong-Context Language Modeling

    School of Artificial Intelligence, Beihang University, Beijing, 100191, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, China · Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University, Beijing, China

  13. Implicit Turn-Wise Policy Optimization for Proactive User-LLM Interaction

    Mar 21, 2026Haoyu Wang, Yuxin Chen, Liang Luo +3RL for Language ModelsProcess Reward Models

    Georgia Institute of Technology · Meta AI

  14. FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization

    Mar 20, 2026Chiyu Ma, Shuo Yang, Kexin Huang +7RL for Language Model ReasoningRL for Language Models

    Qwen Pilot Team, Alibaba Group

  15. LoopRPT: Reinforcement Pre-Training for Looped Language Models

    Mar 20, 2026Guo Tang, Shixin Jiang, Heng Chang +7Efficient Language Model ReasoningImplicit Reasoning in Language Models

    Harbin Institute of Technology, Harbin, China · Tsinghua University, Beijing, China · The Hong Kong University of Science and Technology, Guangzhou, China

  16. Adaptive Layerwise Perturbation: Unifying Off-Policy Corrections for LLM RL

    Mar 19, 2026Chenlu Ye, Xuanchang Zhang, Yifan Hao +6Importance Sampling for RLRL for Language Models

    University of Illinois Urbana-Champaign · Amazon

  17. VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models

    Mar 19, 2026Chonghan Liu, Yimin Du, Qi An +8RL for Language ModelsLow-Resource MT

    1Qiyuan Tech.

  18. Learning to Self-Evolve

    Mar 19, 2026Xiaoyin Chen, Canwen Xu, Yite Wang +3Language Model Self-ImprovementRL for Language Models

    Mila – Quebec AI Institute · University of Montreal · Snowflake

  19. Adaptive Decoding via Test-Time Policy Learning for Self-Improving Generation

    Mar 19, 2026Asmita Bhardwaj, Yuya Jeremy Ong, Eelaaf Zahid +1Language Model DecodingTest-Time Adaptation

    UC San Diego San Diego, CA · Plastic Labs San Jose, CA · IBM Research Cambridge, MA +1

  20. PowerFlow: Unlocking the Dual Nature of LLMs via Principled Distribution Matching

    Mar 19, 2026Ruishuo Chen, Yu Chen, Zhuoran Li +1RL for Language ModelsReinforcement Learning

    Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China

  21. Enhancing Reinforcement Learning Fine-Tuning with an Online Refiner

    Mar 18, 2026Hao Ma, Zhiqiang Pu, Yang Liu +1RL Fine-TuningRL for Language Models

    School of Artificial Intelligence, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences

  22. REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge

    Mar 17, 2026Yasi Zhang, Tianyu Chen, Mingyuan Zhou +3RL for Language ModelsReinforcement Learning

    University of California, Los Angeles · The University of Texas at Austin · Google Research

  23. HIPO: Instruction Hierarchy via Constrained Reinforcement Learning

    Mar 17, 2026Keru Chen, Jun Luo, Sen Lin +4LLM AlignmentRL for Language Models

    School of ECEE Arizona State University · Department of ECE The Ohio State University · Computer Science Department University of Houston +2

  24. Aligning Paralinguistic Understanding and Generation in Speech LLMs via Multi-Task Reinforcement Learning

    Mar 16, 2026Jingxiang Chen, Minseok Kim, Seong-Gyun Leem +13Speech Language ModelsEmotion Representation in Language Models

    Meta Reality Labs

  25. Writer-R1: Dynamic Rubric Mining for Reinforcement Learning of Open-Ended Writing

    Mar 16, 2026Jihao Zhao, Shuaishuai Zu, Zhiyuan Ji +2Memory-Augmented RLRL for Language Models

    School of Information, Renmin University of China, Beijing, China · School of Information, BRAIN, Renmin University of China, Beijing, China

  26. Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation

    Mar 13, 2026Yifeng Liu, Siqi Ouyang, Yatish Hosmane Revanasiddappa +1Low-Resource MTMultilingual Language Models

    Carnegie Mellon University

  27. Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning

    Mar 11, 2026Zichao Li, Jie Lou, Fangchen Dong +8RL for Language ModelsRL Post-Training

    Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Xiaohongshu Inc.

  28. Learning Adaptive LLM Decoding

    Mar 10, 2026Chloe H. Su, Zhe Ye, Samuel Tenka +3Language Model DecodingInference-Time Adaptation of Language Models

    Department of Computer Science, Harvard University, Boston, USA · Kempner Institute, Harvard University, Cambridge, MA, USA · UC Berkeley, CA, USA +1

  29. MAPLE: Elevating Medical Reasoning from Statistical Consensus to Process-Led Alignment

    Mar 9, 2026Kailong Fan, Anqi Pu, Yichen Wu +7Medical LLMsRL for Language Models

    Zhejiang University · Harvard Medical School · Harvard University

  30. How Far Can Unsupervised RLVR Scale LLM Training?

    Mar 9, 2026Bingxiang He, Yuxin Zuo, Zeyuan Liu +18RL for Language ModelsReinforcement Learning with Verifiable Rewards

    Tsinghua University · Shanghai AI Lab · Xi’an Jiaotong University +3