RL for Language Models

RL: Reinforcement Learning

Ask about this topic

Overview

This field studies how feedback can shape a language model’s behavior after pretraining, including how it answers, reasons, uses tools, and handles uncertainty. Common approaches include preference-based reinforcement learning from human or AI feedback, reinforcement learning with verifiable rewards for tasks such as math and coding, and policy methods such as PPO and GRPO; rubrics and step- or turn-level rewards provide finer guidance. Researchers evaluate these methods on reasoning benchmarks as well as dialogue, tool use, multilingual tasks, and domain-specific applications. A central tension is that sparse or imperfect rewards can be costly to obtain and easy to exploit, while reliable credit assignment, exploration, and generalization beyond the training setting remain difficult.

Latest papers

CardsList

441 papers79 papers in the last four weeks, up 126% on the four weeks before. 0.5% of all new papers.

  1. GR2: Generative Reasoning Re-ranker

    Feb 8, 2026Mingfu Liang, Yufei Li, Jay Xu +20RL for Language ModelsRL for Recommendation

    Meta AI

  2. Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis

    Jan 31, 2026Zicheng Kong, Dehua Ma, Zhenbo Xu +9Reward ModelingRL for Language Models

    Beijing University of Posts and Telecommunications, Beijing, P.R. China · ShiFang Technology Inc., Hangzhou, P.R. China · Tsinghua University, Beijing, P.R. China +1

  3. HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning

    Jan 30, 2026Weiqi Wang, Xin Liu, Binxuan Huang +13RL for Language ModelsRL for Language Model Reasoning

    Stores Foundational AI, Amazon Inc., Palo Alto, CA, USA · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China

  4. Retell, Reward, Repeat: Reinforcement Learning for Narrative Theory-Informed Story Retelling

    Jan 23, 2026David Y. Liu, Xanthe Muston, Dipankar Srirag +2Narrative GenerationText Generation

    University of New South Wales, Sydney, Australia

  5. HOMURA: Taming the Sand-Glass for Time-Constrained LLM Translation via Reinforcement Learning

    Jan 15, 2026Ziang Cui, Mengran Yu, Chenyu Shi +2Multilingual Language ModelsRL for Language Models

    Bilibili Inc., Shanghai, China

  6. Trust Region Masking for Long-Horizon LLM Reinforcement Learning

    Dec 28, 2025Yingru Li, Jiacai Liu, Jiawei Xu +4RL for Language ModelsReinforcement Learning

    Fudan University · The Chinese University of Hong Kong, Shenzhen

  7. Stepwise Think-Critique: Interleaved Reasoning and Self-Critique in a Single LLM

    Dec 17, 2025Jiaqi Xu, Cuiling Lan, Xuejin Chen +1LLM Self-CorrectionCoT Reasoning

    University of Science and Technology of China · ⋆This work was done when Jiaqi was an intern at MSRA. · Microsoft Research Asia

  8. Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitation

    Nov 14, 2025Mohamad Amin Mohamadi, Tianhao Wang, Zhiyuan LiLLM AbstentionLLM Hallucination Mitigation

    Toyota Technological Institute at Chicago · University of California, San Diego

  9. OpenReward: Learning to Reward Long-form Agentic Tasks via Reinforcement Learning

    Oct 28, 2025Ziyou Hu, Zhengliang Shi, Minghang Zhu +5Reward ModelingRL for Language Models

    Leiden University, Leiden, Netherland · Shandong University, Qingdao, China · Tsinghua University, Beijing, China

  10. Strengthening LLMs for Tabular Prediction with Structural Priors

    Oct 20, 2025Pengxiang Cai, Zihao Gao, Wanchen Lian +2Tabular Foundation ModelsRL for Language Models

    The Hong Kong University of Science and Technology (Guangzhou) · Zhejiang University

  11. Rethinking On-policy Optimization for Query Augmentation

    Oct 20, 2025Zhichao Xu, Shengyao Zhuang, Xueguang Ma +6RL for Language ModelsLLM Prompting

    University of Utah · 2The University of Queensland · University of Waterloo +5

  12. CLARity: Reasoning Consistency Alone Can Teach Reinforced Experts

    Oct 10, 2025Jiuheng Lin, Cong Jiang, Zirui Wu +2RL for Language ModelsRL for Language Model Reasoning

    Wangxuan Institute of Computer Technology, Peking University

  13. TaoSR-AGRL: Adaptive Guided Reinforcement Learning Framework for E-commerce Search Relevance

    Oct 9, 2025Jianhui Yang, Yiming Jin, Pengkun Jiao +6RL for Language ModelsReward Shaping

    Tsinghua University Beijing, China · Taobao & Tmall Group of Alibaba Beijing, China · Taobao & Tmall Group of Alibaba Hangzhou, China +1

  14. Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Models

    Sep 30, 2025Jaesung R. Park, Junsu Kim, Gyeongman Kim +4Entropy Collapse in RLRL for Language Models

    Department of Mathematics, UCLA · Department of Mathematical Sciences, Seoul National University · KRAFTON +2

  15. Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation

    Sep 18, 2025Yujun Zhou, Zhenwen Liang, Haolin Liu +7Language Model Self-ImprovementRL for Language Models

    Tencent AI Lab · University of Notre Dame · University of Virginia

  16. ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs

    Aug 19, 2025Hongxin Ding, Baixiang Huang, Yue Fang +8RL for Language ModelsHealthcare

    National Engineering Research Center of Software Engineering, Peking University, China · School of Computing and Data Science, The University of Hong Kong · School of Computer Science, Peking University, Beijing, China +2

  17. TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback

    Aug 4, 2025Lei Pang, Jun Luo, Ruinan JinImportance Sampling for RLRL for Language Models

    Peking University Beijing, China · The Ohio State University Columbus, USA · Shugu Yuntai Technology Co., Ltd. +1

  18. Post-Training Large Language Models via Reinforcement Learning from Self-Feedback

    Jul 29, 2025Carel van Niekerk, Renato Vukovic, Benjamin Ruppik +3RL for Language ModelsRL for Language Model Reasoning

    Heinrich Heine University Düsseldorf, Germany

  19. Re:Form -- Reducing Human Annotations in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny

    Jul 22, 2025Chuanhao Yan, Fengdi Che, Xuhan Huang +12RL for Language ModelsRL for Language Model Reasoning

    Shanghai AI Lab · Tsinghua University · University of Alberta +5

  20. A Technical Survey of Reinforcement Learning Techniques for Large Language Models

    Jul 5, 2025Saksham Sahai Srivastava, Vaneet AggarwalRL for Language ModelsLLM Alignment

    University of Georgia, Athens, GA, USA · Purdue University, West Lafayette, IN, USA

  21. AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training

    Jul 2, 2025Zhenyu Han, Ansheng You, Haibo Wang +16RL for Language ModelsAsynchronous RL

    Huawei · Individual Researcher