RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

124 papers in the last four weeks, up 75% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 780

All topics

PA3: Policy-Aware Agent Alignment through Chain-of-Thought
Mar 15, 2026Shubhashis Roy Dipta, Daniel Bis, Kun Zhou +4LLM Alignment RL for Language Model Reasoning
University of Maryland, Baltimore County · Amazon Alexa AI
Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating
Mar 11, 2026Yuning Wu, Ke Wang, Haoran Liu +3Reinforcement Learning RL for Language Model Reasoning
Amazon
MIND: Unified Inquiry and Diagnosis RL with Criteria Grounded Clinical Supports for Psychiatric Consultation
Mar 4, 2026Guoyi Li, Shihao Xu, Jiatong Ma +4Healthcare Evidence-Grounded Reasoning
EverMind AI Inc., California, USA · Venus Team, Ant Group · Tianqiao and Chrissy Chen Institute, Shanghai, China +3
Truncated Step-Level Sampling with Process Rewards for Retrieval-Augmented Reasoning
Feb 26, 2026Chris Samarinas, Haw-Shiuan Chang, Hamed ZamaniReinforcement Learning Process Reward Models
Center for Intelligent Information Retrieval University of Massachusetts Amherst Amherst, MA, United States
Adaptive Time Series Reasoning via Segment Selection
Feb 20, 2026Shvat Messica, Jiawen Zhang, Kevin Li +2Time Series Reasoning Time Series QA
Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA · The Hong Kong University of Science and Technology (Guangzhou) · Massachusetts Institute of Technology +1
On the Emergence of Implicit Curriculum in RLVR Learning Dynamics
Feb 16, 2026Yu Huang, Zixin Wen, Yuejie Chi +4RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Department of Statistics and Data Science, Wharton School, University of Pennsylvania · Machine Learning Department, Carnegie Mellon University · Department of Statistics and Data Science, Yale University +1
Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learning
Feb 14, 2026Yanbo Wang, Minzheng Wang, Jian Liang +3LLM Alignment RL for Language Model Reasoning
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China. · NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing, China. · Ritzz-AI.
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
Feb 10, 2026Pei-Chi Pan, Yingbin Liang, Sen LinReward Modeling Reward Hacking
Department of Computer Science University of Houston · The Ohio State University · University of Houston
rePIRL: Learn PRM with Inverse RL for LLM Reasoning
Feb 8, 2026Xian Wu, Kaijie Zhu, Ying Zhang +2Reinforcement Learning Process Reward Models
Meta AI · Department of Computer Science, University of California, Santa Barbara · Independent Researcher +1
F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare
Feb 6, 2026Daniil Plyusov, Alexey Gorbatovski, Boris Shaposhnikov +4Reinforcement Learning RL for Language Model Reasoning
T-Tech · Saint Petersburg Electrotechnical University “LETI”
An evidence-guided reinforcement learning method to improve psychiatric reasoning in small language models
Feb 6, 2026Xinxin Lin, Guangxin Dai, Yi Zhong +25Healthcare Mental Health
The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR, China · Shandong University, Jinan, China · Peking University Sixth Hospital, Beijing, China +6
The Optimal Token Baseline: Variance Reduction for Long-Horizon LLM-RL
Feb 6, 2026Yingru Li, Jiawei Xu, Ziniu Li +10RL for Language Model Reasoning Policy Gradient Methods
The Chinese University of Hong Kong, Shenzhen · Fudan University · Hong Kong University of Science and Technology +4
Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation
Feb 5, 2026Zhiqi Yu, Zhangquan Chen, Mengting Liu +2RL for Language Model Reasoning Group Relative Policy Optimization
University of Hong Kong · Tsinghua University · Sun Yat-sen University
QUATRO: Query-Adaptive Trust Region Policy Optimization for LLM Fine-tuning
Feb 4, 2026Doyeon Lee, Eunyi Lyou, Hyunsoo Cho +3RL for Language Model Reasoning Policy Optimization
Seoul National University · Department of Artificial Intelligence / Institute for Multiscale Matter and Systems, Ewha Woman University · Geogia Institute of Technology
Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning
Feb 1, 2026Yu Li, Mingyang Yi, Xiuyu Li +6RL for Language Model Reasoning Gradient Interference
School of Information, Renmin University of China, Beijing, China · Bytedance Inc., Beijing, China · Bytedance Inc., San Jose, USA
Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients
Jan 30, 2026Cheng Ge, Caitlyn Heqi Yin, Hao Liang +1RL for Language Model Reasoning Group Relative Policy Optimization
Department of Aeronautics and Astronautics, MIT · Department of Statistics, University of Wisconsin–Madison · Department of Informatics, King’s College London +1
Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning
Jan 30, 2026Arvind Mahankali, Kaiyue Wen, Tengyu MaMulti-Agent LLM Systems CoT Reasoning
Stanford University
HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
Jan 30, 2026Weiqi Wang, Xin Liu, Binxuan Huang +13RL for Language Models RL for Language Model Reasoning
Stores Foundational AI, Amazon Inc., Palo Alto, CA, USA · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
Thinking Seeds: Leveraging Historical Diversity for Position-Aware RL in LLMs
Jan 29, 2026Lei Yang, Wei Bi, Chenxi Sun +2RL for Language Model Reasoning RL Exploration
TJUNLP Lab, College of Intelligence and Computing, Tianjin University, Tianjin, China · Kuaishou Technology
Policy of Thoughts: Scaling Test-Time Training for LLM Reasoning via Online Policy Evolution
Jan 28, 2026Zhengbo Jiao, Hongyu Xian, Qinglong Wang +5RL for Language Model Reasoning Test-Time Training
Zhejiang University · South China Normal University · Shanghai University of Finance and Economics +2
Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
Jan 26, 2026Shobhita Sundaram, John Quan, Ariel Kwiatkowski +3Self-Play RL RL for Language Model Reasoning
1MIT · 2Meta FAIR · 3New York University
Inference-Time Diversity in RL-Trained Lean Theorem Provers: A Diagnostic Study
Jan 22, 2026Zachary BurtonInference-Time Search RL for Language Model Reasoning
Massachusetts Institute of Technology, Cambridge, MA, USA.
CLEANER: Self-Purified Trajectories Boost Agentic Reinforcement Learning
Jan 21, 2026Tianshi Xu, Yuteng Chen, Meng LiAgentic RL Reinforcement Learning
Peking University · NTU, Singapore
R $^2$ PO: Decoupling Rollout and Inference Policies for LLM Reasoning
Jan 17, 2026Jingchu Wang, Bingbing Xu, Yige Yuan +4RL for Language Model Reasoning Policy Optimization
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · National University of Singapore
Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs
Jan 16, 2026Lecheng Yan, Ruizhe Li, Guanhua Chen +5Reward Hacking Memorization in Language Models
Southern University of Science and Technology · University of Science and Technology of China · University of Aberdeen +5
SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models
Jan 7, 2026Yuxuan Jiang, Francis FerraroProcess Reward Models RL for Language Model Reasoning
University of Maryland, Baltimore County
KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering
Dec 10, 2025Xin Sun, Zhongqi Chen, Xing Zheng +6LLM Grounding RL for Language Model Reasoning
University of Science and Technology of China, Hefei, China · Independent · Institute of Automation, Chinese Academy of Sciences, Beijing, China.
Training Language Models to Use Prolog as a Tool
Dec 8, 2025Niklas Mellgren, Peter Schneider-Kamp, Lukas Galke PoechReward Hacking Logical Reasoning
University of Southern Denmark
Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
Nov 25, 2025Chenliang Li, Adel Elmahdy, Alex Boyd +7RL for Language Model Reasoning Importance Sampling
Texas A&M University · GE HealthCare · Independent Researcher +1
ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
Nov 24, 2025Long Lian, Sida Wang, Felix Juefei-Xu +7RL for Language Model Reasoning Inference-Time Scaling
Meta Superintelligence Labs (MSL), Menlo Park, CA, United States · UC Berkeley, Berkeley, CA, United States · UCSF, San Francisco, CA, United States

Mar 15, 2026, cs.CLJ/K move · Enter open · S save

PA3: Policy-Aware Agent Alignment through Chain-of-Thought

Shubhashis Roy Dipta, Daniel Bis, Kun Zhou, Lichao Wang+3

University of Maryland, Baltimore County · Amazon Alexa AI

LLM Alignment RL for Language Model Reasoning

Open paper AskPDF arXiv

Conversational assistants powered by large language models (LLMs) excel at tool-use tasks but struggle with adhering to complex, business-specific rules. While models can reason over business rules provided in context, including all policies for every query introduces high latency and wastes compute. Furthermore, these lengthy prompts lead to long contexts, harming overall performance due to the 'needle-in-a-haystack' problem. To address these challenges, we propose a multi-stage alignment method that teaches models to recall and apply relevant business policies during chain-of-thought reasoning at inference time, without including the full business policy in-context. Furthermore, we introduce a novel Policy Recall reward based on the Jaccard score and a Hallucination Penalty for GRPO training. Altogether, our best model outperforms the baseline by 16 points and surpasses comparable in-context baselines of similar model size by 3 points, while using 40% fewer words.

Momentum

Related topics

Latest papers 780