RL for Language Model Reasoning

RL: Reinforcement Learning

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

1,103 papers170 papers in the last four weeks, up 124% on the four weeks before. 1.0% of all new papers.

Reward Under Attack: Analyzing the Robustness and Hackability of Process Reward Models
Feb 20, 2026Rishabh Tiwari, Aditya Tomar, Udbhav Bamba +5Process Reward Models Reward Hacking
UC Berkeley · Transmute AI · ICSI +1
Adaptive Time Series Reasoning via Segment Selection
Feb 20, 2026Shvat Messica, Jiawen Zhang, Kevin Li +2Time Series Reasoning Time Series QA
Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA · The Hong Kong University of Science and Technology (Guangzhou) · Massachusetts Institute of Technology +1
Not all tokens are needed(NAT): token efficient reinforcement learning
Feb 20, 2026Hejian Sang, Yuanda Xu, Zhengze Zhou +2RL for Language Models Reinforcement Learning
LinkedIn Corporation, CA, USA
MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoning
Feb 19, 2026Xiaoliang Fu, Jiaye Lin, Yangyi Fang +7RL for Language Models RL for Language Model Reasoning
Meituan · Fudan University · Tsinghua University +2
WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning
Feb 19, 2026Gagan Mundada, Zihan Huang, Rohan Surana +8Efficient Language Model Reasoning RL for Language Model Reasoning
University of California, San Diego · University of Toronto · 3Adobe Research +2
Balancing Faithfulness and Performance in Reasoning via Multi-Listener Soft Execution
Feb 18, 2026Nithin Sivakumaran, Shoubin Yu, Hyunji Lee +4CoT Faithfulness Faithfulness of Language Model Explanations
UNC Chapel Hill · Cisco · University of Texas at Austin
STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokens
Feb 17, 2026Shiqi Liu, Zeyu He, Guojian Zhan +10Reinforcement Learning RL for Language Model Reasoning
School of Vehicle and Mobility & College of AI, Tsinghua University · Didi Voyager Labs, DiDi Autonomous Driving
BFS-PO: Best-First Search for Large Reasoning Models
Feb 16, 2026Fiorenzo Parascandolo, Wenhui Tan, Enver Sangineto +2Efficient Language Model Reasoning RL for Language Model Reasoning
Department of Engineering, University of Modena and Reggio Emilia, Italy. · Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China.
On the Emergence of Implicit Curriculum in RLVR Learning Dynamics
Feb 16, 2026Yu Huang, Zixin Wen, Yuejie Chi +4RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Department of Statistics and Data Science, Wharton School, University of Pennsylvania · Machine Learning Department, Carnegie Mellon University · Department of Statistics and Data Science, Yale University +1
Goldilocks RL: Tuning Task Difficulty to Escape Sparse Rewards for Reasoning
Feb 16, 2026Ilia Mahrooghi, Aryo Lotfi, Emmanuel AbbeRL for Language Model Reasoning Curriculum RL
EPFL · Apple
Evolutionary System Prompt Learning for Reinforcement Learning in LLMs
Feb 16, 2026Lunjun Zhang, Ryan Chen, Bradly C. StadieLanguage Model Self-Improvement RL for Language Models
Department of Computer Science, University of Toronto · Department of Statistics and Data Science, Northwestern University · Bridgewater AIA Labs
LACONIC: Length-Aware Constrained Reinforcement Learning for LLM
Feb 16, 2026Chang Liu, Yiran Zhao, Lawrence Liu +3RL for Language Models RL for Language Model Reasoning
University of California, Los Angeles · Salesforce AI Research · National University of Singapore +1
Beyond Token-Level Policy Gradients for Complex Reasoning with Large Language Models
Feb 16, 2026Mufan Xu, Kehai Chen, Xuefeng Bai +4RL for Language Model Reasoning Policy Optimization
School of Computer Science and Technology, Harbin Institute of Technology, China · Baidu Inc., Beijing, China
Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learning
Feb 14, 2026Yanbo Wang, Minzheng Wang, Jian Liang +3LLM Alignment RL for Language Model Reasoning
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China. · NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing, China. · Ritzz-AI.
TGPO: Trace-Guided Policy Optimization for Robot Task Planning via Verifiable Subgoal Generation
Feb 12, 2026Zhihong Liu, Yang Li, Renming Huang +3Robot Task Planning Large Language Model-Based Robot Planning
Shanghai Innovation Institute · Xi’an Jiao Tong University · Nankai University +1
Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models
Feb 12, 2026Xin Xu, Clive Bai, Kai Yang +8RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
HY, Tencent · The Hong Kong University of Science and Technology · The University of Hong Kong
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
Feb 10, 2026Pei-Chi Pan, Yingbin Liang, Sen LinReward Modeling Reward Hacking
Department of Computer Science University of Houston · The Ohio State University · University of Houston
rePIRL: Learn PRM with Inverse RL for LLM Reasoning
Feb 8, 2026Xian Wu, Kaijie Zhu, Ying Zhang +2Reinforcement Learning Process Reward Models
Meta AI · Department of Computer Science, University of California, Santa Barbara · Independent Researcher +1
F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare
Feb 6, 2026Daniil Plyusov, Alexey Gorbatovski, Boris Shaposhnikov +4Reinforcement Learning RL for Language Model Reasoning
T-Tech · Saint Petersburg Electrotechnical University “LETI”
An evidence-guided reinforcement learning method to improve psychiatric reasoning in small language models
Feb 6, 2026Xinxin Lin, Guangxin Dai, Yi Zhong +25Healthcare Mental Health
The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR, China · Shandong University, Jinan, China · Peking University Sixth Hospital, Beijing, China +6
Difficulty-Estimated Policy Optimization
Feb 6, 2026Yu Zhao, Fan Jiang, Tianle Liu +5RL for Language Model Reasoning Policy Optimization
Alibaba International Digital Commerce · School of Software Technology, Dalian University of Technology Dalian, China
The Optimal Token Baseline: Variance Reduction for Long-Horizon LLM-RL
Feb 6, 2026Yingru Li, Jiawei Xu, Ziniu Li +10RL for Language Model Reasoning Policy Gradient Methods
The Chinese University of Hong Kong, Shenzhen · Fudan University · Hong Kong University of Science and Technology +4
LongR: Unleashing Long-Context Reasoning via Reinforcement Learning with Dense Utility Rewards
Feb 5, 2026Qize Yu, Zijun Chen, Chen Wu +6Long-Context Reasoning in Language Models RL for Language Model Reasoning
Shanghai Jiao Tong University · Tsinghua University · Beijing University of Posts and Telecommunications, Beijing, China +1
Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation
Feb 5, 2026Zhiqi Yu, Zhangquan Chen, Mengting Liu +2RL for Language Model Reasoning Group Relative Policy Optimization
University of Hong Kong · Tsinghua University · Sun Yat-sen University
QUATRO: Query-Adaptive Trust Region Policy Optimization for LLM Fine-tuning
Feb 4, 2026Doyeon Lee, Eunyi Lyou, Hyunsoo Cho +3Trust-Region Optimization Policy Learning
Seoul National University · Department of Artificial Intelligence / Institute for Multiscale Matter and Systems, Ewha Woman University · Geogia Institute of Technology
Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning
Feb 1, 2026Yu Li, Mingyang Yi, Xiuyu Li +6RL for Language Model Reasoning Gradient Interference
School of Information, Renmin University of China, Beijing, China · Bytedance Inc., Beijing, China · Bytedance Inc., San Jose, USA
Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients
Jan 30, 2026Cheng Ge, Caitlyn Heqi Yin, Hao Liang +1RL for Language Model Reasoning Group Relative Policy Optimization
Department of Aeronautics and Astronautics, MIT · Department of Statistics, University of Wisconsin–Madison · Department of Informatics, King’s College London +1
Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning
Jan 30, 2026Arvind Mahankali, Kaiyue Wen, Tengyu MaMulti-Agent LLM Systems CoT Reasoning
Stanford University
HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
Jan 30, 2026Weiqi Wang, Xin Liu, Binxuan Huang +13RL for Language Models RL for Language Model Reasoning
Stores Foundational AI, Amazon Inc., Palo Alto, CA, USA · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
Can David Beat Goliath? On Multi-Hop Reasoning with Resource-Constrained Agents
Jan 29, 2026Hojae Han, Heeyun Jung, Jongyoon Kim +1RL for Language Model Reasoning Multi-Hop QA
ETRI · The University of Hong Kong · Seoul National University

Feb 20, 2026J/K move · Enter open · S save

Reward Under Attack: Analyzing the Robustness and Hackability of Process Reward Models

Rishabh Tiwari, Aditya Tomar, Udbhav Bamba, Monishwaran Maheswaran+4

UC Berkeley · Transmute AI · ICSI+1

Process Reward Models Reward Hacking Adversarial Attacks Reasoning Verification RL for Language Model Reasoning

Open paper AskPDF arXiv

Process Reward Models (PRMs) are rapidly becoming the backbone of LLM reasoning pipelines, yet we demonstrate that state-of-the-art PRMs are systematically exploitable under adversarial optimization pressure. To address this, we introduce a three-tiered diagnostic framework that applies increasing adversarial pressure to quantify these vulnerabilities. Static perturbation analysis uncovers a fluency-logic dissociation: high invariance to surface-level style changes reward changes

<

0.1, yet inconsistent detection of logically-corrupted reasoning, with different models failing on different attack types. Adversarial optimization demonstrates that gradient-based attacks inflate rewards on invalid trajectories, with reward landscapes exhibiting wide, exploitable peaks. RL-induced reward hacking exposes the critical failure mode: policies trained on AIME problems achieve near-perfect PRM rewards (

>

0.9), while ground-truth accuracy remains low (below 4%), with 43% of reward gains attributable to stylistic shortcuts. These findings reveal that current PRMs function as fluency detectors rather than reasoning verifiers, creating systematic blind spots that undermine their use as training signals. We release PRM-BiasBench and a diagnostic toolkit to enable robustness evaluation before deployment. The code and dataset are available at https://github.com/SqueezeAILab/reward-under-attack.