RL for Language Model Reasoning

RL: Reinforcement Learning

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

1,105 papers170 papers in the last four weeks, up 124% on the four weeks before. 1.0% of all new papers.

Aryabhata 2: Scaling Reinforcement Learning for Advanced STEM Reasoning
Apr 10, 2026Ritvik Rastogi, Vishal Singh, Tejas Chaudhari +1Scientific Reasoning in Language Models Efficient Language Model Reasoning
PhysicsWallah
Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning
Apr 10, 2026Taojie Zhu, Dongyang Xu, Ding Zou +4LLM Fine-Tuning RL for Language Model Reasoning
Shenzhen International Graduate School, Tsinghua University · Intelligent System Department, Zhongxing Telecom Equipment (ZTE) · Institute of Advanced Interdisciplinary Studies, Chongqing University of Posts and Telecommunications
StaRPO: Stability-Augmented Reinforcement Policy Optimization
Apr 10, 2026Jinghan Zhang, Fengran Mo, Tharindu Cyril Weerasooriya +5RL for Language Model Reasoning Reasoning Consistency in Language Models
Clemson University · Université de Montréal · Accenture (United States) +2
SPPO: Sequence-Level PPO for Long-Horizon Reasoning Tasks
Apr 10, 2026Tianyi Wang, Yixia Li, Long Li +6RL for Language Model Reasoning Proximal Policy Optimization
Southern University of Science and Technology · Beijing University of Posts and Telecommunications · INFLY TECH +3
ExecTune: Effective Steering of Black-Box LLMs with Guide Models
Apr 9, 2026Vijay Lingam, Aditya Golatkar, Anwesan Pal +6Language Model Steering LLM Inference Efficiency
AWS AI
Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?
Apr 9, 2026Chia-Hsuan Lee, Mingyang Zhou, Renkun Ni +6Preference Optimization Pairwise Preference Learning
Capital One
Skip-Connected Policy Optimization for Implicit Advantage
Apr 9, 2026Fengwei Teng, Jinyi Bai, Xinhao Yao +3RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Mohamed bin Zayed University of Artificial Intelligence · Renmin University of China · Carnegie Mellon University +1
SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions
Apr 9, 2026Ashima Suvarna, Kendrick Phan, Mehrab Beikzadeh +2RL for Language Model Reasoning LLM Reasoning
University of California, Los Angeles
TTVS: Boosting Self-Exploring Reinforcement Learning via Test-time Variational Synthesis
Apr 9, 2026Sikai Bai, Haoxi Li, Jie Zhang +2Test-Time Adaptation Test-Time RL
The Hong Kong University of Science and Technology
Beyond Stochastic Exploration: What Makes Training Data Valuable for Agentic Search
Apr 9, 2026Chuzhan Hao, Wenfeng Feng, Guochao Jiang +3Agentic Search RL for Language Model Reasoning
Alibaba Cloud Computing
QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training--Inference Mismatch
Apr 9, 2026Hao Gu, Hao Wang, Jiacheng Liu +9Reinforcement Learning RL Fine-Tuning
The Hong Kong University of Science and Technology · City University of Hong Kong · Zhejiang University
ReRec: Reasoning-Augmented LLM-based Recommendation Assistant via Reinforcement Fine-tuning
Apr 9, 2026Jiani Huang, Shijie Wang, Liangbo Ning +2Large Language Model-Based Recommendation RL Fine-Tuning
The Hong Kong Polytechnic University
Reason in Chains, Learn in Trees: Self-Rectification and Grafting for Multi-turn Agent Policy Optimization
Apr 8, 2026Yu Li, Sizhe Tang, Tian LanRL for Language Model Reasoning RL for LLM Agents
Department of Electrical and Computer Engineering, George Washington University
SubSearch: Intermediate Rewards for Unsupervised Guided Reasoning in Complex Retrieval
Apr 8, 2026Roxana Petcu, Evangelos Kanoulas, Maarten de RijkeRL for Language Model Reasoning Process Reward Models
IRLab, University of Amsterdam
SHAPE: Stage-aware Hierarchical Advantage via Potential Estimation for LLM Reasoning
Apr 8, 2026Zhengyang Ai, Zikang Shan, Xiaodong Ai +3LLM Reasoning Efficient Language Model Reasoning
Huawei Taylor Lab · Center for Data Science, Peking University · Shanghai University of Finance and Economics
Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning
Apr 8, 2026Rohan Surana, Gagan Mundada, Xunyi Jiang +19RL for Language Model Reasoning RL for Language Models
University of California, San Diego · University of Toronto · UIUC +3
Target Policy Optimization
Apr 7, 2026Jean KaddourPolicy Optimization RL Post-Training
Limits of Difficulty Scaling: Hard Samples Yield Diminishing Returns in GRPO-Tuned SLMs
Apr 7, 2026Suraj Yadav, Siddharth Yadav, Parth GoyalRL for Language Model Reasoning Small Language Models
IIIT Delhi
Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs
Apr 7, 2026Hongyuan Yuan, Xinran He, Run Shao +6Efficient Language Model Reasoning Overthinking in Language Models
School of Geosciences and Info-Physics, Central South University, Changsha, China · Baidu Inc., Beijing, China
ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning
Apr 7, 2026Xuan Xiong, Huan Liu, Li Gu +4Efficient Language Model Reasoning CoT Reasoning
University of Toronto · McMaster University · Concordia University +1
Not All Turns Are Equally Hard: Adaptive Thinking Budgets For Efficient Multi-Turn Reasoning in Agents
Apr 6, 2026Neharika Jali, Anupam Nayak, Gauri JoshiMulti-Turn Language Model Reasoning Efficient Language Model Reasoning
Carnegie Mellon University
How Reasoning Evolves from Post-Training Data: An Empirical Study Using Chess
Apr 6, 2026Lucas Dionisopoulos, Nicklas Majamaki, Prithviraj AmmanabroluLanguage Model Post-Training RL for Language Model Reasoning
Department of Computer Science and Engineering, University of California San Diego, San Diego, United States.
QED-Nano: Teaching a Tiny Model to Prove Hard Theorems
Apr 6, 2026LM-Provers, Yuxiao Qu, Amrith Setlur +6Mathematical Reasoning LLM Mathematical Reasoning
CMU · Hugging Face · ETH Zurich +1
Asymmetric Advantage Modulation Calibrates Entropy Dynamics in RLVR
Apr 6, 2026Hengrui Gu, Xiaotian Han, Yujing Bian +2RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
North Carolina State University · Case Western Reserve University · Oak Ridge National Laboratory
Cog-DRIFT: Exploration on Adaptively Reformulated Instances Enables Learning from Hard Reasoning Problems
Apr 6, 2026Justin Chih-Yao Chen, Archiki Prasad, Zaid Khan +4RL for Language Model Reasoning LLM Reasoning
UNC Chapel Hill
Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVR
Apr 6, 2026Kazuki Egashira, Mark Vero, Jasper Dekoninck +3Reinforcement Learning with Verifiable Rewards RL for Language Model Reasoning
ETH Zurich · Max Planck Institute for Intelligent Systems, Tübingen
When Rewards Disagree: Reward Weighting for Token-Efficient Video Reasoning
Apr 6, 2026Zinuo Li, Yongxin Guo, Jun Liu +6Video Reasoning VLM Reasoning
University of Western Australia · Tencent Youtu Lab · CUHKSZ +2
Design Conditions for Intra-Group Learning of Sequence-Level Rewards: Token Gradient Cancellation
Apr 4, 2026Fei Ding, Yongkang Zhang, youwei wang +1RL for Language Model Reasoning Step-Level Credit Assignment in RL
Alibaba Group · Tsinghua University
BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data
Apr 3, 2026Brian Hsu, Ozan Gökdemir, Carlo Siebenschuh +7Scientific Reasoning in Language Models Scientific QA
Department of Computer Science, University of Chicago, Chicago, IL 60637 · Data Science and Learning Division, Argonne National Laboratory, Lemont, IL 60439
Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolution
Apr 3, 2026Jacob Dineen, Aswin RRV, Zhikun Xu +1Curriculum RL RL for Language Model Reasoning
Arizona State University

Apr 10, 2026J/K move · Enter open · S save

Aryabhata 2: Scaling Reinforcement Learning for Advanced STEM Reasoning

Ritvik Rastogi, Vishal Singh, Tejas Chaudhari, Sandeep Varma

PhysicsWallah

Scientific Reasoning in Language Models Efficient Language Model Reasoning RL for Language Model Reasoning

Open paper AskPDF arXiv

Competitive STEM examinations such as JEE and NEET require multi-step symbolic reasoning, precise numerical computation, and deep conceptual understanding across physics, chemistry, and mathematics. Recent large language models perform strongly on common reasoning benchmarks, yet they remain difficult to deploy at scale, where millions of student doubts demand domain-specific, consistently structured problem solving. We introduce Aryabhata 2, a reasoning-focused language model for competitive STEM examinations, trained via reinforcement-learning post-training. Using PhysicsWallah's internal question banks, we construct a high-quality training curriculum and post-train GPT-OSS-20B through reinforcement learning with verifiable rewards. Training combines prolonged reinforcement learning with broadened exploration via progressively larger rollout group sizes. We evaluate Aryabhata 2 on competitive examination benchmarks, including JEE Main, JEE Advanced, and NEET, as well as out-of-distribution reasoning datasets such as AIME, HMMT, MMLU-Pro, MMLU-Redux 2.0, and GPQA. Results show that Aryabhata 2 outperforms its base model GPT-OSS-20B on competitive STEM reasoning while requiring substantially fewer output tokens (up to 64% fewer).