RL for Language Model Reasoning

RL: Reinforcement Learning

Overview

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

CardsList

1,171 papers80 papers in the last four weeks, down 55% on the four weeks before. 0.6% of all new papers.

  1. Budget Internalization: Fixed-Budget RLVR Trades Test-Time Scaling for In-Budget Accuracy

    Nov 19, 2026Ilia Zavidnyi, Alexander Kovrigin, Arina Puchkova +2Test-Time ScalingLLM Reasoning

  2. RL Can Only Reinforce What It Sees Succeed

    Nov 19, 2026Ruben Sanchez-Corcuera, Unai Zulaika, Aritz Bilbao-Jayo +1Reinforcement LearningRL for Language Models

  3. PAIR-MED: Policy-reward Alternation for Iterative Refinement in Multilingual Medical Reasoning

    Nov 19, 2026Fan Gao, Xiaohang Xu, Lang Huang +2RL for Language Model ReasoningProcess Reward Models

  4. Cross-Question Reliable Reinforcement Learning

    Nov 18, 2026Hector G. Rodriguez, Marcus RohrbachLLM AbstentionConfidence Estimation in Language Models

  5. Get Hint-Augmented GRPO Done Right Through Activation Steering

    Nov 17, 2026Khiem Le, Ting Hua, Nitesh V. ChawlaReinforcement LearningRL for Language Model Reasoning

  6. MCTS Is Counting the Wrong Thing: Learning Exploration Counts for LLM Reasoning

    Nov 17, 2026Xuan Zhang, Zhijian Zhou, Zicheng Hu +4Monte Carlo Tree SearchRL for Language Model Reasoning

  7. Procedure without Judgment: What Small Language Models Learn under an Exact Verifier

    Nov 16, 2026Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R. +2LLM Agent Policy ComplianceConcept-Based Explanations

  8. Learning to Reason with Predictive Gains for Generative Recommendation

    Nov 16, 2026Zhuoyang Li, Teng Shi, Jiakai Tang +10Generative RecommendationLarge Language Model-Based Recommendation

  9. GPRM: Global Perspective Process Reward Model

    Nov 16, 2026Zifan Zhang, Zhengbo Jiao, Shaobo Wang +10Process Reward ModelsLanguage Model Reasoning Evaluation

  10. SpeCal: Spectral Calibration for Unsupervised RLVR

    Nov 16, 2026Kai Tang, Xing Zheng, Haobo Wang +7Reinforcement LearningRL for Language Model Reasoning

  11. Interactive Structured Environment Scaling for Large Language Model Agentic Reasoning

    Nov 15, 2026Peng Yu, Luoyi Fu, Xinbing Wang +1RL for Language Model ReasoningRL for LLM Agents

  12. Policy Optimization With Multiple Verifiers via Pareto Rankings

    Nov 14, 2026Kai Yamashita, Shohei Taniguchi, Hiroki Furuta +2RL for Language Model ReasoningReinforcement Learning with Verifiable Rewards

  13. ARCHER: From Rationale Imitation to Executable Reasoning in Small Language Models

    Nov 13, 2026Dhruv Dixit, Khushi Patel, Congcong Xu +1LLM ReasoningStructured Reasoning in Language Models

  14. Negative-Only Policy Optimization for One-Sided Verifiable Rewards

    Nov 12, 2026Jiacheng Xu, Shuo He, Fuxiang Zhang +2Reinforcement LearningReinforcement Learning with Verifiable Rewards

  15. The Value of Thinking: Learning When to Reason in Hybrid Decision Models

    Nov 12, 2026Zhengxu Yu, Ke Xu, Meng Fang +3Adaptive Reasoning Resource Allocation for Language ModelsAdaptive Reasoning in Language Models

  16. Turning Multi-Agent Communication into Token-Level Credit with On Policy Self Distillation

    Nov 12, 2026Zixu Wang, Yu Zhao, Xing Zheng +7RL for Language Model ReasoningMulti-Agent LLMs

  17. The Illusion of Diversity: Aligning LLM Exploration via Effective Entropy

    Nov 11, 2026Xiaoliang Fu, Jiaye Lin, Yangyi Fang +4RL for Language Model ReasoningRL Exploration

  18. Towards Better Legal Reasoning LLMs: Signal Balancing and Reward Scheduling

    Nov 11, 2026Yinghao Hu, Qian Xiao, Yuting Huang +7Legal ReasoningStructured Reasoning in Language Models

  19. HyperSkill: Training-Free Omnimodal GRPO via Hypergraph-Indexed Skill-Library Evolution

    Nov 11, 2026Haoran Luo, Shangkai Lin, Jinyang Wu +2RL for Language Model ReasoningMLLM Reasoning

  20. Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data

    Nov 11, 2026Yunlong Hou, Fengzhuo Zhang, Yuan Cheng +3Language Model GeneralizationSupervised Fine-Tuning

  21. AutoScaffold: AI-Guided Scaffolding for Reinforcement Learning of Language Models

    Nov 10, 2026Zijian Zhang, Rizhen Hu, Jiangweizhi Peng +4RL for Language ModelsLLM-Guided RL

  22. SLIC: Reinforcement Fine-Tuning Small LMs for Multi-Turn Analog Circuit Optimization

    Nov 10, 2026Yan Xu, Zhengqi Gao, Ching-Yun Ko +6Electronic Design AutomationAnalog Circuit Sizing