RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
May 13, 2026Feng Zhang, Xinhong Ma, Ziqiang Dong +5RL for Language Model Reasoning Group Relative Policy Optimization
Beijing Institute of Technology · Qwen Business Unit of Alibaba · The Chinese University of Hong Kong, Shenzhen +1
AIS: Adaptive Importance Sampling for Quantized RL
May 13, 2026Jiajun Zhou, Wei Shao, Lingchao Zheng +2RL for Language Models Reinforcement Learning
Huawei · The University of Hong Kong
Multi-Rollout On-Policy Distillation via Peer Successes and Failures
May 12, 2026Weichen Yu, Xiaomin Li, Yizhou Zhao +8RL for Language Model Reasoning Language Model Distillation
1Carnegie Mellon University · 2Microsoft · 3Purdue University
Learning, Fast and Slow: Towards LLMs That Adapt Continually
May 12, 2026Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal +6Continual Learning for LLMs RL for Language Model Reasoning
1UC Berkeley · 4Eragon · 2Mila +3
Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
May 12, 2026Yuanda Xu, Hejian Sang, Zhengze Zhou +3Reinforcement Learning RL for Language Model Reasoning
Holder Policy Optimisation
May 12, 2026Yuxiang Chen, Dingli Liang, Yihang Chen +8RL for Language Model Reasoning Group Relative Policy Optimization
University College London, London, United Kingdom. · 3The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · 2Shanghai Jiao Tong University, Shanghai, China.
StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning
May 12, 2026Hao Wang, Rui Li, Lei Sha +1RL for Language Model Reasoning Execution-Guided Code Generation
Beihang University, Beijing, China · Peking University, Beijing, China · Zhongguancun Laboratory, Beijing, China +1
GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation
May 12, 2026Sijia Li, Yuchen Huang, Zifan Liu +7RL for Language Model Reasoning Credit Assignment in RL
Hong Kong University of Science and Technology · Microsoft Research Asia
Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control
May 12, 2026Jiazheng Zhang, Ziche Fu, Junrui Shen +17RL for Language Model Reasoning Policy Optimization
Fudan NLP Group, Honor Device Co., Ltd · University of Hong Kong, Shanghai Jiao Tong University and Tencent Hunyuan
Seirênes: Adversarial Self-Play with Evolving Distractions for LLM Reasoning
May 12, 2026Chi Zhang, Haibo Qiu, Qiming Zhang +3RL for Language Model Reasoning Adversarial Attacks on LLMs
School of Computer Science, Wuhan University · Meituan Inc · Independent Researchers +1
Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoning
May 12, 2026Zhaomeng Zhou, Lan Zhang, Junyang Wang +2RL for Language Model Reasoning Efficient Language Model Reasoning
University of Science and Technology of China · The Chinese University of Hong Kong
Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information
May 12, 2026Guobin Shen, Xiang Cheng, Chenxiao Zhao +4On-Policy Self-Distillation RL for Language Model Reasoning
1Xiaohongshu Inc. · Institute of Automation, Chinese Academy of Sciences
Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting
May 12, 2026Cheng Wang, Qin Liu, Wenxuan Zhou +1Reinforcement Learning RL for Language Model Reasoning
National University of Singapore · University of California, Davis · University of Southern California
Selective Off-Policy Reference Tuning with Plan Guidance
May 12, 2026Duc Anh Le, Tien-Phat Nguyen, Thien Huu Nguyen +2Reinforcement Learning RL for Language Model Reasoning
Independent Author · Hanoi University of Science and Technology, Hanoi, Vietnam · University of Oregon, Eugene, OR, USA +1
Understanding and Preventing Entropy Collapse in RLVR with On-Policy Entropy Flow Optimization
May 12, 2026Huimin Xu, Shuai Zhao, Xiaobao Wu +1Reinforcement Learning RL for Language Model Reasoning
Nanyang Technological University, Singapore · Shanghai Jiao Tong University, China · 3VinUniversity, Vietnam
Drop the Act: Probe-Filtered RL for Faithful Chain-of-Thought Reasoning
May 12, 2026Swapnil Parekh, Naman GoyalCoT Faithfulness Faithfulness of Language Model Explanations
Intuit
Breaking $\textit{Winner-Takes-All}$ : Cooperative Policy Optimization Improves Diverse LLM Reasoning
May 12, 2026Haoxuan Chen, Tianming Liang, Wei-Shi Zheng +1Reinforcement Learning RL for Language Model Reasoning
1ISEE Lab, Sun Yat-sen University
Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning
May 12, 2026Zihao Han, Tiangang Zhang, Huaibin Wang +1On-Policy Self-Distillation RL for Language Model Reasoning
ByteDance Douyin
fg-expo: Frontier-guided exploration-prioritized policy optimization via adaptive kl and gaussian curriculum
May 12, 2026Mingxiong Lin, Zhangquan Gong, Maowen Tang +6Reinforcement Learning RL for Language Model Reasoning
OPPO AI Center
Internalizing Curriculum Judgment for LLM Reinforcement Fine-Tuning
May 11, 2026Han Zheng, Yining Ma, Karthick Gunasekaran +4Reinforcement Learning Meta-Learning
MIT · Amazon AGI
Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVR
May 11, 2026Jeonghye Kim, Jiwon Jeon, Dongsheng Li +1RL for Language Model Reasoning Language Model Distillation
Microsoft Research · KAIST
Equilibrium Residuals Expose Three Regimes of Matrix-Game Strategic Reasoning in Language Models
May 11, 2026Wenhua Nie, Binhan Luo, Zijie Meng +2Nash Equilibrium LLM Evaluation
National Taiwan University
TMAS: Scaling Test-Time Compute via Multi-Agent Synergy
May 11, 2026George Wu, Nan Jing, Qing Yi +7Multi-Agent LLM Systems RL for Language Model Reasoning
1IQuest Research · 2Beihang University
ReCrit: Transition-Aware Reinforcement Learning for Scientific Critic Reasoning
May 11, 2026Wanghan Xu, Yuhao Zhou, Hengyuan Zhao +8LLM Sycophancy Scientific Reasoning
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · This work was done during his internship at Shanghai Artificial Intelligence Laboratory. +4
Relative Score Policy Optimization for Diffusion Language Models
May 11, 2026Zichao Yu, Shengze Xu, Bingqing Jiang +2RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
University of Science and Technology of China · The Chinese University of Hong Kong · The University of Hong Kong
STEPS: Selective On-Policy Self-Distillation for Reasoning
May 11, 2026Jiaxuan Wang, Xuan Ouyang, Zhiyu Chen +2On-Policy Self-Distillation RL for Language Model Reasoning
State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · AMAP, Alibaba Group +2
expo: Exploration-prioritized policy optimization via adaptive kl regulation and gaussian curriculum sampling
May 11, 2026Mingxiong Lin, Zhangquan Gong, Maowen Tang +6Reinforcement Learning KL-Regularized RL
OPPO AI Center
Verifier-Free RL for LLMs via Intrinsic Gradient-Norm Reward
May 11, 2026Xuexiang Wen, Hang Yu, Linchao Zhu +1Intrinsic Reward Methods Reinforcement Learning
1Zhejiang University · 2Ant Group
Exploration-Driven Optimization for Test-Time Large Language Model Reasoning
May 11, 2026Changhao Li, Yuchen Zhuang, Chenxiao Gao +4RL for Language Models Inference-Time Optimization
Georgia Institute of Technology
LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models
May 10, 2026Songtao Wei, Yi Li, Zhikai Li +7RL for Language Model Reasoning Efficient Language Model Reasoning
University of Texas at Dallas · Emory University · University of Texas at Arlington +2

May 13, 2026, cs.LGJ/K move · Enter open · S save

Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective

Feng Zhang, Xinhong Ma, Ziqiang Dong, Xi Leng+4

Beijing Institute of Technology · Qwen Business Unit of Alibaba · The Chinese University of Hong Kong, Shenzhen+1

RL for Language Model Reasoning Group Relative Policy Optimization Policy Optimization Reinforcement Learning with Verifiable Rewards RL Post-Training Contrastive RL

Open paper AskPDF arXiv

Group Relative Policy Optimization (GRPO) is one of the most widely adopted RLVR algorithms for post-training large language models on reasoning tasks. We first show that GRPO admits an equivalent discriminative reformulation, in which policy optimization maximizes the expected score gap between verified positive and negative rollouts. This reformulation reveals two objective-level limitations: likelihood-misaligned surrogate scores, in which clipped ratio-based scores are optimized rather than the sequence likelihoods that govern generation, and score-insensitive credit assignment, in which rollout-level credit does not reflect the current score gaps between positive and negative rollouts. To address these limitations, we propose ConSPO, a Contrastive Sequence-level Policy Optimization method that uses length-normalized sequence log-probabilities as rollout scores and contrasts verified positive rollouts against negative distractors within the same group. ConSPO optimizes a group-wise InfoNCE-style objective to adaptively strengthen updates for poorly separated positives and high-scoring negatives, together with a curriculum-scheduled margin that preserves separation pressure as training progresses. Experiments across diverse settings show that ConSPO outperforms strong baselines on challenging reasoning benchmarks. Code will be released upon paper acceptance.

Momentum

Related topics

Latest papers 707