RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts
Aug 5, 2026Nhat Minh Pham, Duy Tung Doan, Thi Duyen Ngo +2RL for Language Model Reasoning Speculative Decoding
VNU University of Engineering and Technology, Vietnam National University, Hanoi, Vietnam · Viettel AI, Viettel Group, Hanoi, Vietnam
TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning
Aug 4, 2026Changle Qu, Sunhao Dai, Hengyi Cai +4On-Policy Self-Distillation RL for Language Model Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China · Baidu Inc.
ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning
Aug 4, 2026Jinhe Bi, Chennan Zhou, Zengjie Jin +10RL for Language Model Reasoning RL Post-Training
National University of Singapore · Ludwig Maximilian University of Munich · Huawei Heisenberg Research Center +6
DiagLoop: A Counterfactual Data Flywheel with Stage-Localized Reinforcement for Diagnostic LLMs
Aug 4, 2026Jian Zhang, Bingyi Wang, Yizhi LiuRL for Language Model Reasoning Causal Reasoning in Language Models
Zhejiang University
Hi-TTRL: Regulating Consensus with Hints for Test-Time Reinforcement Learning
Aug 4, 2026Kunbin Xu, Xingzuo Li, Xuefeng Bai +1RL for Language Model Reasoning Test-Time RL
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China
When Correct Solutions Repeat: Rarity-Aware Credit Redistribution for GRPO
Aug 4, 2026Zhe Cao, Miaowen Wen, Fangjiong ChenRL for Language Model Reasoning Group Relative Policy Optimization
South China University of Technology
Don't Peek at the Answer: Outcome-Masked Group Relative Policy Optimization for Label-Free RLVR
Aug 4, 2026Yongshi Ye, Liang Zhang, Yidong Chen +2RL for Language Model Reasoning Group Relative Policy Optimization
Xiamen University · Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism
PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translation
Aug 4, 2026Yongshi Ye, Biao Fu, Chongxuan Huang +2RL for Language Model Reasoning Process Supervision
Institute of Artificial Intelligence, Xiamen University · Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism · School of Informatics, Xiamen University
CVPO: Enhancing LLM Reinforcement Learning Reasoning via Value-Variance Adaptation and Dynamic Curriculum Learning
Aug 4, 2026Ziqi Jia, Yalu Ouyang, Bo Pang +5Numerical Reasoning in Language Models RL for Language Model Reasoning
AI Cloud Group, Baidu · Tsinghua University · University of California, San Diego
BODHI: Do LLMs Branch Out and Discover Heterogeneous Inferences?
Aug 3, 2026Soumadeep Saha, Krish Sharma, Akshay Chaturvedi +1Reasoning Diversity RL for Language Model Reasoning
ANITI, Université de Toulouse · LINAGORA Labs
Start Classifying: Categorical Critics for LLM Reinforcement Learning
Aug 3, 2026Zhijian Zhou, Long Li, Xuan Zhang +7RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Fudan University · Shanghai Innovation Institute · Tencent YoutuLab +2
Beyond the Mean: Multi-Moment Policy Optimization for LLM Reasoning
Aug 3, 2026Yijun Zhang, Yule Xie, Jiaxin Ding +4RL for Language Model Reasoning Policy Optimization
Shanghai Jiao Tong University
Self-Improving Large Language Models via Progressive Experience Evolution
Aug 3, 2026Shijie Ren, Xiting Wang, Meng Li +8RL for Language Model Reasoning Language Model Self-Improvement
Gaoling School of Artificial Intelligence, Renmin University of China · Weixin AI, Tencent Inc, China
Instruction-Conditioned Exploration for Reinforcement Learning with Self-Distillation to an Unconditioned Policy
Aug 3, 2026Jim Dilkes, Vahid Yazdanpanah, Sebastian SteinRL for Language Model Reasoning RL Post-Training
University of Southampton Southampton United Kingdom
AdaThinkV: Adaptive Thinking for Token-Efficient Video Reasoning
Aug 3, 2026Jingqi Tian, Haoji Zhang, Lin Chen +7Efficient VLM Inference RL for Language Model Reasoning
Tsinghua Shenzhen International Graduate School, Tsinghua University · Alibaba Group · Peking University
Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models
Aug 3, 2026Wonseok Lee, Jimyeong Kim, Jungmin Ko +1RL for Language Model Reasoning RL for Diffusion Models
Interdisciplinary Program in Artificial Intelligence, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea · Artificial Intelligence Institute, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea · Department of Intelligence and Information, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea
TCPO: Turn-Level Credit Policy Optimization
Aug 3, 2026Sicong Liao, Zhi Chen, Yaohua TangRL for Language Model Reasoning Credit Assignment in RL
Moore Threads AI
Latent Thought Credit: Multi-Answer Credit Assignment for Latent Reasoning
Aug 3, 2026Xuyang Zhao, Liting Zhang, Zichen Xu +4Numerical Reasoning in Language Models RL for Language Model Reasoning
TMCC, College of Computer Science, Nankai University, Tianjin, China · Lingxi (Beijing) Technology Co., Ltd.
Reusing Rollouts under Policy Lag: Prefix-Normalized Policy Optimization for LLM Reinforcement Learning
Aug 2, 2026Wenhao Zhang, Yibo Xie, Rui Wang +9Reinforcement Learning RL for Language Model Reasoning
1Tencent · 2Harbin Institute of Technology · 3Jinan University +1
Gaokerena: A Small Persian Medical Language Model Family
Aug 2, 2026Mehrdad Ghassabi, Hamidreza Baradaran Kashani, Pedram Rostami +5RL for Language Model Reasoning Small Language Models
University of Isfahan, Isfahan, Iran · University of Tehran, Tehran, Iran · University of Windsor, Windsor, ON, Canada +3
Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance
Aug 1, 2026Zhuowen Han, Jinwei Xiao, Zhengxi Lu +9Self-Training for Language Models RL for Language Model Reasoning
TJUNLP Lab, School of Computer Science and Technology, Tianjin University · Meituan Longcat Team
Verifier-Induced Support Reshaping in On-Policy Optimization
Jul 31, 2026Shaohang Wei, Zikun Su, Feifan Song +4RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Peking University · BUPT
Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translation
Jul 31, 2026Yongshi Ye, Biao Fu, Chongxuan Huang +2RL for Language Model Reasoning Efficient Language Model Inference
Institute of Artificial Intelligence, Xiamen University · School of Informatics, Xiamen University · Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism
Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning
Jul 31, 2026Xinyan Guan, Jiali Zeng, Chunlei Xin +5Overthinking in Language Models RL for Language Model Reasoning
1Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · 3Weixin AI, Tencent Inc, China
SAF-OPD: Stable Advantage Fusion for On-Policy Distillation
Jul 31, 2026Yifan Ding, Xincheng Wei, Yoshua Y. Li +7RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Shanghai University of Finance and Economics · Meituan, LongCat Team · The Chinese University of Hong Kong, Shenzhen +1
$β$ -OPSD: Deriving with Policy Optimization, Training with Self-Distillation
Jul 30, 2026Jiawei Xu, Minghui Liu, Juzheng Zhang +2KL-Regularized RL On-Policy Self-Distillation
University of Maryland, College Park
SVR: Self-Verifying Refinement via Joint Verdict-Confidence Reinforcement Learning for Adaptive Test-Time Compute
Jul 30, 2026Hongyu Chen, Liang Lin, Guangrun WangRL for Language Model Reasoning Language Model Self-Assessment
1Sun Yat-sen University · 2Guangdong Key Laboratory of Big Data Analysis and Processing · 3X-Era AI Lab
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Jul 30, 2026Xingjian Wu, Junlin Liu, Xingchen Liu +6Agentic RL RL for Language Model Reasoning
Meituan
Not All Tokens Deserve Equal Credit: Counterfactual Sensitivity Credit Reallocation for Long-CoT Reasoning
Jul 30, 2026Qiangqiang He, Zhongheng Wu, ZiJian WangToken-Level Credit Assignment RL for Language Model Reasoning
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Institute of Wireless Communications Technology, Shanghai Jiao Tong University, Shanghai, China
LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts
Jul 30, 2026Ken DingReinforcement Learning RL for Language Model Reasoning
NVIDIA

Aug 5, 2026, cs.LGJ/K move · Enter open · S save

SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts

Nhat Minh Pham, Duy Tung Doan, Thi Duyen Ngo, Vinh Van Nguyen+1

VNU University of Engineering and Technology, Vietnam National University, Hanoi, Vietnam · Viettel AI, Viettel Group, Hanoi, Vietnam

RL for Language Model Reasoning Speculative Decoding LLM Inference Acceleration

Open paper AskPDF arXiv

Reinforcement learning (RL) post-training improves the reasoning capabilities of large language models, but autoregressive rollout generation remains a major efficiency bottleneck. Speculative decoding can accelerate generation, yet applying it during RL is difficult because the target policy continually evolves: static proposers become stale, while frequent drafter updates add substantial overhead. We introduce SpecRoll, a speculative rollout engine that preserves the target model's sampling distribution while adapting at two timescales. Lightweight future-token heads generate parallel proposals, while our proposed Reflex module uses delayed verifier feedback to perform bounded, trajectory-local hidden-state corrections without backpropagation. A complementary slow path updates the head parameters only when sustained degradation is detected. SpecRoll combines these mechanisms with concurrency-aware sparse-tree verification and exact target verification, leaving the target rollout distribution and GRPO objective unchanged. Across five models ranging from 1.5B to 14B and three mathematical reasoning datasets, SpecRoll achieves 1.26-2.15x generation speedup and 1.21-2.04x end-to-end speedup over vanilla GRPO. It also outperforms FastGRPO in both generation and end-to-end time across all 15 matched settings, with an average pairwise end-to-end gain of 1.18x. Controlled ablations show that the fast and slow adaptation paths provide complementary benefits. Our source code is available at https://anonymous.4open.science/r/SpecRoll-26062006.

Momentum

Related topics

Latest papers 707