RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

Learning What to Learn: Stage-Specific Data Sets for SFT-then-RL in Small Language Model Reasoning
Jun 3, 2026Chongyang He, Rui Zhang, Zixuan Wang +1Supervised Fine-Tuning RL for Language Model Reasoning
1Tsinghua University · 2National University of Singapore · 3DiDi +1
Read the Trace, Steer the Path: Trajectory-Aware Reinforcement Learning for Diffusion Language Models
Jun 3, 2026Anant Khandelwal, Manish GuptaRL for Language Model Reasoning Credit Assignment in RL
Microsoft AI, India
QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards
Jun 2, 2026Rongzhi Zhang, Rui Feng, Zhihan Zhang +8Reinforcement Learning RL for Language Model Reasoning
Amazon · Georgia Institute of Technology
PyraMathBench: Evaluating and Improving Mathematical Capability in Large Language Models
Jun 2, 2026Zetian Ouyang, Linlin Wang, Gerard de Melo +1Numerical Reasoning in Language Models Mathematical Reasoning Benchmarks
East China Normal University · Hasso Plattner Institute, University of Potsdam
Tool-Aware Optimization with Entropy Guidance for Efficient Agentic Reinforcement Learning
Jun 2, 2026Hongye Cao, Nuo Yan, Haoyuan Deng +5Agentic RL RL for Language Model Reasoning
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China · Nanyang Technological University, Singapore · China Mobile NineVerse Artificial Intelligence Technology (Beijing) Co., Ltd., and Institute of Artificial Intelligence, NineVerse, Beijing 100032, China
Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verification
Jun 2, 2026Jiahui Li, Jianfeng Shan, Wenpei Chen +5Reinforcement Learning LLM Answer Verification
1Sun Yat-Sen University · University of Science and Technology of China · 3National University of Singapore
CAPER: Clause-Aligned Process Supervision for Text-to-SQL
Jun 2, 2026Lujie Ban, Jiasheng Shi, Jinyang Li +3Process Reward Models RL for Language Model Reasoning
The Chinese University of Hong Kong, Shenzhen · The University of Hong Kong · Northwestern Polytechnical University +1
Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning
Jun 2, 2026Ziyue Wang, Aomufei Yuan, Yongfu Zhu +10LLM Alignment RL for Language Model Reasoning
1Peking University · 2JINGDONG · 3Shanghai Innovation Institute +2
FGRPO: Federated GRPO with Adaptive Aggregation on Non-IID Data
Jun 2, 2026Pengyu Chen, Shaowei Li, Kai Wang +4Federated Learning Aggregation Federated RL
School of Computer Science and Technology, Shandong University, Qingdao China · School of Mathematical Science, Peking University, China · School of Computer Science and Artificial Intelligence, Shanghai University of Finance and Economics, Shanghai, China +1
ASymPO: Asymmetric-Scale Policy Optimization for Asynchronous LLM Post-Training Without Behavior Information
Jun 2, 2026Zehua Liu, Yuxuan Yao, Xiaojin Fu +2Asynchronous RL Reinforcement Learning
Huawei Technologies
Hint-Guided Diversified Policy Optimization for LLM Reasoning
Jun 2, 2026Zhiyu Cao, Kaixin Wu, Mingjie Zhong +4Reasoning Diversity RL for Language Model Reasoning
School of Computer Science and Technology, Soochow University, Suzhou, China · Ant Group, Hangzhou, China
Learning When to Translate for Multilingual Reasoning
Jun 1, 2026Deokhyung Kang, Hyounghun Kim, Gary Geunbae LeeMultilingual Language Models Cross-Lingual Reasoning in Language Models
1Graduate School of Artificial Intelligence · POSTECH, South Korea · Department of Computer Science & Engineering
Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learning
Jun 1, 2026Liuji Chen, Dianxing Tang, Xing Shi +4Agentic RL RL for Language Model Reasoning
1NLPR, Institute of Automation, Chinese Academy of Sciences · 3Zhejiang University · 2ByteDance
HMPO: Hybrid Median-length Policy Optimization for Chain-of-Thought Compression
Jun 1, 2026Minghui Zheng, Hongxu Chen, Huimin Ren +8LLM Compression RL for Language Model Reasoning
Li Auto Inc.
CAPF: Guiding Search-Agent Rollouts with Credit-Attenuated Privileged Feedback
Jun 1, 2026Bin Chen, Xinye Liao, Yiming Liu +2RL for Language Model Reasoning Credit Assignment in RL
OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification
May 31, 2026Yuhang Zhou, Lizhu Zhang, Yifan Wu +5RL for Language Model Reasoning Language Model Distillation
Meta AI
Cross-lingual Self-Consistency for Multilingual Reasoning with Language Models
May 31, 2026Ahmed Elhady, Eneko Agirre, Mikel ArtetxeMultilingual Language Models Cross-Lingual Reasoning in Language Models
HiTZ Center, University of the Basque Country (UPV/EHU) · Reka AI
RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning
May 31, 2026Yixiu Mao, Yun Qu, Qi Wang +2Reinforcement Learning RL for Language Model Reasoning
Department of Automation, Tsinghua University
Deep Research as Rubric for Reinforcement Learning
May 31, 2026Wangyi Mei, Zhouhong Gu, Zhenhan Bai +9Open-Ended Generation Reinforcement Learning
Fudan University · Xiaohongshu Inc. · Beijing University of Posts and Telecommunications
Enhancing LLM Metacognition via Cognitive Pairwise Training
May 30, 2026Weitao Li, Hao Zhou, Xuanyu Lei +11Pairwise Preference Learning RL for Language Model Reasoning
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China · Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China · WeChat AI, Tencent Inc., China
Internalize the Temperature: On-Policy Self-Distillation as Policy Reheater for Reinforcement Learning
May 30, 2026Xuewei Yang, Jiachen Yu, Jie Wu +3On-Policy Self-Distillation RL for Language Model Reasoning
Tsinghua University
The Paradox of Outcome Optimization: A Causal Information-Theoretic Bound on Reasoning Shortcuts in LLMs
May 30, 2026Zihan Chen, Yiming Zhang, Wenxiang Geng +2Process Reward Models RL for Language Model Reasoning
1HFIPS, Chinese Academy of Sciences · University of Science and Technology of China
PR2: Predictive Routing Replay for MoE-Based LLM Reinforcement Learning
May 29, 2026Daize Dong, Junlin Chen, Haolong Jia +9RL for Language Model Reasoning Expert Routing
♣Rutgers University · ♦AMD · ⋆MBZUAI
ARCA: Adapter-Residual Credit Assignment When Token Signals Degenerate
May 29, 2026Rodney Lafuente-MercadoToken-Level Credit Assignment RL for Language Model Reasoning
Scale AI
LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards
May 29, 2026Nianyi Lin, Jiajie Zhang, Lei Hou +1LLM Grounding RL for Language Model Reasoning
Tsinghua University
Are Full Rollouts Necessary for On-Policy Distillation?
May 29, 2026Yaocheng Zhang, Jiajun Chai, Yuqian Fu +7RL for Language Model Reasoning Language Model Distillation
Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Meituan +1
CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPO
May 29, 2026Yang Li, Gongle Xue, Yijia Guo +3RL for Language Model Reasoning Group Relative Policy Optimization
School of Software and Microelectronics, Peking University, Beijing · School of Artificial Intelligence, Peking University, Beijing · School of Computer Science, Peking University, Beijing +1
EchoRL: Reinforcement Learning via Rollout Echoing
May 29, 2026Jinhe Bi, Aniri, Minglai Yang +9Reinforcement Learning RL for Language Model Reasoning
Huawei Heisenberg Research Center · LMU Munich · Munich Center for Machine Learning +5
AdaptR1: Reinforcement Learning Based Adaptive Interleaved Thinking in Multi-hop Question Answering
May 29, 2026Yuxin Wang, Jiahao Lu, Qifeng Wu +5Multi-Hop QA RL for Language Model Reasoning
1Computer Science, Fudan University · Institute of Modern Languages and Linguistics, Fudan University · 3Shanghai Innovation Institute +1
Sample-Efficient Post-Training for LEGO Spatial-Physics Reasoning
May 29, 2026Yuhuan Yuan, Zhouliang Yu, Minghao Liu +2Physical Reasoning Visual Spatial Reasoning
HKUST(GZ) · CUHK · ZODA

Jun 3, 2026, cs.CLJ/K move · Enter open · S save

Learning What to Learn: Stage-Specific Data Sets for SFT-then-RL in Small Language Model Reasoning

Chongyang He, Rui Zhang, Zixuan Wang, Xin Li

1Tsinghua University · 2National University of Singapore · 3DiDi+1

Supervised Fine-Tuning RL for Language Model Reasoning Small Language Models Curriculum Learning Language Model Post-Training

Open paper AskPDF arXiv

Post-training Small Language Models (SLMs) for reasoning typically follows an SFT-then-RL pipeline, yet existing work rarely considers what data should be learned at each stage. We argue that data strategy should be aligned with the distinct roles of SFT and RL: SFT is better suited for acquiring not-yet-mastered reasoning skills, while RL is better suited for consolidating skills that the model can already partially access. Based on this principle, we propose a difficulty-aware SFT-then-RL framework that organizes training data into stage-specific sets. For hard samples in the SFT stage, we introduce a Bridge mechanism that transforms raw teacher-generated reasoning traces into more learnable supervision for SLMs. For hard samples that remain unsolved during RL, we apply Critique Fine-Tuning by converting all-zero-reward failures into diagnostic, repair, and new reasoning trace supervision for the next SFT stage. Experiments on two SLMs across five reasoning benchmarks show that our method consistently improves over representative SFT, distillation, and RL baselines. Our results highlight the importance of coordinating data difficulty across SFT and RL for effective SLM reasoning post-training.

Momentum

Related topics

Latest papers 707