RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold
Jul 30, 2026Songshuo Lu, Zhi Chen, Yaohua TangRL for Language Model Reasoning Multi-Teacher Knowledge Distillation
Moore Threads AI
Hierarchical Latent Reasoning for LLM-based Recommendation
Jul 30, 2026Peiyu Hu, Siying Gu, Weihai Lu +8Sequential Recommendation RL for Language Model Reasoning
1Xi’an Jiaotong-Liverpool University · 2Xiaohongshu · 3Peking University +1
ReDiPPO: Reference-Guided Value Calibration and Discrepancy-Aware Token Reweighting for Mathematical Reasoning
Jul 30, 2026Zhenrong Zhang, Fei Wu, Jun Du +2RL for Language Model Reasoning Credit Assignment in RL
1IFLYTEK Research · University of Science and Technology of China
Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning
Jul 30, 2026Haodong Zhu, Yangyang Ren, Yanjing Li +4RL for Language Model Reasoning Curriculum RL
1Beihang University · 2Zhongguancun Academy · 3Nanyang Technological university +1
ReCo: Reweighting GRPO Against Distributional Concentration
Jul 29, 2026Junoh Park, Junseo Hwang, Wonguk Cho +1Reasoning Diversity RL for Language Model Reasoning
Graduate School of Data Science, Seoul National University
Early Verdicts, Better Budgets: Sequential Adaptive Rollout Allocation for Compute-Efficient RLVR
Jul 28, 2026Pixel Nomand, Elena Voss, Marcus Hale +1Reinforcement Learning RL for Language Model Reasoning
University of Wisconsin–Madison · University of Washington
Training Language Models to Cooperate with Inference-Time Controllers
Jul 26, 2026Moumita Choudhury, Vanshaj Khattar, Jing Liu +4RL for Language Model Reasoning LLM Post-Training
College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, Massachusetts, USA · Mitsubishi Electric Research Laboratories, Cambridge, Massachusetts, USA
Offline-Online Curriculum RL for Multimodal Reasoning
Jul 26, 2026Wendi Deng, Hang Du, Guoshun Nan +11Reinforcement Learning RL for Language Model Reasoning
Beijing University of Posts and Telecommunications, Beijing, 100876, China. · BUPT Shenzhen Institute, ShenZhen, 518038, China. · Carnegie Mellon University, Pittsburgh, USA. +4
On the Impossibility of Unbiased and Length-Invariant Policy Optimization with Outcome Rewards
Jul 25, 2026Fei Ding, Yongkang Zhang, Yuhao Liao +2Policy Gradient RL for Language Model Reasoning
Alibaba Group · Tsinghua University
Traceable LLM Reasoning for Fake-Order Fraud Detection
Jul 25, 2026Siqi You, Bingsong Xu, Zhixian Zheng +4RL for Language Model Reasoning Financial Fraud Detection
ByteDance · Hangzhou, China · Beijing, China +2
Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills
Jul 24, 2026Siyuan Huang, Pengyu Cheng, Haotian Liu +10RL for Language Model Reasoning LLM Agent Skill Learning
Qwen Large Model Application Team, Alibaba · The Chinese University of Hong Kong · Renmin University of China +5
Deconstructing Off-Policy Ratios: Entropy-Normalized Trust Regions for Asynchronous Reinforcement Learning
Jul 24, 2026Guanqun Zhao, Zijun Xie, Binbin Zheng +5Asynchronous RL RL for Language Model Reasoning
Beijing University of Posts and Telecommunications · Baidu Inc. · Peking University +2
Learning as Reasoning Unfolds: Progressive Rollout Allocation for Efficient Reinforcement Learning
Jul 24, 2026Heyang Jiang, Henry Liu, Baharan MirzasoleimanRL for Code Generation RL for Language Model Reasoning
University of California, Los Angeles
QLPO: Quadrant-weighted Sampling for Length-aware Policy Optimization
Jul 23, 2026Siwei Chen, Siqi Chen, Xupeng Miao +1Overthinking in Language Models RL for Language Model Reasoning
School of Computer Science, Peking University · Beijing Key Laboratory of Software and Hardware Cooperative Artificial Intelligence Systems · Department of Electronic Engineering, Tsinghua University +1
AdaKP: Online Adaptive Knowledge-Point Selection for Reasoning-Oriented Reinforcement Learning
Jul 23, 2026Zibin Meng, Zhenyu Zhao, Chunqiang RunKnowledge Augmentation for Language Models RL for Language Model Reasoning
The Hong Kong University of Science and Technology · University of the Chinese Academy of Sciences
Training Large Language Models for Self-Explanation Faithfulness
Jul 23, 2026Yeoktatt Cheah, María Pérez-Ortiz, Noah Y. Siegel +1Faithfulness of Language Model Explanations LLM Interpretability
Centre for AI, Department of Computer Science University College London · Imperial College London
The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning
Jul 23, 2026Ishan S. KshirsagarRL for Language Model Reasoning Continuous Latent Reasoning
Independent Researcher
PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference
Jul 22, 2026Niqi Lyu, Pengtao Shi, Wei Qiu +4RL for Language Model Reasoning Cost-Aware Inference
Pyromind Dynamics Inc.
SLPO: Scaling Latent Reasoning via a Surrogate Policy
Jul 22, 2026Runyang You, Zhiyuan Liu, Yongqi Li +1Reinforcement Learning RL for Language Model Reasoning
The Hong Kong Polytechnic University · Sichuan University · Work done as an intern at PolyU.
Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning
Jul 21, 2026Lizhe Fang, Weizhou Shen, Tianyi Tang +1LLM Grounding Reward Shaping
Peking University · Qwen Team of Alibaba Group · Alibaba Group ATH
Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information
Jul 21, 2026Priyank Agrawal, Ankur Samanta, Shervin Ghasemlou +4Reinforcement Learning RL for Language Model Reasoning
1Meta AI · 2Columbia University · *Work done at Meta
The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation
Jul 21, 2026Michael Jungo, Aixiu AnRL for Language Models RL for Language Model Reasoning
AIBEX, University of Fribourg Fribourg, Switzerland · iCoSys, University of Applied Sciences and Arts Western Switzerland Fribourg, Switzerland
REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning
Jul 21, 2026Yunjie Chen, Xiaoxin Chen, Fang WangRL for Language Models RL for Language Model Reasoning
vivo AI Lab, Shenzhen, China · Department of Computer Science, Sun Yat-Sen University, Guangzhou, China
Fishing Out Free Riders: Shapley-Based Reward Attribution for Parallel Reasoning via Reinforcement Learning
Jul 21, 2026Wentao Zhang, Haoyu Zhang, Xinke Jiang +7RL for Language Model Reasoning Shapley Value Attribution
ShanghaiTech University · City University of Hong Kong · Peking University +2
H $^2$ SD: Hybrid Hindsight Self-Distillation
Jul 21, 2026Qiye Cai, Yichuan Ma, Peiji Li +6RL for Language Model Reasoning Language Model Distillation
1Shanghai Artificial Intelligence Laboratory · 2Harbin Institute of Technology · 3Fudan University +1
Reasoning Error from Known Fact: Step-Level Self-Consistency Group Relative Policy Optimization for LLM
Jul 21, 2026Xiaomeng Hu, Jiaqi Hu, Hao Chen +4RL for Language Model Reasoning Hallucination in Language Models
Zhejiang University
Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learning
Jul 20, 2026Jia Ao Sun, Hao Yu, Fengran Mo +4LLM Reasoning with Graphs RL for Language Model Reasoning
Université de Montréal · Mila – Québec AI Institute · McGill University +2
RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts
Jul 20, 2026Yuxin Xiong, Xunyi Jiang, Rohan Surana +8Reinforcement Learning RL for Language Model Reasoning
University of California San Diego · 2Adobe Research
MADA-RL: Multi-Agent Debate-Aware Reinforcement Learning for Parameter-Efficient Reasoning in Compact Models
Jul 20, 2026Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov +3RL for Language Model Reasoning Multi-Agent Debate
Bosch Center for Artificial Intelligence, Germany · LMU Munich, Germany · University of Oslo, Norway +2
DeLIVeR: Decomposed Learning for Information-grounded Veracity Recognition via Reinforced Knowledge Graph Exploration
Jul 20, 2026Cong Hoan Nguyen, Thomas Hoang, Hieu Minh Duong +1LLM Grounding RL for Language Model Reasoning
University of Louisville, Louisville KY 40292, USA. · Denison University, Granville, Ohio 43023, USA.

Jul 30, 2026, cs.LGJ/K move · Enter open · S save

Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold

Songshuo Lu, Zhi Chen, Yaohua Tang

Moore Threads AI

RL for Language Model Reasoning Multi-Teacher Knowledge Distillation On-Policy Distillation

Open paper AskPDF arXiv

A single reinforcement-learning run can produce a strong reasoner yet an incomplete teacher: it often amplifies only a subset of the valid solution modes. We argue that reinforcement learning (RL)-trained policies should therefore be viewed as local probes of a multi-basin reasoning solution manifold, rather than as globally reliable supervisors. Based on this view, we propose an expand-then-compress framework that couples teacher construction with multi-teacher policy distillation. In the expansion stage, Residual Group Relative Policy Optimization (RGRPO) trains a sequence of teachers from a common initialization and redirects each later round toward examples not yet covered by the accumulated teacher union. In the compression stage, reliability-gated Teacher-Union On-policy Distillation (TU-OPD) lets the student learn from its own response prefixes. For each example, only reliable teachers contribute, and their sampled-token OPD losses are weighted by their per-example quality. We further introduce Consensus-Residual Decomposition, which preserves a winner teacher's excess token preferences over its reliable peers, preventing specialist behavior from being suppressed during teacher aggregation. Experiments on mathematical reasoning, code generation, and instruction following show that the resulting Qwen3-1.7B student consistently outperforms the strongest individual teacher across all three domains, yielding relative improvements of 2.0%, 8.3%, and 6.9%, respectively, while retaining single-model inference. These results establish a simple but powerful principle: stronger students can be obtained not by selecting a single better teacher, but by deliberately constructing and compressing a complementary teacher union.

Momentum

Related topics

Latest papers 707