RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

Spectral Rewiring for Exploration, Purification, and Model Merging
Jul 3, 2026Zhilong Zhang, Hongli Yu, Huan-ang Gao +5RL for Language Model Reasoning RL Post-Training
SIA-Lab of Tsinghua AIR and ByteDance Seed · Institute for AI Industry Research (AIR), Tsinghua University
Reinforcement Learning for Evidence-Seeking Diagnostic Reasoning with Large Language Models
Jul 3, 2026Shengyi Hua, Kangzhe Hu, Conghui He +2Medical Diagnosis RL for Language Model Reasoning
Qing Yuan Research Institute, Shanghai Jiao Tong University · Shanghai AI Laboratoty · Shanghai Innovation Institute
Reward Granularity in RLVR: Comparing Process and Outcome Reward Structures for Mathematical Reasoning in Small Language Models
Jul 3, 2026Anagha Radhakrishna Palandye, Rebecca Glick, Osheen KaulProcess Reward Models RL for Language Model Reasoning
New York University
DemoPSD: Disagreement-Modulated Policy Self-Distillation
Jul 2, 2026Yunhe Li, Hao Shi, Wenhao Liu +5On-Policy Self-Distillation RL for Language Model Reasoning
1City University of Hong Kong · 2Tsinghua University · 3Shenzhen University of Advanced Technology +1
Evidence-State Rewards for Long-Context Reasoning
Jul 2, 2026Ya Gao, Pekka MarttinenProcess Reward Models RL for Language Model Reasoning
1Aalto University
Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL
Jul 1, 2026Juliette Decugis, Sean O'Brien, Francis Bach +2Policy Gradient RL for Language Model Reasoning
FAIR at Meta · Work done at Meta, now working at the University of California, San Diego · Inria, Ecole Normale Supérieure
FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning
Jul 1, 2026Zhiyun Zhang, Liwen Sun, Xiang Qian +1RL for Language Model Reasoning Process Supervision
Carnegie Mellon University · Stanford University School of Medicine · Xlue
Graph-Native Reinforcement Learning Enables Traceable Scientific Hypothesis Generation through Conceptual Recombination
Jul 1, 2026Subhadeep Pal, Shashwat Sourav, Tirthankar Ghosal +1LLM Reasoning with Graphs RL for Language Model Reasoning
Department of Civil and Environmental Engineering MASSACHUSETTS INSTITUTE OF TECHNOLOGY Cambridge, MA, USA · Department of Physics Washington University in St. Louis St. Louis, MO, USA · Computing and Computational Sciences Directorate Oak Ridge National Laboratory Oak Ridge, TN, USA +3
Linguistic Relative Policy Optimization for Video Anomaly Reasoning
Jul 1, 2026Jiaxu Leng, Jiankang Zheng, Mengjingcheng Mo +4VLM Adaptation RL for Language Model Reasoning
School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China · Chongqing College of Artificial Intelligence, Chongqing, China
Know When to Stop: Segment-Level Credit Assignment for Reducing Overthinking
Jul 1, 2026Chia-Hsuan Lee, Sihui Dai, Mingyang Zhou +4LLM Self-Correction Overthinking in Language Models
Capital One
SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
Jun 30, 2026Ruikang Zhao, Zhenting Wang, Han Gao +1Reinforcement Learning RL for Language Model Reasoning
Technical University of Denmark · MBZUAI Institute of Foundation Models · Iowa State University +1
Verifiable Rewards for Calibrated Probabilistic Forecasting
Jun 30, 2026Sadanand Singh, Allam Reddy, Manan ChopraProbability Calibration RL for Language Model Reasoning
Cascade Research
GRPO, Dr. GRPO, and DAPO Are Three Operations on One Number: The Group-Standard-Deviation Identity
Jun 30, 2026Yong Yi Bay, Kathleen A. YearickReinforcement Learning RL for Language Model Reasoning
PhD, University of Illinois at Urbana-Champaign
RareDxR1: Autonomous Medical Reasoning for Rare Disease Diagnosis Beyond Human Annotation
Jun 30, 2026Deyang Jiang, Haoran Wu, Ziyi Wang +4Rare Disease Diagnosis RL for Language Model Reasoning
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Peking Union Medical College Hospital, Beijing, China · School of Future Technology, University of Chinese Academy of Sciences, Beijing, China
Addressing Over-Refusal in LLMs with Competing Rewards
Jun 30, 2026Taeyoun Kim, Aviral KumarRL for Language Model Reasoning LLM Safety
Carnegie Mellon University · SynthLabs
Which Tokens Matter? Adaptive Token Selection for RLVR with the Relative Surprisal Index
Jun 30, 2026Outongyi Lv, Yanzhao Zheng, Yuanwei Zhang +5Reinforcement Learning RL for Language Model Reasoning
1ModelScope Team, 2Alibaba Group, Hangzhou, China. · 3Shanghai Jiao Tong University, Shanghai, China.
Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models
Jun 29, 2026Subramanyam Sahoo, Aman Chadha, Vinija Jain +1Reward Modeling Reward Hacking
1Horizon Research · 2Apple · 3Meta +1
Experience Augmented Policy Optimization for LLM Reasoning
Jun 29, 2026Jinda Lu, Kexin Huang, Junkang Wu +7Reinforcement Learning RL for Language Model Reasoning
University of Science and Technology of China · 2Peking University · 3Dartmouth College +1
DRIFT: Difficulty Routing Self-DIstillation with Rhythm-Gated Exploration and Success BuFfer Training
Jun 29, 2026Haisen Luo, Yiwei Liu, Haoning Wang +13Self-Training for Language Models RL for Language Model Reasoning
3Beike Language and Intelligence · 2ENS Paris-Saclay · 1Tsinghua University
Are We Measuring Strategy or Phrasing? The Gap Between Surface- and Approach-Level Diversity in LLM Math Reasoning
Jun 29, 2026Sangmook Lee, Minbeom Kim, Jeonghye Kim +3LLM Evaluation Reasoning Diversity
Seoul National University · KAIST
LatentRevise: Learning from Zero-Hit Reasoning
Jun 29, 2026Yiqiu Guo, Xueting Han, Qi Jia +2RL for Language Model Reasoning Mathematical Reasoning
1Fudan University · 2Microsoft Research Asia · 3Shanghai AI Laboratory +1
PS-PPO: Prefix-Sampling PPO for Critic-Free RLHF
Jun 29, 2026Doo Hwan Hwang, Kee-Eung KimReinforcement Learning RL for Language Model Reasoning
Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea.
The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning
Jun 28, 2026Jing Liang, Hongyao Tang, Yi Ma +9Reinforcement Learning RL for Language Model Reasoning
1Tianjin · 2Alibaba
To Reason or to Fabricate: Reasoning Without Shortcuts via Hint-Anchored Pairwise Aggregation
Jun 28, 2026Jiuheng Lin, Chen Zhang, Yansong FengCoT Faithfulness Reward Modeling
Wangxuan Institute of Computer Technology, Peking University
Process Advantage Signal Shaping: A Paradigm-Agnostic Middleware for Process-Supervised RL in LLM Reasoners
Jun 28, 2026Chao Wang, Hongtao Tian, Tao Yang +3Reinforcement Learning Process Reward Models
Tsinghua University · Work done while interning at Tencent. · WeChat, Tencent
BV-Blend: Uncertainty-Weighted Historical Baselines for Stable Critic-Free RL with Verifiable Rewards
Jun 27, 2026Yupeng Chang, Yuan Wu, Yi ChangReinforcement Learning RL for Language Model Reasoning
School of Artificial Intelligence, Jilin University · 2Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China · 3International Center of Future Science, Jilin University
Tandem Reinforcement Learning with Verifiable Rewards
Jun 26, 2026Difan Jiao, Raghav Singhal, Robert West +1Reinforcement Learning RL for Language Model Reasoning
University of Toronto · EPFL
MER-R1: Multimodal Emotion Reasoning via Slow-Fast Thinking Synergy
Jun 26, 2026Zhiyuan Han, Beier Zhu, Wenwen Tong +8RL for Language Model Reasoning Multimodal Large Language Models
University of Science and Technology of China · 2SenseTime Research · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center +1
Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
Jun 25, 2026Zhenhua Xu, Dongsheng Chen, Jian Li +7Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models
1Zhejiang University · 2Tencent Youtu Lab
RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning
Jun 25, 2026Rongjian Chen, Jianmin Hu, Kejiang Ye +1Reinforcement Learning RL for Language Model Reasoning
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · University of Chinese Academy of Sciences, Beijing, China · Southern University of Science and Technology, Shenzhen, China

Jul 3, 2026, cs.LGJ/K move · Enter open · S save

Spectral Rewiring for Exploration, Purification, and Model Merging

Zhilong Zhang, Hongli Yu, Huan-ang Gao, Hanlin Wu+4

SIA-Lab of Tsinghua AIR and ByteDance Seed · Institute for AI Industry Research (AIR), Tsinghua University

RL for Language Model Reasoning RL Post-Training Model Merging

Open paper AskPDF arXiv

Reinforcement learning has become a standard post-training recipe for large language models, but dense full-parameter updates create two deployment-relevant bottlenecks: suppressed reasoning performance, often reflected by premature saturation of test-time scaling, and interference when consolidating multiple capabilities through multi-domain training or model merging. We show that the reasoning-effective component of these updates is largely concentrated in the base model's spectral space, motivating Subspace-Aligned Rewiring (SAR), a post-hoc editing method that retains this spectral core while removing orthogonal components. SAR therefore preserves reasoning gains and filters residual update directions that suppress performance or amplify cross-domain interference. Across several model families and scales, SAR extracts compact reasoning cores using as little as approximately 0.58% of total parameters: it preserves over 99% of post-training performance and improves high-k exploration in mathematical reasoning, and generalizes to agentic coding by improving six of seven open benchmarks on an in-house model. SAR also purifies mixed-domain training updates by releasing suppressed coding capability while maintaining math reasoning and instruction following. It further enables model merging across experts, yielding cross-domain generalization that surpasses previous merging baselines and even the best single-domain experts. Overall, SAR shows that extracting reasoning-effective updates from parameter geometry can serve as a training-free mechanism to improve reasoning and multi-domain performance.

Momentum

Related topics

Latest papers 707