RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs
Jun 9, 2026Zhichen Dong, Yang Li, Yuhan Sun +9Token-Level Credit Assignment RL for Language Model Reasoning
1Shanghai Jiao Tong University · 2Alibaba Group · 3Shanghai Artificial Intelligence Laboratory
Reasoning or Memorization? Direction-Aware Diversity Exploration in LLM Reinforcement Learning
Jun 9, 2026Jiangnan Xia, Yucheng Shi, Yu Yang +3Memorization in Language Models RL for Language Model Reasoning
University of Georgia · Tencent AI Lab · The Education University of Hong Kong +1
Dropout-GRPO: Variational Stochasticity for Continuous Latent Reasoning
Jun 8, 2026Wooil JungRL for Language Model Reasoning Group Relative Policy Optimization
Reasoning Arena: Trace Tournaments When Verifiable Rewards Fall Short
Jun 8, 2026Han Zhou, Adam X. Yang, Laurence Aitchison +2Pairwise Preference Learning Reward Modeling
University of Cambridge · 2Mistral AI · *Work done as an AI Scientist Intern at Mistral AI.
From Shortcuts to Reasoning: Robust Post-Training of Theory of Mind with Reinforcement Learning
Jun 8, 2026Jike Zhong, Yuxiang Lai, Ming Li +5Theory of Mind RL for Language Model Reasoning
University of Southern California · 2Emory University · University of Maryland, College Park +4
The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning
Jun 8, 2026Aakriti Agrawal, Souradip Chakraborty, Armin Saghafian +6Process Reward Models RL for Language Model Reasoning
University of Maryland, College Park · Capital One · University of Central Florida
INFUSER: Influence-Guided Self-Evolution Improves Reasoning
Jun 8, 2026Siyu Chen, Miao Lu, Beining Wu +7RL for Language Model Reasoning Group Relative Policy Optimization
Yale University · Stanford University · University of Chicago +2
Diverse Thinking Schemata Elicit Better Reasoning in Large Language Models
Jun 8, 2026Xinyue Liang, Yizhe Yang, Yu Bai +3Reasoning Diversity RL for Language Model Reasoning
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China
Knowledge Graphs and Reasoning LLMs for Finding Simple Yet Effective Transcriptomic Perturbation Predictors
Jun 7, 2026Jake Fawkes, Liam Hodgson, Jason HartfordRL for Language Model Reasoning Single-Cell Perturbation Modeling
Department of Computer Science, University College London, UK · Valence Labs, London, UK · Recursion, London, UK +1
Momentum for Reasoning: Dense Intrinsic Signals in Policy Optimization
Jun 7, 2026Hao Chen, Zhanming Shen, Liyao Li +8Intrinsic Reward Methods Reinforcement Learning
Zhejiang University · The Chinese University of Hong Kong · Eastern Institute of Technology
PAEC: Position-Aware Entropy Calibration for LLM Reasoning in RLVR
Jun 7, 2026Shumeng Yang, Yisu Liu, Jiayi Zheng +2RL for Language Model Reasoning Maximum Entropy RL
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · School of Artificial Intelligence, Beijing University of Posts and Telecommunications +2
Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models
Jun 7, 2026Yawen Shao, Jie Xiao, Kai Zhu +6RL for Language Model Reasoning Credit Assignment in RL
University of Science and Technology of China · Tongyi Lab · Northeastern University
Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Models
Jun 7, 2026Yang Zhou, Ranajoy Sadhukhan, Zhaofeng Sun +7RL for Language Model Reasoning Sparse Attention
Carnegie Mellon University · Cornell University · Intel +1
CATPO: Critique-Augmented Tree Policy Optimization
Jun 6, 2026Ayush Singh, Umang Goyal, Ankur DahiyaRL for Language Models Reinforcement Learning
Vision and Language Group Indian Institute of Technology Roorkee Roorkee, Uttarakhand, India
ConSteer-RL: Steering Reasoning Capabilities in Large Language Models via Confidence-Aware Reinforcement Learning
Jun 6, 2026Qing Miao, Yiming Zhao, Jing Yang +5Reward Shaping RL for Language Model Reasoning
Xi’an Jiaotong University · University of Science and Technology of China
Variational Proximal Policy Optimization
Jun 6, 2026Ousmane Amadou DiaRL for Language Model Reasoning Proximal Policy Optimization
The Easy, the Hard, and the Learnable: Confidence and Difficulty-Adaptive Policy Optimization for LLM Reasoning
Jun 6, 2026Zhanke Zhou, Xiangyu Lu, Chentao Cao +4RL for Language Model Reasoning Policy Optimization
1TMLR Group, Department of Computer Science, Hong Kong Baptist University · 2Stanford University · 3Sydney AI Centre, The University of Sydney
From Correctness to Utility: Gain-Based Prefix Evaluation for LLM Reasoning
Jun 5, 2026Yuhang Zhou, Yixin Cao, Guangnan YeProcess Reward Models RL for Language Model Reasoning
Fudan University · Shanghai Innovation Institute
RREDCoT: Segment-Level Reward Redistribution for Reasoning Models
Jun 4, 2026Mykyta Ielanskyi, Kajetan Schweighofer, Lukas Aichberger +1Reinforcement Learning RL for Language Model Reasoning
ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria · Cognizant AI Lab, San Francisco, USA · Work done while at Johannes Kepler University Linz. +1
TARPO: Token-Wise Latent-Explicit Reasoning via Action-Routing Policy Optimization
Jun 4, 2026Liting Zhang, Shiwan Zhao, Xuyang Zhao +3RL for Language Model Reasoning RL Fine-Tuning
TMCC, College of Computer Science, Nankai University, Tianjin, China
Cross-Epoch Adaptive Rollout Optimization for RL Post-Training
Jun 4, 2026Yiming Zong, Yige Wang, Jiashuo JiangSequential Decision Making RL for Language Model Reasoning
Department of Industrial Engineering & Decision Analytics, Hong Kong University of Science and Technology
Step-by-Step Optimization-like Reasoning in LLMs over Expanding Search Spaces
Jun 3, 2026Nicolás Astorga, Nabeel Seedat, Mihaela van der SchaarInference-Time Search Large Language Model-Guided Optimization
University of Cambridge
Selective-Advantage Entropy-Adaptive Horizon GRPO: Asymmetric Token-Level Discounting for Efficient Reinforcement Learning of Language Models
Jun 3, 2026Chirag Chawla, Rohan Charudatt Salvi, Madhav S. BaidyaReinforcement Learning RL for Language Model Reasoning
Indian Institute of Technology (BHU), Varanasi, India · Department of Computer Science, University of Illinois Chicago, IL 60607, USA
Reinforcement Learning from Rich Feedback with Distributional DAgger
Jun 3, 2026Rishabh Agrawal, Jacob Fein-Ashley, Paria RashidinejadDistributional RL RL for Language Model Reasoning
University of Southern California
GRAIL: Gradient-Reweighted Advantages for Reinforcement Learning with Verifiable Rewards
Jun 3, 2026Tej Deep Pala, Vernon Toh, Soujanya PoriaRL for Language Model Reasoning Credit Assignment in RL
DeCLaRe Lab, Nanyang Technological University
SocraticPO: Policy Optimization via Interactive Guidance
Jun 3, 2026Zirui Liu, Jie Ouyang, Qi Liu +8RL for Language Model Reasoning LLM-Guided RL
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China · iFLYTEK AI Research (Central China), iFLYTEK Co., Ltd
Rollout-Level Advantage-Prioritized Experience Replay for GRPO
Jun 3, 2026Gyeongtae Yoo, Sanghyeok Park, Soohyuk Jang +2RL for Language Model Reasoning Group Relative Policy Optimization
Department of Electrical and Computer Engineering, Seoul National University · Interdisciplinary Program in AI, Seoul National University · AIIS, ASRI, INMC, and ISRC, Seoul National University
GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling
Jun 3, 2026Guangcheng Zhu, Shenzhi Yang, Haobo Wang +9RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
1Zhejiang University · 2Ant Group
TD-Grokking: Learning from Zero-Reward Problems by Training-Time Decomposition
Jun 3, 2026Ningyuan Xi, Hao Xu, Hongsheng Xin +1RL for Language Model Reasoning Mathematical Reasoning
Department of Data Science, City University of Hong Kong. · Hong Kong Institute of AI for Science, City University of Hong Kong. · Li Auto Inc.
Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots
Jun 3, 2026Guangcheng Zhu, Shenzhi Yang, Haobo Wang +7Uncertainty Quantification RL for Language Model Reasoning
1Zhejiang University · 2Ant Group

Jun 9, 2026, cs.LGJ/K move · Enter open · S save

How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs

Zhichen Dong, Yang Li, Yuhan Sun, Weixun Wang+8

1Shanghai Jiao Tong University · 2Alibaba Group · 3Shanghai Artificial Intelligence Laboratory

Token-Level Credit Assignment RL for Language Model Reasoning

Open paper AskPDF arXiv

Token-level credit assignment remains a key obstacle for reinforcement learning (RL) in large language models (LLMs), where RL recipes typically treat all tokens equally, failing to distinguish decisive reasoning steps from routine formatting or fluent filler. Recent attempts leverage model-internal signals to assign finer-grained credit, but these are often point-wise heuristics that ignore the global structure of information propagation. We propose FlowTracer, an RL framework that traces answer-targeted reasoning flow on an attention-induced directed acyclic graph in which nodes correspond to tokens and edge capacities come from aggregated attention weights and derives token credit from this global structure. The edge capacities are reweighted to retain only the influence that can reach the answer region, while enforcing local flow conservation so intermediate tokens neither lose nor gain effective mass due to path length or irrelevant branches. On this graph, FlowTracer extracts an information-flow backbone connecting the question to the answer and scores tokens by flow throughput, revealing high-impact hubs and aggregation checkpoints that mediate long-range dependencies. These derived importances are used to shape token-level rewards, enabling learning signals to focus precisely on the tokens that route information toward (or away from) correct answers and delivering consistent performance gains across a range of reasoning tasks.

Momentum

Related topics

Latest papers 707