Sparse-Reward RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

19 papers in the last four weeks, up 138% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 111

All topics

The Two-Hump Problem: Bridging the Difficulty Gap in Mathematical Reinforcement Learning
Jun 19, 2026Lucas Fagan, Michele Tarquini, Ali Shehper +6RL Benchmarks Reinforcement Learning
Department of Mathematics, California Institute of Technology, Pasadena, CA, USA · Institute of Mathematics, University of Zurich, Zurich, Switzerland · Zero Latency Labs +1
Reward-free Pretraining for Reinforcement Learning via Occupancy Coverage Maximization
Jun 19, 2026Marco Pratticò, Pietro Novelli, Massimiliano Pontil +1Reinforcement Learning Exploration-Exploitation Tradeoff
Computational Statistics and Machine Learning - Istituto Italiano di Tecnologia, 16100 Genova, Italy · 2AI Centre, Computer Science Department, University College London, London, UK.
Evolutionary Discovery of Developmental Reward Schedules in Deep Reinforcement Learning
Jun 18, 2026Alan Nadelsticher RuvalcabaReinforcement Learning Sparse-Reward RL
College of Computing at the Georgia Institute of Technology, Georgia, USA · School of Engineering and Applied Science at the University of Pennsylvania, Pennsylvania, USA
Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes
Jun 15, 2026Tongyan Fang, Siyuan Huang, Naiyu Fang +6Credit Assignment in RL Robotic RL
1ACE Robotics · 2Shenzhen International Graduate School, Tsinghua University · 3The Chinese University of Hong Kong
3SPO: State-Score-Supervised Policy Optimization for LLM Agents
Jun 8, 2026Yu Han, Kailing Li, Yang Jiao +4Credit Assignment in RL Step-Level Credit Assignment in RL
School of Computer Science and Technology, East China Normal University · Fudan University · KAUST +1
PBSD: Privileged Bayesian Self-Distillation for Long-Horizon Credit Assignment
Jun 8, 2026Yang Tian, Rui Wang, Xumeng Wen +5Step-Level Credit Assignment in RL Reinforcement Learning with Verifiable Rewards
School of AI, Shanghai Jiao Tong University · 2XYZ AI Lab
Momentum for Reasoning: Dense Intrinsic Signals in Policy Optimization
Jun 7, 2026Hao Chen, Zhanming Shen, Liyao Li +8Intrinsic Reward Methods Reinforcement Learning
Zhejiang University · The Chinese University of Hong Kong · Eastern Institute of Technology
StainFlow: Entity-Stain Tracking and Evidence Linking for Process Rewards in GUI Agents
Jun 5, 2026Haojie Hao, Longkun Hao, Yihang Lou +8Reinforcement Learning RL for GUI Agents
Beihang University · Peking University · Renmin University of China +4
Uncertainty-Aware LLM-Guided Policy Shaping for Sparse-Reward Reinforcement Learning
Jun 4, 2026Ujjwal Bhatta, Utsabi Dangol, Sumaly Bajracharya +2Reinforcement Learning LLM-Guided RL
USD AI Research, Department of Computer Science University of South Dakota Vermillion, SD 57069, USA
RREDCoT: Segment-Level Reward Redistribution for Reasoning Models
Jun 4, 2026Mykyta Ielanskyi, Kajetan Schweighofer, Lukas Aichberger +1Reinforcement Learning RL for Language Model Reasoning
ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria · Cognizant AI Lab, San Francisco, USA · Work done while at Johannes Kepler University Linz. +1
When Denser Credit Is Not Enough: Evidence-Calibrated Policy Optimization for Long-Horizon LLM Agent Training
Jun 4, 2026Yuanfan Li, Qi Zhou, Wenjing Duan +1Credit Assignment in RL Step-Level Credit Assignment in RL
X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Faculty of Electronic and Information Engineering, Xi’an Jiaotong University
TD-Grokking: Learning from Zero-Reward Problems by Training-Time Decomposition
Jun 3, 2026Ningyuan Xi, Hao Xu, Hongsheng Xin +1RL for Language Model Reasoning Mathematical Reasoning
Department of Data Science, City University of Hong Kong. · Hong Kong Institute of AI for Science, City University of Hong Kong. · Li Auto Inc.
Episodic Memory Temporal Consistency for Cooperative Multi-Agent Reinforcement Learning
Jun 3, 2026Zicheng Zhao, Yu Lan, Chengzhengxu Li +2Reinforcement Learning Game-Playing Agents
Xi’an Jiaotong University · Queen Mary University of London
ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents
Jun 2, 2026Zheng Liu, Longxiang Zhang, Xintong Wang +8Reward Modeling Process Reward Models
Tsinghua University · Alibaba Group · Peking University +1
Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
Jun 1, 2026Christian Scherer, Joe Watson, Theo Gruner +3Reinforcement Learning Robot Skill Learning
Technical University of Darmstadt · Zuse School ELIZA · University of Oxford +2
HPO: Hysteretic Policy Optimization for Stable and Efficient Training under Sparse-Reward Regime
May 28, 2026Mohamed Sana, Nicola Piovesan, Antonio De Domenico +2Group Relative Policy Optimization Policy Optimization
Paris Research Center, Huawei Technologies, Boulogne-Billancourt, France · Huawei Technologies, China
LLM-ALSO: LLM-Driven Adaptive Learning-Signal Optimization for Multi-Agent Reinforcement Learning
May 28, 2026Xiaoguang Wu, Zhi Zheng, Hui XiongReward Shaping LLM-Guided RL
University of Science and Technology of China · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR, China · 2Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)
When LLM Reward Design Fails: Diagnostic-Driven Refinement for Sparse Structured RL
May 27, 2026Youting Wang, Yuan Tang, Bowen Liu +2Reward Shaping LLM-Guided RL
ARMS: Automatic Reward Shaping for Sparse-Reward Multi-Agent Reinforcement Learning
May 22, 2026Elie Abboud, Oren GalMulti-Agent System Optimization Sparse-Reward RL
Department of Marine Technologies University of Haifa
LamPO: A Lambda Style Policy Optimization for Reasoning Language Models
May 20, 2026Redacted by arXivRL for Language Model Reasoning Group Relative Policy Optimization
Pinterest, San Francisco, CA 94107, USA
PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization
May 18, 2026Wonjoong Kim, Yeonjun In, Sangwu Park +2Reward Modeling Credit Assignment in RL
KAIST · Yonsei University
SAPO: Step-Aligned Policy Optimization for Reasoning-Based Generative Recommendation
May 17, 2026Zaiyi Zheng, Guanghui Min, Yaochen Zhu +4RL for Language Model Reasoning Credit Assignment in RL
University of Virginia · 2Nokia
Mind Dreamer: Untethering Imagination via Active Causal Intervention on Latent Manifolds
May 15, 2026Shaojun Xu, Xiaoling Zhou, Yihan Lin +4Reinforcement Learning Credit Assignment in RL
Center for Brain-Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Pen-Tung Sah Institute of Micro-Nano Science and Technology, Xiamen University, Xiamen, China +1
VSPO: Vector-Steered Policy Optimization for Behavioral Control
May 15, 2026Xuechen Zhang, Zijian Huang, Kai Yang +3Language Model Steering RL for Language Model Reasoning
University of Michigan
Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
May 12, 2026Yuanda Xu, Hejian Sang, Zhengze Zhou +3Reinforcement Learning RL for Language Model Reasoning
Offline Policy Evaluation for Manipulation Policies via Discounted Liveness Formulation
May 12, 2026Hao Wang, Joshua Bowden, Colton Crosby +1RL for Robotics Reinforcement Learning
University of Southern California · Stanford University
RankQ: Offline-to-Online Reinforcement Learning via Self-Supervised Action Ranking
May 11, 2026Andrew Choi, Wei XuQ-Learning Robotic RL
Horizon Robotics
XQCfD: Accelerating Fast Actor-Critic Algorithms with Prior Data and Prior Policies
May 11, 2026Daniel Palenicek, Florian Vogt, Joe Watson +3Robotic RL Learning from Demonstration
Technical University of Darmstadt · KTH Royal Institute of Technology · University of Oxford +2
PhysEDA: Physics-Aware Learning Framework for Efficient EDA With Manhattan Distance Decay
May 11, 2026Zetao YangEfficient Neural Network Inference Reward Shaping
School of Mathematics and Statistics Beijing Institute of Technology Beijing, China
ACSAC: Adaptive Chunk Size Actor-Critic with Causal Transformer Q-Network
May 10, 2026Qian Chen, Junqiao Zhao, Hongtu Zhou +4Reinforcement Learning Transformer-Based RL
Tongji University

Jun 19, 2026, cs.LGJ/K move · Enter open · S save

The Two-Hump Problem: Bridging the Difficulty Gap in Mathematical Reinforcement Learning

Lucas Fagan, Michele Tarquini, Ali Shehper, Maksymilian Manko+5

Department of Mathematics, California Institute of Technology, Pasadena, CA, USA · Institute of Mathematics, University of Zurich, Zurich, Switzerland · Zero Latency Labs+1

RL Benchmarks Reinforcement Learning Sparse-Reward RL

Open paper AskPDF arXiv

Mathematical search problems present a unique challenge for Reinforcement Learning (RL) due to vast search spaces and sparse rewards. In previous works, the Andrews-Curtis (AC) conjecture was established as an illustrative example of such problems. In this work, we identify a critical structural barrier in the AC landscape: a "Two-Hump" distribution, where problem instances are either trivially solvable or effectively impossible, with a scarcity of intermediate "hard-but-solvable" instances required for effective learning. We tackle this challenge through two primary avenues: novel data generation techniques to populate the difficulty gap, and significant algorithmic enhancements including the introduction of supermoves and Transformer-based architectures. We demonstrate substantial performance improvements over previous baselines, and release new comprehensive benchmark datasets including AC-19 (125,192 AC-trivial presentations of varying difficulty with length at most 19) and AC-1M (1,136,154 hard AC-trivial presentations of length at most 30), the first large-scale, publicly available datasets of this kind.

Momentum

Related topics

Latest papers 111