Sparse-Reward RL

RL: Reinforcement Learning

Sparse-reward reinforcement learning asks how agents can discover useful behavior when environments provide little feedback, a common obstacle in long-horizon robotics, games, and interactive tasks. Exploration methods such as intrinsic motivation, goal relabeling, curricula, and hierarchical skills help agents reach informative states in settings like MiniGrid, Montezuma’s Revenge, and MuJoCo. Other approaches derive denser credit from successful or failed trajectories, learned progress rewards, value estimates, or language-model feedback, including for agents evaluated on ALFWorld and WebShop. A central tension is that extra guidance can improve learning efficiency but may be noisy, encourage reward hacking, or shift the objective, so reliable credit assignment and transfer beyond familiar benchmarks remain open challenges.

Latest papers

164 papers31 papers in the last four weeks, up 107% on the four weeks before. 0.2% of all new papers.

Adaptive Correlation-Weighted Intrinsic Rewards for Reinforcement Learning
Feb 27, 2026Viet Bac Nguyen, Phuong Thai NguyenIntrinsic Reward Methods Reward Design for RL
Institute for Artificial Intelligence, VNU-University of Engineering and Technology, Hanoi, Vietnam
MAGE: Multi-scale Autoregressive Generation for Offline Reinforcement Learning
Feb 27, 2026Chenxing Lin, Xinhui Gao, Haipeng Zhang +7Offline RL Transformer-Based RL
Fujian Key Laboratory of Urban Intelligent Sensing and Computing, School of Informatics, Xiamen University (XMU), China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, XMU, China · Hong Kong University of Science and Technology +2
Task-Aware Exploration via a Predictive Bisimulation Metric
Feb 21, 2026Dayang Liang, Ruihan Liu, Lipeng Wan +2RL Exploration Visual RL
Department of Automation, Xiamen University, Xiamen, China · Department of Computer Science, Beijing Normal-Hong Kong Baptist University, Zhuhai, China · School of Artificial Intelligence, Xian Jiaotong University, Xian, China +1
PRISM: Parallel Reward Integration with Symmetry for MORL
Feb 20, 2026Finn van der Knaap, Kejiang Qian, Zheng Xu +1Equivariant Policy Learning Reinforcement Learning
University of Edinburgh · Meta Superintelligence Labs.
Memory-Based Advantage Shaping for LLM-Guided Reinforcement Learning
Feb 20, 2026Narjes Nourzad, Carlee Joe-WongMemory-Augmented RL Reinforcement Learning
University of Southern California · Carnegie Mellon University
MIRA: Memory-Integrated Reinforcement Learning Agent with Limited LLM Guidance
Feb 20, 2026Narjes Nourzad, Carlee Joe-WongMemory-Augmented RL Reinforcement Learning
University of Southern California, Electrical and Computer Engineering, Los Angeles, CA 90007, USA · Carnegie Mellon University, Electrical and Computer Engineering, Pittsburgh, PA 15213, USA
Retrospective In-Context Learning for Temporal Credit Assignment with Large Language Models
Feb 19, 2026Wen-Tse Chen, Jiayu Chen, Fahim Tajwar +4Step-Level Credit Assignment in RL Sparse-Reward RL
Carnegie Mellon University · The University of Hong Kong · Stanford University
HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents
Feb 18, 2026Jiangweizhi Peng, Yuanxin Liu, Ruida Zhou +4RL for LLM Agents Hierarchical RL
University of Minnesota · Northwestern University · Amazon AGI +2
Goldilocks RL: Tuning Task Difficulty to Escape Sparse Rewards for Reasoning
Feb 16, 2026Ilia Mahrooghi, Aryo Lotfi, Emmanuel AbbeRL for Language Model Reasoning Curriculum RL
EPFL · Apple
TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents
Feb 12, 2026Aladin Djuhera, Swanand Kadhe, Farhan Ahmed +3Tree Search LLM Agent Training
Technical University Munich · IBM Research
Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
Jan 26, 2026Shobhita Sundaram, John Quan, Ariel Kwiatkowski +3Self-Play RL RL for Language Model Reasoning
1MIT · 2Meta FAIR · 3New York University
BaNEL: Exploration Posteriors for Generative Modeling Using Only Negative Rewards
Oct 10, 2025Sangyun Lee, Brandon Amos, Giulia FantiGenerative Modeling RL for Generative Models
Carnegie Mellon University
Self-Supervised Goal-Reaching Results in Multi-Agent Cooperation and Exploration
Sep 12, 2025Chirayu Nimonkar, Shlok Shah, Catherine Ji +1Multi-Agent Coordination RL Exploration
Princeton University
CONTHER: Context-Aware Reinforcement Learning for Robotic Manipulation with Sparse Rewards
Mar 20, 2025Maria Makarova, Qian Liu, Dzmitry TsetserukouReinforcement Learning Experience Replay
Intelligent Space Robotics Laboratory, Center for Digital Engineering, Skolkovo Institute of Science and Technology (Skoltech), 121205 Moscow, Russia · Department of Computer Science and Technology, Dalian University of Technology, China

Feb 27, 2026J/K move · Enter open · S save

Adaptive Correlation-Weighted Intrinsic Rewards for Reinforcement Learning

Viet Bac Nguyen, Phuong Thai Nguyen

Institute for Artificial Intelligence, VNU-University of Engineering and Technology, Hanoi, Vietnam

Intrinsic Reward Methods Reward Design for RL Sparse-Reward RL

Open paper AskPDF arXiv

We propose ACWI (Adaptive Correlation Weighted Intrinsic), an adaptive intrinsic reward scaling framework designed to dynamically balance intrinsic and extrinsic rewards for improved exploration in sparse reward reinforcement learning. Unlike conventional approaches that rely on manually tuned scalar coefficients, which often result in unstable or suboptimal performance across tasks, ACWI learns a state dependent scaling coefficient online. Specifically, ACWI introduces a lightweight Beta Network that predicts the intrinsic reward weight directly from the agent state through an encoder based architecture. The scaling mechanism is optimized using a correlation based objective that encourages alignment between the weighted intrinsic rewards and discounted future extrinsic returns. This formulation enables task adaptive exploration incentives while preserving computational efficiency and training stability. We evaluate ACWI on a suite of sparse reward environments in MiniGrid. Experimental results demonstrate that ACWI consistently improves sample efficiency and learning stability compared to fixed intrinsic reward baselines, achieving superior performance with minimal computational overhead.