cs.LGFeb 3, 2026

SLOPE: Optimistic Potential Landscape Shaping for Model-based Reinforcement Learning

Authors: Yao-Hui Li, Zeyu Wang, Xin Li, Wei Pang, Yingfang Yuan, Zhengkun Chen, Boya Zhang, Riashat Islam, 

Organizations: Beijing Institute of Technology, Beijing, China · Heriot-Watt University, Edinburgh, U.K. · Shenzhen Institutes of Advanced Technology, CAS, Shenzhen, China · Microsoft Research NY, USA · 

Abstract

Model-based reinforcement learning (MBRL) is sample-efficient but struggles in sparse reward settings. A critical bottleneck arises from the lack of informative gradients in sparse settings, where standard reward models often yield flat landscapes that struggle to guide planning. To address this challenge, we propose Shaping Landscapes with Optimistic Potential Estimates (SLOPE), a novel framework that shifts reward modeling from predicting sparse scalars to constructing informative potential landscapes. SLOPE employs optimistic distributional regression to estimate high-confidence upper bounds, which amplifies rare success signals and ensures sufficient exploration gradients. Evaluations on 30+ tasks across 5 benchmarks and real-world robotic deployments, demonstrate that SLOPE consistently outperforms leading baselines in fully sparse, semi-sparse, and dense rewards. The project code is available at https://github.com/Arya87/SLOPE.

Figures & tables

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Progress Reward Model for Reinforcement Learning via Large Language Models

    Oct 15, 2026Xiuhui Zhang, Ning Gao, Xingyu Jiang +4Reward Design for RLLLM-Guided RL

  2. Uncertainty-Aware LLM-Guided Policy Shaping for Sparse-Reward Reinforcement Learning

    Jun 4, 2026Ujjwal Bhatta, Utsabi Dangol, Sumaly Bajracharya +2Reinforcement LearningLLM-Guided RL

  3. Automating Potential-based Reward Shaping with Vision Language Model Guidance

    Jun 25, 2026Henrik Müller, Daniel KudenkoReward ShapingSparse-Reward RL