Sparse-Reward RL

RL: Reinforcement Learning

Overview

Sparse-reward reinforcement learning asks how agents can discover useful behavior when environments provide little feedback, a common obstacle in long-horizon robotics, games, and interactive tasks. Exploration methods such as intrinsic motivation, goal relabeling, curricula, and hierarchical skills help agents reach informative states in settings like MiniGrid, Montezuma’s Revenge, and MuJoCo. Other approaches derive denser credit from successful or failed trajectories, learned progress rewards, value estimates, or language-model feedback, including for agents evaluated on ALFWorld and WebShop. A central tension is that extra guidance can improve learning efficiency but may be noisy, encourage reward hacking, or shift the objective, so reliable credit assignment and transfer beyond familiar benchmarks remain open challenges.

Latest papers

CardsList

175 papers9 papers in the last four weeks, down 74% on the four weeks before. 0.1% of all new papers.

  1. WATERFALL: Workflow for Adaptive Training with Evolutionary Reward Formulation and Automated Learning Loops

    Nov 17, 2026Eleftherios Triantafyllidis, Filippos Christianos, Zhibin Li +1Reward Design for RLSparse-Reward RL

  2. MaxIM: Maximally Informative Incremental Summarization via Reinforcement Learning

    Nov 9, 2026Jihwan Jeong, Guy Tennenholtz, Yinlam Chow +2Text SummarizationMemory-Augmented RL

  3. Learning to Act with World Models by Valuing What Comes Next

    Nov 9, 2026Chenghao Gao, Zhi Zhang, Dian Jin +2Model-Based RLSparse-Reward RL

  4. The Blindfolded Controller: Decision Model-Guided Training for Reinforcement Learning

    Nov 5, 2026Muhammad Ibrahim Khan, Aniket Dixit, Faizan Ahmed +1Reinforcement LearningSparse-Reward RL

  5. Using Experience Twice: Dynamic Process Rubrics for Privileged Token-Level Credit Assignment

    Nov 4, 2026Shumao Wu, Kongcheng Zhang, Jianqiang Huang +1Token-Level Credit AssignmentFine-Grained Credit Assignment

  6. Hindsight Credit Assignment over Trees for Agentic RL

    Nov 2, 2026Penghong Zhao, Yihao Chen, Maoheng Yuan +8Reinforcement LearningRL for LLM Agents

  7. Reinforcement Learning for the Game of Balatro: A Fast Simulator and Two-Policy Training Approach

    Oct 28, 2026Ajay Vikram Periasami, Anjaneya Gupta, Ethan Cheng +1RL for Game AIMulti-Agent Reinforcement Learning

  8. Effectiveness of Curriculum Learning Depends on Reward Sparsity and Competing Optima

    Oct 22, 2026John J. Vastola, Ann Huang, Satpreet H. Singh +2Curriculum RLReinforcement Learning

  9. Progress Reward Model for Reinforcement Learning via Large Language Models

    Oct 15, 2026Xiuhui Zhang, Ning Gao, Xingyu Jiang +4Reward Design for RLLLM-Guided RL

  10. Bellman-Consistent Trajectory RL for LLM Arithmetic Reasoning

    Oct 15, 2026Lianfa LiRL for Language Model ReasoningReinforcement Learning

  11. Long-Horizon RL Needs Muon

    Oct 14, 2026Kai Ruan, Jinghao Lin, Tao Chen +3Long-Horizon RLLong-Horizon LLM Agents

  12. Reinforcement Learning via Advantage Guidance

    Oct 14, 2026Zhiyuan Zhou, Sergey LevineReinforcement LearningOffline RL

  13. Contextual Credit Policy Optimization

    Oct 13, 2026Huiwen Han, Ziyao Zeng, Zihao Zhu +6RL for LLM AgentsStep-Level Credit Assignment in RL

  14. Adaptive-Group Process Rewards for GUI Agents

    Oct 13, 2026Zirui Shang, Yang Liu, Xin Shu +3RL for GUI AgentsProcess Reward Models

  15. DeltaTTG: Training Multi-Turn Agents with a Token-to-Go Progress Meter

    Oct 13, 2026Xuyan Ye, Xiyan Fu, Shengda Fan +3RL for LLM AgentsReinforcement Learning

  16. LLM Value Functions for Credit Assignment in Agentic RL

    Oct 11, 2026Kaijie Zhu, Weixiang Yan, Omar Attia +3RL for LLM AgentsAgentic RL

  17. Harnessing VLM in Embodied RL: From Reward Regressor to Success Detector via Exploration-Augmented Key-Observation Gating

    Oct 11, 2026Xin Liu, Yutong Jiang, Jian Su +4Reward Design for RLReinforcement Learning

  18. PTGPO: Prefix Tree Group Policy Optimization

    Oct 10, 2026Ruolin Chen, Yang Li, Jihang Wang +4RL for LLM AgentsTemporal Credit Assignment

  19. AlphaNIST: Emergent Assembly Skills from Binary Task Rewards

    Oct 9, 2026Octi Zhang, Iretiayo Akinola, Rosario Scalise +12Robotic AssemblyLong-Horizon Robotic Manipulation

  20. Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight

    Oct 6, 2026Haoxiang Zhang, Qinglin Chen, Hiroaki Hayashi +9Agentic RLRL for Language Model Reasoning

  21. Task-Space Imitation Guidance for Efficient Reinforcement Learning

    Oct 5, 2026Salar Asayesh, Hossein Darani, Todd Cao +2Reward ShapingImitation Learning

  22. Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding

    Oct 5, 2026Hoang Phan, Minh Pham, Chau Pham +3RL for Language Model ReasoningLLM-Guided RL

  23. Bidirectional Voronoi-biased Exploration Curriculum for Reinforcement Learning

    Oct 2, 2026Juri Pfammatter, Kaixian Qu, Clemens Schwarke +2Curriculum RLRL Exploration

  24. EVOL: Simulator-Guided Evolutionary Expert Synthesis for Deployment-Free Learning Path Recommendation

    Oct 2, 2026Geonwoo Bang, Dongho Kim, Moohong MinRecommender SystemsSparse-Reward RL

  25. AdaStep: Adaptive Step Credit Weighting for Agentic Reinforcement Learning

    Oct 2, 2026Xin Wang, Wenhao Wu, Menghao Zhang +3Step-Level Credit Assignment in RLRL for LLM Agents

  26. Understanding Enrichment in Reinforcement Learning

    Oct 2, 2026Jinwoo Kim, Shraddha BarkeImportance Sampling for RLReinforcement Learning

  27. Dependency-Aware Reward Shaping for Agentic Reinforcement Learning

    Oct 1, 2026Ziyi Chen, Yan Zhang, Jianhui Wei +2RL for Language ModelsAgentic RL