Hierarchical Reward

Hierarchical reward structures in reinforcement learning aim to improve agent training by decomposing complex tasks into simpler sub-goals, providing more effective guidance and faster learning. Current research focuses on developing methods to automatically discover or design these hierarchies, often employing techniques like contrastive learning and incorporating domain knowledge (e.g., physics models) to enhance efficiency and generalization. This approach is proving particularly valuable in challenging domains such as robotics and language model alignment, where sparse or inconsistent feedback makes traditional reinforcement learning methods less effective, leading to improved performance and sample efficiency.

Papers

March 11, 2024

ALaRM: Align Language Models via Hierarchical Rewards Modeling
Yuhang Lai, Siyuan Wang, Shujun Liu, Xuanjing Huang, Zhongyu Wei
Large Language Model Language Model Large Language Model Alignment Hierarchical Reward Alarm Inefficiency

March 4, 2024

Purpose for Open-Ended Learning Robots: A Computational Taxonomy, Definition, and Operationalisation
Gianluca Baldassarre, Richard J. Duro, Emilio Cartoni, Mehdi Khamassi, Alejandro Romero, Vieri Giuliano Santucci
Comprehensive Taxonomy Open Ended Definition Defining Redefinition Self Generated Sensory Grounding Hierarchical Reward

July 7, 2023

Discovering Hierarchical Achievements in Reinforcement Learning via Contrastive Learning
Seungyong Moon, Junyoung Yeom, Bumsoo Park, Hyun Oh Song
Reinforcement Learning Contrastive Learning Long Term Proximal Policy Optimization Teacher Student Distillation Hierarchical Reward

April 30, 2023

Learning Achievement Structure for Structured Exploration in Domains with Sparse Reward
Zihan Zhou, Animesh Garg
Reinforcement Learning Environment Exploration Sparse Reward Domain Name Directed Exploration Learning Structure Hierarchical Reward

January 21, 2023

The configurable tree graph (CT-graph): measurable problems in partially observable and distal reward environments for lifelong reinforcement learning
Andrea Soltoggio, Eseoghene Ben-Iwhiwhu, Christos Peridis, Pawel Ladosz, Jeffery Dick, Praveen K. Pilly, Soheil Kolouri
Reward Function Sparse Reward Sparse Reward Environment Lifelong Reinforcement Learning Memory Less Agent Solved Problem Hierarchical Reward

May 26, 2022

Hierarchical Reward

Papers

ALaRM: Align Language Models via Hierarchical Rewards Modeling

Purpose for Open-Ended Learning Robots: A Computational Taxonomy, Definition, and Operationalisation

Discovering Hierarchical Achievements in Reinforcement Learning via Contrastive Learning

Learning Achievement Structure for Structured Exploration in Domains with Sparse Reward

The configurable tree graph (CT-graph): measurable problems in partially observable and distal reward environments for lifelong reinforcement learning

Physics-Guided Hierarchical Reward Mechanism for Learning-Based Robotic Grasping

Multi-Phase Multi-Objective Dexterous Manipulation with Adaptive Hierarchical Curriculum