Directed Exploration

Directed exploration in reinforcement learning aims to efficiently guide agents towards discovering optimal policies, especially in complex environments with sparse rewards or high-dimensional state spaces. Current research focuses on leveraging structured exploration strategies, such as incorporating intrinsic rewards based on prediction error or uncertainty quantification, and employing advanced algorithms like Wasserstein Actor-Critic or those utilizing temporal logic specifications. These advancements improve sample efficiency and enable successful learning in challenging scenarios, impacting fields like robotics, autonomous navigation, and game playing through more efficient and effective agent training.

Papers

August 18, 2024

Directed Exploration in Reinforcement Learning from Linear Temporal Logic
Marco Bagatella, Andreas Krause, Georg Martius
Reinforcement Learning Temporal Logic Directed Exploration Temporal Logic Formula Markov Reward Process

May 6, 2024

Playing Games with your PET: Extending the Partial Exploration Tool to Stochastic Games
Tobias Meggendorfer, Maximilian Weininger
Stochastic Game Directed Exploration Game Play Probabilistic System

March 12, 2024

Hallmarks of Optimization Trajectories in Neural Networks: Directional Exploration and Redundancy
Sidak Pal Singh, Bobby He, Thomas Hofmann, Bernhard Schölkopf
Neural Network Optimization Purpose Optimal Trajectory Information Redundancy Rich Attribute Directed Exploration BatchNorm Layer

July 28, 2023

Curiosity-Driven Reinforcement Learning based Low-Level Flight Control
Amir Ramezani Dooraki, Alexandros Iosifidis
Flight Control Curiosity Inducing Situation Independent Learning Directed Exploration Artificial Curiosity Curiosity Driven Reinforcement Learning

May 23, 2023

Deep Reinforcement Learning-based Multi-objective Path Planning on the Off-road Terrain Environment for Ground Vehicles
Shuqiao Huang, Xiru Wu, Guoming Huang
Path Planning Deep Reinforcement Road Environment Ground Vehicle Directed Exploration Unstructured Off Road

April 30, 2023

Learning Achievement Structure for Structured Exploration in Domains with Sparse Reward
Zihan Zhou, Animesh Garg
Reinforcement Learning Environment Exploration Sparse Reward Domain Name Directed Exploration Learning Structure Hierarchical Reward

March 9, 2023

Exploiting Contextual Structure to Generate Useful Auxiliary Tasks
Benedict Quartey, Ankit Shah, George Konidaris
Reinforcement Learning Auxiliary Task Directed Exploration Contextual Structure

March 4, 2023

Wasserstein Actor-Critic: Directed Exploration via Optimism for Continuous-Actions Control
Amarildo Likmeta, Matteo Sacco, Alberto Maria Metelli, Marcello Restelli
Reinforcement Learning Q Learning Epistemic Uncertainty Distributional Reinforcement Learning Wasserstein Dictionary Directed Exploration Continuous Action

February 5, 2023

Autonomous Exploration Method for Fast Unknown Environment Mapping by Using UAV Equipped with Limited FOV Sensor
Yinghao Zhao, Li Yan, Hong Xie, Jicheng Dai, Pengcheng Wei
Unmanned Aerial Vehicle Autonomous Exploration Directed Exploration Exploration Efficiency Autonomous Exploration Planner

January 28, 2023

STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning
Souradip Chakraborty, Amrit Singh Bedi, Alec Koppel, Mengdi Wang, Furong Huang, Dinesh Manocha
Reinforcement Learning Model Based Reinforcement Learning Steering Technique Bayesian Regret Directed Exploration Incentivized Exploration

January 6, 2023

Centralized Cooperative Exploration Policy for Continuous Control Tasks
Chao Li, Chen Gong, Qiang He, Xinwen Hou, Yu Liu
Deep Reinforcement Learning Exploration Performance Continuous Control Task Directed Exploration

November 15, 2022

Structured Exploration Through Instruction Enhancement for Object Navigation
Matthias Hutsebaut-Buysse, Kevin Mets, Tom De Schepper, Steven Latré
Task Planning Language Grounding Unseen Environment Hierarchical Learning Object Navigation Directed Exploration Instruction Augmentation

June 21, 2022

Using cognitive psychology to understand GPT-3
Marcel Binz, Eric Schulz
Multi Armed Bandit Model Based Reinforcement Learning Causal Reasoning GPT 3 Directed Exploration Data Driven Causal Cognitive Psychology

April 29, 2022

Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery
Daesol Cho, Jigang Kim, H. Jin Kim
Robot Manipulation Manipulation Skill Unsupervised Reinforcement Learning Directed Exploration

March 9, 2022

ASET: Ad-hoc Structured Exploration of Text Collections [Extended Abstract]
Benjamin Hättasch, Jan-Micha Bodensohn, Carsten Binnig
Entity Recognition Text Based Structured Data Directed Exploration Text Collection

March 2, 2022

Follow your Nose: Using General Value Functions for Directed Exploration in Reinforcement Learning
Durgesh Kalwar, Omkar Shelke, Somjit Nath, Hardik Meisheri, Harshad Khadilkar
Reinforcement Learning Value Function Sparse Reward Exploration Method Exploration Strategy Directed Exploration

Directed Exploration

Papers

Directed Exploration in Reinforcement Learning from Linear Temporal Logic

Playing Games with your PET: Extending the Partial Exploration Tool to Stochastic Games

Hallmarks of Optimization Trajectories in Neural Networks: Directional Exploration and Redundancy

Curiosity-Driven Reinforcement Learning based Low-Level Flight Control

Deep Reinforcement Learning-based Multi-objective Path Planning on the Off-road Terrain Environment for Ground Vehicles

Learning Achievement Structure for Structured Exploration in Domains with Sparse Reward

Exploiting Contextual Structure to Generate Useful Auxiliary Tasks

Wasserstein Actor-Critic: Directed Exploration via Optimism for Continuous-Actions Control

Autonomous Exploration Method for Fast Unknown Environment Mapping by Using UAV Equipped with Limited FOV Sensor

STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning

Centralized Cooperative Exploration Policy for Continuous Control Tasks

Structured Exploration Through Instruction Enhancement for Object Navigation

Using cognitive psychology to understand GPT-3

Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery

ASET: Ad-hoc Structured Exploration of Text Collections [Extended Abstract]

Follow your Nose: Using General Value Functions for Directed Exploration in Reinforcement Learning