Action Space

Action space, in reinforcement learning, refers to the set of all possible actions an agent can take within an environment. Current research focuses on efficiently handling large or complex action spaces, particularly in multi-agent systems and continuous control problems, employing techniques like action discretization, factorization, and the use of large language models for guidance. These advancements are crucial for scaling reinforcement learning to real-world applications, such as robotics and resource management, where high-dimensional and nuanced action choices are common. Improved methods for handling action spaces directly impact the sample efficiency and overall performance of reinforcement learning algorithms.

Papers

August 22, 2022

Efficient Planning in a Compact Latent Action Space
Zhengyao Jiang, Tianjun Zhang, Michael Janner, Yueying Li, Tim Rocktäschel, Edward Grefenstette, Yuandong Tian
Action Space Efficient Planning Latent Action Space

August 5, 2022

Planning under periodic observations: bounds and bounding-based solutions
Federico Rossi, Dylan Shell
Non Humanoid Robot Markov Decision Process Action Space Autonomous Navigation General Bound Uncertain Environment New Bounding

August 2, 2022

Implicit Two-Tower Policies
Yunfan Zhao, Qingkai Pan, Krzysztof Choromanski, Deepali Jain, Vikas Sindhwani
Action Space DeepMind Control Suite Implicit Policy

July 27, 2022

Safe and Robust Experience Sharing for Deterministic Policy Gradient Algorithms
Baturay Saglam, Dogan C. Cicek, Furkan B. Mutlu, Suleyman S. Kozat
Action Space eXperience Report Memory Replay Policy Deep Reinforcement Learning Deterministic Policy Gradient

July 23, 2022

Hierarchical Kickstarting for Skill Transfer in Reinforcement Learning
Michael Matthews, Mikayel Samvelyan, Jack Parker-Holder, Edward Grefenstette, Tim Rocktäschel
Reinforcement Learning Action Space Artificial Agent Skill Transfer

July 20, 2022

Quantifying the Effect of Feedback Frequency in Interactive Reinforcement Learning for Robotic Tasks
Daniel Harnack, Julie Pivin-Bachler, Nicolás Navarro-Guerrero
Reinforcement Learning Mixed Effect Action Space Robot Control Robotic Task Interactive Reinforcement Learning

July 14, 2022

Parallel Monte Carlo Tree Search with Batched Rigid-body Simulations for Speeding up Long-Horizon Episodic Robot Planning
Baichuan Huang, Abdeslam Boularias, Jingjin Yu
Action Space Monte Carlo Tree Search Long Horizon Episodic Markov Decision Process Episodic Reinforcement Learning

July 12, 2022

June 25, 2022

Towards Modern Card Games with Large-Scale Action Spaces Through Action Representation
Zhiyuan Yao, Tianyu Shi, Site Li, Yiting Xie, Yuanyuan Qin, Xiongjie Xie, Huan Lu, Yan Zhang
Action Space Action Representation Card Game Hybrid Reinforcement Learning

June 21, 2022

Safe and Psychologically Pleasant Traffic Signal Control with Reinforcement Learning using Action Masking
Arthur Müller, Matthia Sabatelli
Reinforcement Learning Action Space Intersection Scenario Traffic Signal Control Traffic Management Action Masking

June 10, 2022

Deep Multi-Agent Reinforcement Learning with Hybrid Action Spaces based on Maximum Entropy
Hongzhi Hua, Kaigui Wu, Guixuan Wen
Multi Agent Action Space Multi Agent Deep Reinforcement Learning Maximum Entropy Soft Actor Critic Algorithm

June 9, 2022

Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations
Cong Lu, Philip J. Ball, Tim G. J. Rudner, Jack Parker-Holder, Michael A. Osborne, Yee Whye Teh
Technical Challenge Offline Reinforcement Learning Action Space Emerging Opportunity Policy Learning Visual Exploration Vision Based Reinforcement Learning

June 1, 2022

RLSS: A Deep Reinforcement Learning Algorithm for Sequential Scene Generation
Azimkhon Ostonov, Peter Wonka, Dominik L. Michels
Action Space Proximal Policy Optimization Reinforcement Learning Algorithm Generative Approach Scene Generation

May 30, 2022

DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems
Pierre Schumacher, Daniel Häufle, Dieter Büchler, Syn Schmitt, Georg Martius
Reinforcement Learning Action Space Plasticity Rule Muscle Actuation Actuated UAV Platform Musculoskeletal System

May 24, 2022

Concurrent Credit Assignment for Data-efficient Reinforcement Learning
Emmanuel Daucé
Action Space Data Efficient Credit Assignment Policy Actor Critic Effective Reinforcement Learning Variational Optimization Occupancy Model

May 23, 2022

Learning Long-Horizon Robot Exploration Strategies for Multi-Object Search in Continuous Action Spaces
Fabian Schmalstieg, Daniel Honerkamp, Tim Welschehold, Abhinav Valada
Action Space Vision Based Navigation Multi Object Search

May 13, 2022

Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking
Hanna Krasowski, Jakob Thumm, Marlon Müller, Lukas Schäfer, Xiao Wang, Matthias Althoff
Reinforcement Learning Timely Survey Action Space Reinforcement Learning Algorithm Safe Reinforcement Learning Conceptual Analysis

April 8, 2022

Data-Driven Evaluation of Training Action Space for Reinforcement Learning
Rajat Ghosh, Debojyoti Dutta
Reinforcement Learning Action Space

April 5, 2022

GAIL-PT: A Generic Intelligent Penetration Testing Framework with Generative Adversarial Imitation Learning
Jinyin Chen, Shulong Hu, Haibin Zheng, Changyou Xing, Guomin Zhang
Action Space Penetration Testing Generative Adversarial Imitation Learning

Action Space

Papers

Efficient Planning in a Compact Latent Action Space

Planning under periodic observations: bounds and bounding-based solutions

Implicit Two-Tower Policies

Safe and Robust Experience Sharing for Deterministic Policy Gradient Algorithms

Hierarchical Kickstarting for Skill Transfer in Reinforcement Learning

Quantifying the Effect of Feedback Frequency in Interactive Reinforcement Learning for Robotic Tasks

Parallel Monte Carlo Tree Search with Batched Rigid-body Simulations for Speeding up Long-Horizon Episodic Robot Planning

Contextual Bandits with Smooth Regret: Efficient Learning in Continuous Action Spaces

Contextual Bandits with Large Action Spaces: Made Practical

Towards Modern Card Games with Large-Scale Action Spaces Through Action Representation

Safe and Psychologically Pleasant Traffic Signal Control with Reinforcement Learning using Action Masking

Deep Multi-Agent Reinforcement Learning with Hybrid Action Spaces based on Maximum Entropy

Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations

RLSS: A Deep Reinforcement Learning Algorithm for Sequential Scene Generation

DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems

Concurrent Credit Assignment for Data-efficient Reinforcement Learning

Learning Long-Horizon Robot Exploration Strategies for Multi-Object Search in Continuous Action Spaces

Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking

Data-Driven Evaluation of Training Action Space for Reinforcement Learning

GAIL-PT: A Generic Intelligent Penetration Testing Framework with Generative Adversarial Imitation Learning