Action Space

Action space, in reinforcement learning, refers to the set of all possible actions an agent can take within an environment. Current research focuses on efficiently handling large or complex action spaces, particularly in multi-agent systems and continuous control problems, employing techniques like action discretization, factorization, and the use of large language models for guidance. These advancements are crucial for scaling reinforcement learning to real-world applications, such as robotics and resource management, where high-dimensional and nuanced action choices are common. Improved methods for handling action spaces directly impact the sample efficiency and overall performance of reinforcement learning algorithms.

Papers

January 10, 2025

Investigating the Impact of Observation Space Design Choices On Training Reinforcement Learning Solutions for Spacecraft Problems
Nathaniel Hamilton, Kyle Dunlap, Kerianne L Hobbs
Reinforcement Learning Training Data Global Impact Action Space Optimal Behavior Autonomous Control Unmanned Spacecraft Objective Space Reinforcement Learning Solution

January 7, 2025

Hybrid Machine Learning Model with a Constrained Action Space for Trajectory Prediction
Alexander Fertig, Lakshman Balasubramanian, Michael Botsch
Deep Learning Autonomous Driving Trajectory Prediction Action Space Driving Scenario Hybrid Machine Learning Hybrid Method

December 28, 2024

Investigating the Impact of Communication-Induced Action Space on Exploration of Unknown Environments with Decentralized Multi-Agent Reinforcement Learning
Gabriele Calzolari (1), Vidya Sumathy (1), Christoforos Kanellakis (1), George Nikolakopoulos (1) ((1) Luleå University of Technology)
Reinforcement Learning Global Impact Environment Exploration Action Space Efficient Exploration Unknown Environment Policy Optimization Agent Communication Decentralized Multi Agent

December 19, 2024

Cutting Sequence Diffuser: Sim-to-Real Transferable Planning for Object Shaping by Grinding
Takumi Hachimine, Jun Morimoto, Takamitsu Matsubara
Robot Person Arbitrary Object Action Space Action Sequence Shape Change Real World Planning

December 18, 2024

Tree-of-Code: A Hybrid Approach for Robust Complex Task Planning and Execution
Ziyi Ni, Yifan Li, Daxiang Dong
Action Space AI Agent LLM Based Agent Hybrid Approach Tree of Thought Robust Planning Function Decomposition Tree

December 17, 2024

An Advantage-based Optimization Method for Reinforcement Learning in Large Action Space
Hai Lin, Cheng Huang, Zhihong Chen
Reinforcement Learning Action Space Reinforcement Learning Benchmark Reinforcement Learning Task Value Based Reinforcement Learning

December 9, 2024

Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi
F. Bredell, H.A. Engelbrecht, J.C. Schoeman
Multi Agent Reinforcement Learning Action Space Long Term Cooperation Multiple Agent Academic Conference Multi Agent Problem Game Hanabi

December 5, 2024

Action Mapping for Reinforcement Learning in Continuous Environments with Constraints
Mirco Theile, Lukas Dirnberger, Raphael Trumpp, Marco Caccamo, Alberto L. Sangiovanni-Vincentelli
Reinforcement Learning Deep Reinforcement Learning Action Space Participation Constraint Continuous Environment Feasible Action Action Mapping

December 1, 2024

Decision Transformer vs. Decision Mamba: Analysing the Complexity of Sequential Decision Making in Atari Games
Ke Yan
Reinforcement Learning Action Space Complexity Matter Mamba in Mamba Decision Transformer Sequential Decision Making Reinforcement Learning Model ATARI Game Game Model

November 22, 2024

Enhancing Exploration with Diffusion Policies in Hybrid Off-Policy RL: Application to Non-Prehensile Manipulation
Huy Le, Miroslav Gabriel, Tai Hoang, Gerhard Neumann, Ngo Anh Vien
Application Proficiency Action Space Policy Reinforcement Learning Diffusion Policy Non Prehensile Manipulation Exploration Performance Maximum Entropy Reinforcement Learning Continuous Control Policy

November 21, 2024

Rethinking the Intermediate Features in Adversarial Attacks: Misleading Robotic Models via Adversarial Distillation
Ke Zhao (1), Huayang Huang (1), Miao Li (1), Yu Wu (1) ((1) Wuhan University)
Adversarial Attack Action Space Adversarial Prompt Language Conditioned Robot Model Adversarial Distillation

November 17, 2024

An Investigation of Offline Reinforcement Learning in Factorisable Action Spaces
Alex Beeson, David Ireland, Giovanni Montana
Offline Reinforcement Learning Action Space Comprehensive Investigation

November 4, 2024

Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks
Pranav Guruprasad, Harshvardhan Sikka, Jaewoo Song, Yangyue Wang, Paul Pu Liang
Human Language Action Space Robot Learning Action Model Vision Language Action Action Generation General Purpose Vision Benchmark Robot Task

October 28, 2024

October 25, 2024

October 22, 2024

Composing Diffusion Policies for Few-shot Learning of Movement Trajectories
Omkar Patil, Anant Sah, Nakul Gopalan
Shot Learning Action Space Handwritten Trajectory Movement Related Information Policy Diffusion

October 21, 2024

Diffusion Transformer Policy
Zhi Hou, Tianyi Zhang, Yuwen Xiong, Hengjun Pu, Chengyang Zhao, Ronglei Tong, Yu Qiao, Jifeng Dai, Yuntao Chen
Action Space Diffusion Transformer Robotic Datasets Vision Language Action

October 18, 2024

Online Reinforcement Learning with Passive Memory
Anay Pattanaik, Lav R. Varshney
Action Space Online Reinforcement Learning Theoretical Guarantee

Action Space

Papers

Investigating the Impact of Observation Space Design Choices On Training Reinforcement Learning Solutions for Spacecraft Problems

Hybrid Machine Learning Model with a Constrained Action Space for Trajectory Prediction

Investigating the Impact of Communication-Induced Action Space on Exploration of Unknown Environments with Decentralized Multi-Agent Reinforcement Learning

Cutting Sequence Diffuser: Sim-to-Real Transferable Planning for Object Shaping by Grinding

Tree-of-Code: A Hybrid Approach for Robust Complex Task Planning and Execution

An Advantage-based Optimization Method for Reinforcement Learning in Large Action Space

Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi

Action Mapping for Reinforcement Learning in Continuous Environments with Constraints

Decision Transformer vs. Decision Mamba: Analysing the Complexity of Sequential Decision Making in Atari Games

Enhancing Exploration with Diffusion Policies in Hybrid Off-Policy RL: Application to Non-Prehensile Manipulation

Rethinking the Intermediate Features in Adversarial Attacks: Misleading Robotic Models via Adversarial Distillation

An Investigation of Offline Reinforcement Learning in Factorisable Action Spaces

Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks

Learning State Conditioned Linear Mappings for Low-Dimensional Control of Robotic Manipulators

Offline Reinforcement Learning With Combinatorial Action Spaces

Random Policy Enables In-Context Reinforcement Learning within Trust Horizons

An Enhanced Hierarchical Planning Framework for Multi-Robot Autonomous Exploration

Composing Diffusion Policies for Few-shot Learning of Movement Trajectories

Diffusion Transformer Policy

Online Reinforcement Learning with Passive Memory