Action Space

Action space, in reinforcement learning, refers to the set of all possible actions an agent can take within an environment. Current research focuses on efficiently handling large or complex action spaces, particularly in multi-agent systems and continuous control problems, employing techniques like action discretization, factorization, and the use of large language models for guidance. These advancements are crucial for scaling reinforcement learning to real-world applications, such as robotics and resource management, where high-dimensional and nuanced action choices are common. Improved methods for handling action spaces directly impact the sample efficiency and overall performance of reinforcement learning algorithms.

Papers

November 21, 2023

Scheduling Distributed Flexible Assembly Lines using Safe Reinforcement Learning with Soft Shielding
Lele Li, Liyong Lin
Reinforcement Learning Action Space Safe Reinforcement Learning Task Scheduling Scheduling Problem Assembly Line Novel Shielding Mechanism

November 20, 2023

Approximate Linear Programming for Decentralized Policy Iteration in Cooperative Multi-agent Markov Decision Processes
Lakshmi Mandal, Chandrashekar Lakshminarayanan, Shalabh Bhatnagar
Multi Agent Action Space Approximate Optimization Decentralized Policy

November 6, 2023

Low-Rank MDPs with Continuous Action Spaces
Andrew Bennett, Nathan Kallus, Miruna Oprescu
Reinforcement Learning Markov Decision Process Action Space Low Rank

November 2, 2023

C3DM: Constrained-Context Conditional Diffusion Models for Imitation Learning
Vaibhav Saxena, Yotto Koga, Danfei Xu
Diffusion Model Imitation Learning Action Space Action Representation Offline Imitation Learning

October 30, 2023

From External to Swap Regret 2.0: An Efficient Reduction and Oblivious Adversary for Large Action Spaces
Yuval Dagan, Constantinos Daskalakis, Maxwell Fishelson, Noah Golowich
Action Space Regret Minimization Constructive Reduction Regret Algorithm Oblivious Adversary Swap Regret

October 24, 2023

Off-Policy Evaluation for Large Action Spaces via Policy Convolution
Noveen Sachdeva, Lequn Wang, Dawen Liang, Nathan Kallus, Julian McAuley
Action Space Policy Evaluation Importance Sampling Policy Estimation Action Embeddings Policy Blending

October 23, 2023

Reinforcement learning in large, structured action spaces: A simulation study of decision support for spinal cord injury rehabilitation
Nathan Phelps, Stephanie Marrocco, Stephanie Cornell, Dalton L. Wolfe, Daniel J. Lizotte
Reinforcement Learning Simulation Study Action Space Physical Rehabilitation Decision Support Optimal Treatment Spinal Cord Injury

October 20, 2023

October 18, 2023

LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following
Cheng-Fu Yang, Yen-Chun Chen, Jianwei Yang, Xiyang Dai, Lu Yuan, Yu-Chiang Frank Wang, Kai-Wei Chang
Contrastive Learning Action Space Natural Language Instruction Instruction Following Embodied Instruction

October 8, 2023

The Reinforce Policy Gradient Algorithm Revisited
Shalabh Bhatnagar
Policy Gradient Action Space Free Task

October 4, 2023

Multi-Agent Reinforcement Learning for Power Grid Topology Optimization
Erica van der Sar, Alessandro Zocca, Sandjai Bhulai
Reinforcement Learning Multi Agent Reinforcement Learning Action Space Power Grid Single Agent Reinforcement Learning Topology Control

October 1, 2023

Skip-Plan: Procedure Planning in Instructional Videos via Condensed Action Space Learning
Zhiheng Li, Wenjia Geng, Muheng Li, Lei Chen, Yansong Tang, Jiwen Lu, Jie Zhou
Action Space Action Prediction Action Sequence Instructional Video State Action Pair Skip and Recover Procedure Planning

September 24, 2023

RL-I2IT: Image-to-Image Translation with Deep Reinforcement Learning
Xin Wang, Ziwei Luo, Jing Hu, Chengming Feng, Shu Hu, Bin Zhu, Xi Wu, Hongtu Zhu, Xin Li, Siwei Lyu
Deep Reinforcement Learning Action Space Image to Image Translation Actor Critic

September 18, 2023

Learning Optimal Contracts: How to Exploit Small Action Spaces
Francesco Bacchiocchi, Matteo Castiglioni, Alberto Marchesi, Nicola Gatti
Action Space Principal Agent Optimal Contract Adaptive Incentive

September 9, 2023

Verifiable Reinforcement Learning Systems via Compositionality
Cyrus Neary, Aryaman Singh Samyal, Christos Verginis, Murat Cubuktepe, Ufuk Topcu
Action Space Partial Observability Compositional Language Deep Reinforcement Learning Agent Compositional Reinforcement Learning

September 8, 2023

Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning
David Yunis, Justin Jung, Falcon Dai, Matthew Walter
Reinforcement Learning Action Space Sparse Reward Skill Learning Sub Word Robust Skill

September 6, 2023

August 31, 2023

Multi-Objective Decision Transformers for Offline Reinforcement Learning
Abdelghani Ghanem, Philippe Ciblat, Mounir Ghogho
Offline Reinforcement Learning Action Space Decision Transformer Trajectory Data Trajectory Representation

Action Space

Papers

Scheduling Distributed Flexible Assembly Lines using Safe Reinforcement Learning with Soft Shielding

Approximate Linear Programming for Decentralized Policy Iteration in Cooperative Multi-agent Markov Decision Processes

Low-Rank MDPs with Continuous Action Spaces

C3DM: Constrained-Context Conditional Diffusion Models for Imitation Learning

From External to Swap Regret 2.0: An Efficient Reduction and Oblivious Adversary for Large Action Spaces

Off-Policy Evaluation for Large Action Spaces via Policy Convolution

Reinforcement learning in large, structured action spaces: A simulation study of decision support for spinal cord injury rehabilitation

PathRL: An End-to-End Path Generation Method for Collision Avoidance via Deep Reinforcement Learning

ToolChain: Efficient Action Space Navigation in Large Language Models with A Search

LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following

The Reinforce Policy Gradient Algorithm Revisited

Multi-Agent Reinforcement Learning for Power Grid Topology Optimization

Skip-Plan: Procedure Planning in Instructional Videos via Condensed Action Space Learning

RL-I2IT: Image-to-Image Translation with Deep Reinforcement Learning

Learning Optimal Contracts: How to Exploit Small Action Spaces

Verifiable Reinforcement Learning Systems via Compositionality

Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning

Near-continuous time Reinforcement Learning for continuous state-action spaces

Pre- and post-contact policy decomposition for non-prehensile manipulation with zero-shot sim-to-real transfer

Multi-Objective Decision Transformers for Offline Reinforcement Learning

Action Space

Papers

Scheduling Distributed Flexible Assembly Lines using Safe Reinforcement Learning with Soft Shielding

Approximate Linear Programming for Decentralized Policy Iteration in Cooperative Multi-agent Markov Decision Processes

Low-Rank MDPs with Continuous Action Spaces

C3DM: Constrained-Context Conditional Diffusion Models for Imitation Learning

From External to Swap Regret 2.0: An Efficient Reduction and Oblivious Adversary for Large Action Spaces

Off-Policy Evaluation for Large Action Spaces via Policy Convolution

Reinforcement learning in large, structured action spaces: A simulation study of decision support for spinal cord injury rehabilitation

PathRL: An End-to-End Path Generation Method for Collision Avoidance via Deep Reinforcement Learning

ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search

LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following

The Reinforce Policy Gradient Algorithm Revisited

Multi-Agent Reinforcement Learning for Power Grid Topology Optimization

Skip-Plan: Procedure Planning in Instructional Videos via Condensed Action Space Learning

RL-I2IT: Image-to-Image Translation with Deep Reinforcement Learning

Learning Optimal Contracts: How to Exploit Small Action Spaces

Verifiable Reinforcement Learning Systems via Compositionality

Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning

Near-continuous time Reinforcement Learning for continuous state-action spaces

Pre- and post-contact policy decomposition for non-prehensile manipulation with zero-shot sim-to-real transfer

Multi-Objective Decision Transformers for Offline Reinforcement Learning

ToolChain: Efficient Action Space Navigation in Large Language Models with A Search