Action Space

Action space, in reinforcement learning, refers to the set of all possible actions an agent can take within an environment. Current research focuses on efficiently handling large or complex action spaces, particularly in multi-agent systems and continuous control problems, employing techniques like action discretization, factorization, and the use of large language models for guidance. These advancements are crucial for scaling reinforcement learning to real-world applications, such as robotics and resource management, where high-dimensional and nuanced action choices are common. Improved methods for handling action spaces directly impact the sample efficiency and overall performance of reinforcement learning algorithms.

Papers

February 3, 2023

DiSProD: Differentiable Symbolic Propagation of Distributions for Planning
Palash Chatterjee, Ashutosh Chapagain, Weizhe Chen, Roni Khardon
Task Planning Action Space Stochastic Environment Transition Probability Prediction Boosted Planning Framework

January 31, 2023

Learning Universal Policies via Text-Guided Video Generation
Yilun Du, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Joshua B. Tenenbaum, Dale Schuurmans, Pieter Abbeel
Action Space Generalist Agent Policy Network Conditioned Policy Goal Specification Text Driven Video Generation

January 29, 2023

Distilling Internet-Scale Vision-Language Models into Embodied Agents
Theodore Sumers, Kenneth Marino, Arun Ahuja, Rob Fergus, Ishita Dasgupta
Vision Language Model Action Space Language Grounding Embodied Agent Categorical Structure Sensory Grounding

January 24, 2023

GePA*SE: Generalized Edge-Based Parallel A* for Slow Evaluations
Shohin Mukherjee, Maxim Likhachev
Robotics Domain Action Space Efficient Planning Efficient Evaluation Parallel Algorithm Edge Attribute

January 20, 2023

ApproxED: Approximate exploitability descent via learned best responses
Carlos Martin, Tuomas Sandholm
Action Space Nash Equilibrium Game Theoretic Approximate Nash Equilibrium Response Quality

January 10, 2023

Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments
Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Ritvik Singh, Yunrong Guo, Hammad Mazhar, Ajay Mandlekar, Buck Babich, Gavriel State, Marco Hutter, Animesh Garg
Imitation Learning Action Space Robot Learning Robotic Platform Simulation Framework Orbital Motion Robot Environment

January 9, 2023

On the Minimax Regret for Linear Bandits in a wide variety of Action Spaces
Debangshu Banerjee, Aditya Gopalan
Action Space Optimal Regret Linear Bandit Minimax Regret Wide Variety

December 29, 2022

December 18, 2022

Planning Immediate Landmarks of Targets for Model-Free Skill Transfer across Agents
Minghuan Liu, Zhengbang Zhu, Menghui Zhu, Yuzheng Zhuang, Weinan Zhang, Jianye Hao
Agent Smith Action Space Multiple Target Planning Model Soft Landmark

December 15, 2022

Active Inference and Reinforcement Learning: A unified inference on continuous state and action spaces under partial observability
Parvin Malekzadeh, Konstantinos N. Plataniotis
Reinforcement Learning Markov Decision Process Action Space Active Inference Partial Observability Reward Maximization Continuous POMDP Inference Scheme

December 14, 2022

Learning and Predicting Multimodal Vehicle Action Distributions in a Unified Probabilistic Model Without Labels
Charles Richter, Patrick R. Barragán, Sertac Karaman
LeArning Abstract Variational Inference Action Space Label Information Action Representation Multimodal Trajectory Prediction

December 12, 2022

Safe Reinforcement Learning with Probabilistic Guarantees Satisfying Temporal Logic Specifications in Continuous Action Spaces
Hanna Krasowski, Prithvi Akella, Aaron D. Ames, Matthias Althoff
Action Space Safe Reinforcement Learning Safety Guarantee Probabilistic Safety Vanilla Reinforcement Learning Probabilistic Temporal Logic

December 7, 2022

EASpace: Enhanced Action Space for Policy Transfer
Zheng Zhang, Qingrui Zhang, Bo Zhu, Xiaohan Wang, Tianjiang Hu
Action Space Policy Transfer Expert Policy

December 5, 2022

A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat
Jiajun Chai, Wenzhang Chen, Yuanheng Zhu, Zong-xin Yao, Dongbin Zhao
Reinforcement Learning Action Space REinforcement Learning Unmanned Combat Fictitious Self Play

December 2, 2022

Multi-Agent Reinforcement Learning with Reward Delays
Yuyang Zhang, Runyu Zhang, Yuantao Gu, Na Li
Multi Agent Reinforcement Learning Action Space Reward Delay

November 30, 2022

Policy Optimization over General State and Action Spaces
Guanghui Lan
Reinforcement Learning Action Space Policy OpTimization Internal State Policy Mirror Descent

November 29, 2022

Continuous Neural Algorithmic Planners
Yu He, Petar Veličković, Pietro Liò, Andreea Deac
Action Space Neural Algorithmic Reasoning Neural Planner Model Based Planning Graph Based Planning

November 28, 2022

Inapplicable Actions Learning for Knowledge Transfer in Reinforcement Learning
Leo Ardon, Alberto Pozanco, Daniel Borrajo, Sumitra Ganesh
Reinforcement Learning Action Space Knowledge Transfer Reinforcement Learning Algorithm Action Model Action Learning

November 23, 2022

Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning
Tingting Zhao, Ying Wang, Wei Sun, Yarui Chen, Gang Niub, Masashi Sugiyama
Reinforcement Learning Deep Reinforcement Learning Representation Learning Action Space Policy Learning Efficient Policy Learning

Action Space

Papers

DiSProD: Differentiable Symbolic Propagation of Distributions for Planning

Learning Universal Policies via Text-Guided Video Generation

Distilling Internet-Scale Vision-Language Models into Embodied Agents

GePASE: Generalized Edge-Based Parallel A for Slow Evaluations

ApproxED: Approximate exploitability descent via learned best responses

Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments

On the Minimax Regret for Linear Bandits in a wide variety of Action Spaces

On the Geometry of Reinforcement Learning in Continuous State and Action Spaces

Invariance to Quantile Selection in Distributional Continuous Control

Planning Immediate Landmarks of Targets for Model-Free Skill Transfer across Agents

Active Inference and Reinforcement Learning: A unified inference on continuous state and action spaces under partial observability

Learning and Predicting Multimodal Vehicle Action Distributions in a Unified Probabilistic Model Without Labels

Safe Reinforcement Learning with Probabilistic Guarantees Satisfying Temporal Logic Specifications in Continuous Action Spaces

EASpace: Enhanced Action Space for Policy Transfer

A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat

Multi-Agent Reinforcement Learning with Reward Delays

Policy Optimization over General State and Action Spaces

Continuous Neural Algorithmic Planners

Inapplicable Actions Learning for Knowledge Transfer in Reinforcement Learning

Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning

Action Space

Papers

DiSProD: Differentiable Symbolic Propagation of Distributions for Planning

Learning Universal Policies via Text-Guided Video Generation

Distilling Internet-Scale Vision-Language Models into Embodied Agents

GePA*SE: Generalized Edge-Based Parallel A* for Slow Evaluations

ApproxED: Approximate exploitability descent via learned best responses

Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments

On the Minimax Regret for Linear Bandits in a wide variety of Action Spaces

On the Geometry of Reinforcement Learning in Continuous State and Action Spaces

Invariance to Quantile Selection in Distributional Continuous Control

Planning Immediate Landmarks of Targets for Model-Free Skill Transfer across Agents

Active Inference and Reinforcement Learning: A unified inference on continuous state and action spaces under partial observability

Learning and Predicting Multimodal Vehicle Action Distributions in a Unified Probabilistic Model Without Labels

Safe Reinforcement Learning with Probabilistic Guarantees Satisfying Temporal Logic Specifications in Continuous Action Spaces

EASpace: Enhanced Action Space for Policy Transfer

A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat

Multi-Agent Reinforcement Learning with Reward Delays

Policy Optimization over General State and Action Spaces

Continuous Neural Algorithmic Planners

Inapplicable Actions Learning for Knowledge Transfer in Reinforcement Learning

Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning

GePASE: Generalized Edge-Based Parallel A for Slow Evaluations