State Action Space

State-action space, in reinforcement learning, refers to the combined set of all possible states an agent can be in and the actions it can take. Research focuses on efficiently exploring and representing this space, particularly in continuous domains, using techniques like Thompson sampling for optimistic exploration, adaptive discretization, and low-rank tensor approximations for value function estimation. These advancements aim to improve the sample efficiency and robustness of reinforcement learning algorithms, impacting fields like robotics, multi-agent systems, and healthcare where efficient learning from limited data is crucial.

Papers

October 25, 2024

Provably Adaptive Average Reward Reinforcement Learning for Metric Spaces
Avik Kar, Rahul Singh
State Space Finite Horizon North Star Metric State Action Space Mixed Space Average Reward Reinforcement Learning

October 22, 2024

Scalable spectral representations for multi-agent reinforcement learning in network MDPs
Zhaolin Ren, Runyu Zhang, Bo Dai, Na Li
Markov Decision Process Multi Agent State Action Space Spectral Representation Network Dynamic Continuous Time Markov Decision Process

October 7, 2024

Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling
Jasmine Bayrooti, Carl Henrik Ek, Amanda Prorok
High Efficiency Model Based Reinforcement Learning Sparse Reward Thompson Sampling State Action Space Sample Efficient Reinforcement Learning Optimistic Exploration

September 22, 2024

A Comparative Study on State-Action Spaces for Learning Viewpoint Selection and Manipulation with Diffusion Policy
Xiatao Sun, Francis Fan, Yinxing Chen, Daniel Rakita
Comparative Study Manipulation Task Strategic Manipulation State Action Space Visuomotor Policy Policy Diffusion

August 9, 2024

Performance Prediction of Hub-Based Swarms
Puneet Jain, Chaitanya Dwivedi, Vigynesh Bhatt, Nick Smith, Michael A Goodrich
State Action Space Performance Prediction Hub Structure Collective Navigation Swarm Feature

June 22, 2024

Learning Abstract World Model for Value-preserving Planning with Options
Rafael Rodriguez-Sanchez, George Konidaris
Task Planning Model Learning State Action Space Continuous Option Temporal Action MDP Model

June 3, 2024

How to discretize continuous state-action spaces in Q-learning: A symbolic control approach
Sadek Belamfedel Alaoui, Adnane Saoud
Q Learning State Action Space Controller Synthesis Symbolic Model

May 29, 2024

Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning
Avik Kar, Rahul Singh
Optimal Policy Finite Horizon State Action Space Episodic Reinforcement Learning Metric Space Continuous MDPs

May 27, 2024

May 3, 2024

Closing the Gap: Achieving Global Convergence (Last Iterate) of Actor-Critic under Markovian Sampling with Neural Network Parametrization
Mudit Gaur, Amrit Singh Bedi, Di Wang, Vaneet Aggarwal
Neural Network Global Convergence State Action Space Sample Complexity Bound Critic Model Markovian Sampling Local Optimality Gradient Dominance Property

May 1, 2024

MESA: Cooperative Meta-Exploration in Multi-Agent Learning through Exploiting State-Action Space Structure
Zhicheng Zhang, Yancheng Liang, Yi Wu, Fei Fang
Multi Agent Reinforcement Learning Efficient Exploration Multi Agent Learning State Action Space Exploration Policy Multi Agent Particle Cooperative Multi Agent Learning Mesa Optimization

March 12, 2024

Online Adaptation of Sampling-Based Motion Planning with Inaccurate Models
Marco Faroni, Dmitry Berenson
Robotic Manipulation Robot Planning State Action Space Online Adaptation Sampling Based Motion Planning Kinodynamic Motion Low Quality Model

January 27, 2024

Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning
Chenyu Zhang, Han Wang, Aritra Mitra, James Anderson
Multi Agent Collaboration Federated Reinforcement Learning Finite Time State Action Space Reinforcement Learning Task

January 10, 2024

Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces
Yaqi Duan, Martin J. Wainwright
Reinforcement Learning Offline Reinforcement Learning Core Stability Online Reinforcement Learning State Action Space Tame Function Continuous Time Markov Decision Process

December 7, 2023

CODEX: A Cluster-Based Method for Explainable Reinforcement Learning
Timothy K. Mathes, Jessica Inman, Andrés Colón, Simon Khan
Reinforcement Learning Counterfactual Explanation Reinforcement Learning Agent Agent Behavior State Action Space Explainable Reinforcement Learning Clustering Based Approach Game Representation

October 15, 2023

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces
Ammar N. Abbas, Georgios C. Chasparis, John D. Kelleher
Deep Reinforcement Learning Safe Reinforcement Learning Complexity Level State Action Space Residual Reinforcement Learning M$ Base Controller

October 3, 2023

Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learning
Finn Rietz, Erik Schaffernicht, Stefan Heinrich, Johannes Andreas Stork
Reinforcement Learning State Action Space Scalar Reward

September 6, 2023

Near-continuous time Reinforcement Learning for continuous state-action spaces
Lorenzo Croissant, Marc Abeille, Bruno Bouchard
Reinforcement Learning Reward Function Action Space State Action Space Reinforcement Learning Problem Linear Time Structured State Evolution

June 26, 2023

Safe Navigation in Unstructured Environments by Minimizing Uncertainty in Control and Perception
Junwon Seo, Jungwi Mun, Taekyung Kim
External Control High Uncertainty Anticipation Perception Aware State Action Space Unstructured Environment Safe Navigation Autonomous Navigation System Vehicle Dynamic Traversability Prediction Uncertainty Aware Model

State Action Space

Papers

Provably Adaptive Average Reward Reinforcement Learning for Metric Spaces

Scalable spectral representations for multi-agent reinforcement learning in network MDPs

Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling

A Comparative Study on State-Action Spaces for Learning Viewpoint Selection and Manipulation with Diffusion Policy

Performance Prediction of Hub-Based Swarms

Learning Abstract World Model for Value-preserving Planning with Options

How to discretize continuous state-action spaces in Q-learning: A symbolic control approach

Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning

Tensor Low-rank Approximation of Finite-horizon Value Functions

Symmetric Reinforcement Learning Loss for Robust Learning on Diverse Tasks and Model Scales

Closing the Gap: Achieving Global Convergence (Last Iterate) of Actor-Critic under Markovian Sampling with Neural Network Parametrization

MESA: Cooperative Meta-Exploration in Multi-Agent Learning through Exploiting State-Action Space Structure

Online Adaptation of Sampling-Based Motion Planning with Inaccurate Models

Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning

Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces

CODEX: A Cluster-Based Method for Explainable Reinforcement Learning

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces

Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learning

Near-continuous time Reinforcement Learning for continuous state-action spaces

Safe Navigation in Unstructured Environments by Minimizing Uncertainty in Control and Perception