Continuous MDPs

Continuous Markov Decision Processes (CMDPs) model sequential decision-making in systems with continuous state and/or action spaces, aiming to find optimal policies maximizing cumulative rewards. Current research focuses on developing efficient algorithms, such as adaptive discretization methods and constraint-generation approaches, to handle the complexities of continuous spaces and diverse reward structures, including those defined by logical specifications. These advancements are improving the applicability of reinforcement learning to real-world problems with continuous dynamics, offering solutions with bounded performance guarantees and enhanced explainability, particularly in domains like robotics and resource management.

Papers

May 29, 2024

Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning
Avik Kar, Rahul Singh
Optimal Policy Finite Horizon State Action Space Episodic Reinforcement Learning Metric Space Continuous MDPs

January 20, 2024

Constraint-Generation Policy Optimization (CGPO): Nonlinear Programming for Policy Optimization in Mixed Discrete-Continuous MDPs
Michael Gimelfarb, Ayal Taitler, Scott Sanner
Optimal Policy Policy OpTimization Nonlinear Programming Constrained Policy Optimization Continuous MDPs

March 16, 2023

Reinforcement Learning for Omega-Regular Specifications on Continuous-Time MDP
Amin Falah, Shibashis Guha, Ashutosh Trivedi
Reinforcement Learning Markov Decision Process Continuous Time Markov Decision Process Omega Regular Continuous MDPs

October 3, 2022

Square-root regret bounds for continuous-time episodic Markov decision processes
Xuefeng Gao, Xun Yu Zhou
Reinforcement Learning Markov Decision Process Episodic Markov Decision Process Episodic Reinforcement Learning Continuous Time Markov Decision Process Continuous MDPs

March 23, 2022

Sample-efficient Iterative Lower Bound Optimization of Deep Reactive Policies for Planning in Continuous MDPs
Siow Meng Low, Akshat Kumar, Scott Sanner
Task Planning DRL Algorithm Continuous MDPs

November 12, 2021

Q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuity
Ali Devran Kara, Naci Saldi, Serdar Yüksel
Markov Decision Process Q Learning Quantization Operator State Space Near Optimality Continuous MDPs

Continuous MDPs

Papers

Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning

Constraint-Generation Policy Optimization (CGPO): Nonlinear Programming for Policy Optimization in Mixed Discrete-Continuous MDPs

Reinforcement Learning for Omega-Regular Specifications on Continuous-Time MDP

Square-root regret bounds for continuous-time episodic Markov decision processes

Sample-efficient Iterative Lower Bound Optimization of Deep Reactive Policies for Planning in Continuous MDPs

Q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuity