Continuous Time Markov Decision Process

Continuous-time Markov decision processes (CTMDPs) model sequential decision-making in systems with continuous time and stochastic transitions, aiming to find optimal policies that maximize expected rewards. Current research emphasizes efficient exploration strategies, particularly model-free approaches, and developing stability guarantees for learning algorithms in continuous state-action spaces, often employing techniques like linear function approximation. These advancements are crucial for tackling real-world problems, such as epidemic control and autonomous navigation, where accurate modeling of continuous dynamics and efficient learning are essential. The development of rigorous theoretical bounds on learning performance, alongside practical algorithms for solving CTMDPs with complex objectives, is a major focus.

Papers

October 31, 2024

Local Linearity: the Key for No-regret Reinforcement Learning in Continuous MDPs
Davide Maran, Alberto Maria Metelli, Matteo Papini, Marcello Restelli
Regret Algorithm Linear MDPs Linear Markov Decision Process Local Linearity Continuous Time Markov Decision Process

October 22, 2024

Scalable spectral representations for network multiagent control
Zhaolin Ren, Runyu (Cathy) Zhang, Bo Dai, Na Li
Markov Decision Process Multi Agent State Action Space Spectral Representation Network Dynamic Continuous Time Markov Decision Process

October 15, 2024

Zero-shot Model-based Reinforcement Learning using Large Language Models
Abdelhakim Benechehab, Youssef Attia El Hili, Ambroise Odonnat, Oussama Zekri, Albert Thomas, Giuseppe Paolo, Maurizio Filippone, Ievgen Redko, Balázs Kégl
Reinforcement Learning Zero Shot Disentangled Learning Continuous Time Markov Decision Process

June 30, 2024

Model-Free Active Exploration in Reinforcement Learning
Alessio Russo, Alexandre Proutiere
Reinforcement Learning Ensemble Based Based Exploration Continuous Time Markov Decision Process

January 10, 2024

Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces
Yaqi Duan, Martin J. Wainwright
Reinforcement Learning Offline Reinforcement Learning Core Stability Online Reinforcement Learning State Action Space Tame Function Continuous Time Markov Decision Process

March 16, 2023

Reinforcement Learning for Omega-Regular Specifications on Continuous-Time MDP
Amin Falah, Shibashis Guha, Ashutosh Trivedi
Reinforcement Learning Markov Decision Process Continuous Time Markov Decision Process Omega Regular Continuous MDPs

October 3, 2022

Square-root regret bounds for continuous-time episodic Markov decision processes
Xuefeng Gao, Xun Yu Zhou
Reinforcement Learning Markov Decision Process Episodic Markov Decision Process Episodic Reinforcement Learning Continuous Time Markov Decision Process Continuous MDPs

August 31, 2022

Recent Advances in Modeling and Control of Epidemics using a Mean Field Approach
Amal Roy, Chandramani Singh, Y. Narahari
External Control Recent Advance Mean Field Disease Outbreak Epidemic Model Compartmental Model Continuous Time Markov Decision Process

June 3, 2022

PAC Statistical Model Checking of Mean Payoff in Discrete- and Continuous-Time MDP
Chaitanya Agarwal, Shibashis Guha, Jan Křetínský, M. Pazhamalai
Markov Decision Process Discrete Environment Payoff Function Transition Probability Continuous Time Markov Decision Process

May 23, 2022

Logarithmic regret bounds for continuous-time average-reward Markov decision processes
Xuefeng Gao, Xun Yu Zhou
Reinforcement Learning Markov Decision Process Logarithmic Regret Instance Dependent Regret Average Reward Markov Decision Process Finite Time Regret Continuous Time Markov Decision Process

December 13, 2021

Solving the non-preemptive two queue polling model with generally distributed service and switch-over durations and Poisson arrivals as a Semi-Markov Decision Process
Dylan Solms
Service Provider Poisson Process Discrete Event Continuous Time Markov Decision Process Semi Markov Decision Process Semi Markov

November 16, 2021

Kernel-based diffusion approximated Markov decision processes for autonomous navigation and control on unstructured terrains
Junhong Xu, Kai Yin, Zheng Chen, Jason M. Gregory, Ethan A. Stump, Lantao Liu
Markov Decision Process External Control Autonomous Navigation Unstructured Terrain Unstructured Off Road Continuous Time Markov Decision Process Diffusion Kernel Decision Time Planning

Continuous Time Markov Decision Process

Papers

Local Linearity: the Key for No-regret Reinforcement Learning in Continuous MDPs

Scalable spectral representations for network multiagent control

Zero-shot Model-based Reinforcement Learning using Large Language Models

Model-Free Active Exploration in Reinforcement Learning

Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces

Reinforcement Learning for Omega-Regular Specifications on Continuous-Time MDP

Square-root regret bounds for continuous-time episodic Markov decision processes

Recent Advances in Modeling and Control of Epidemics using a Mean Field Approach

PAC Statistical Model Checking of Mean Payoff in Discrete- and Continuous-Time MDP

Logarithmic regret bounds for continuous-time average-reward Markov decision processes

Solving the non-preemptive two queue polling model with generally distributed service and switch-over durations and Poisson arrivals as a Semi-Markov Decision Process

Kernel-based diffusion approximated Markov decision processes for autonomous navigation and control on unstructured terrains