Observable Markov Decision Process

Observable Markov Decision Processes (POMDPs) provide a powerful framework for modeling and solving sequential decision-making problems under uncertainty, where the complete state of the environment is not always known. Current research focuses on developing efficient algorithms to solve POMDPs, particularly for complex applications like autonomous driving, robotics, and human-robot interaction, often employing hierarchical planning, deep reinforcement learning, and Monte Carlo tree search. These advancements are significantly impacting various fields by enabling more robust and adaptable autonomous systems capable of operating effectively in uncertain and dynamic environments.

Papers

February 1, 2023

Deep reinforcement learning for the olfactory search POMDP: a quantitative benchmark
Aurore Loisy, Robin A. Heinonen
Deep Reinforcement Learning Sequential Decision Making Problem Observable Markov Decision Process Benchmark Score POMDP Solver

January 31, 2023

Learning in POMDPs is Sample-Efficient with Hindsight Observability
Jonathan N. Lee, Alekh Agarwal, Christoph Dann, Tong Zhang
LeArning Abstract Observable Markov Decision Process C POMDPs Full Observability POMDP Policy

January 22, 2023

Game Theoretic Decision Making by Actively Learning Human Intentions Applied on Autonomous Driving
Siyu Dai, Sangjae Bae, David Isele
Autonomous Driving Autonomous Vehicle Observable Markov Decision Process Human Intention Cooperative Game Theory Driving Task Autonomous Traffic

January 14, 2023

Semantic and Effective Communication for Remote Control Tasks with Dynamic Feature Compression
Pietro Talli, Francesco Pase, Federico Chiariotti, Andrea Zanella, Michele Zorzi
Deep Reinforcement Learning Mobile Robot Robot Swarm Semantic Description Observable Markov Decision Process Feature Compression Vector Quantized Variational Effective Communication

January 2, 2023

On the Challenges of using Reinforcement Learning in Precision Drug Dosing: Delay and Prolongedness of Action Effects
Sumana Basu, Marc-André Legault, Adriana Romero-Soriano, Doina Precup
Reinforcement Learning Reward Function Observable Markov Decision Process Long Span C POMDPs Account Time Delay Dosing Model

December 17, 2022

Comparison of Model-Free and Model-Based Learning-Informed Planning for PointGoal Navigation
Yimeng Li, Arnab Debnath, Gregory J. Stein, Jana Kosecka
Consistent Comparison Learning Based Model Free Navigation Task Observable Markov Decision Process Reinforcement Learning Approach

November 15, 2022

Reinforcement Learning Methods for Wordle: A POMDP/Adaptive Control Approach
Siddhant Bhambri, Amrita Bhattacharjee, Dimitri Bertsekas
Reinforcement Learning Sequential Decision Making Problem Observable Markov Decision Process Adaptive Control Partially Observable Markov Decision Process Optimal Strategy Wordle Game

November 3, 2022

GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond
Han Zhong, Wei Xiong, Sirui Zheng, Liwei Wang, Zhaoran Wang, Zhuoran Yang, Tong Zhang
Markov Decision Process Unified Framework Efficient Learning Efficient Reinforcement Learning Observable Markov Decision Process Partially Observable Markov Decision Process Interactive Decision Making

October 25, 2022

A POMDP Model for Safe Geological Carbon Sequestration
Anthony Corso, Yizheng Wang, Markus Zechner, Jef Caers, Mykel J. Kochenderfer
Observable Markov Decision Process Partially Observable Markov Decision Process Geological Carbon Storage Carbon Sequestration

October 2, 2022

Safe Reinforcement Learning From Pixels Using a Stochastic Latent Representation
Yannick Hogewind, Thiago D. Simao, Tal Kachman, Nils Jansen
Safe Reinforcement Learning Tetromino Pixel Observable Markov Decision Process Partial Observability Reward Maximization Safety Critic

September 21, 2022

Partially Observable Markov Decision Processes in Robotics: A Survey
Mikko Lauri, David Hsu, Joni Pajarinen
Timely Survey Markov Decision Process Robotics Domain Observable Markov Decision Process Partially Observable Markov Decision Process C POMDPs

September 20, 2022

Macro-Action-Based Multi-Agent/Robot Deep Reinforcement Learning under Partial Observability
Yuchen Xiao
Multi Agent Reinforcement Learning Policy Gradient Observable Markov Decision Process Partial Observability C POMDPs

September 2, 2022

Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning
Pablo G. Morato, Charalampos P. Andriotis, Konstantinos G. Papakonstantinou, Philippe Rigo
Deep Reinforcement Learning Scientific Inference Policy OpTimization Bayesian Network Global Optimization Observable Markov Decision Process Reinforcement Learning Approach Dynamic Decision Probabilistic Dependency Optimal Execution

August 31, 2022

Partial Counterfactual Identification for Infinite Horizon Partially Observable Markov Decision Process
Aditya Kelvianto Sidharta
Q Learning Observable Markov Decision Process Infinite Horizon Causal Query Counterfactual Query

August 4, 2022

AG2U -- Autonomous Grading Under Uncertainties
Yakov Miron, Yuval Goldfracht, Chana Ross, Dotan Di Castro, Itzik Klein
Robust Policy Observable Markov Decision Process High Uncertainty Value Automatic Grading Localization Uncertainty Surface Roughness

July 24, 2022

Towards Using Fully Observable Policies for POMDPs
András Attila Sulyok, Kristóf Karacs
Observable Markov Decision Process Partially Observable Markov Decision Process Based Policy C POMDPs Greedy Policy

July 11, 2022

Uncertainty-Aware Online Merge Planning with Learned Driver Behavior
Liam A. Kruse, Esen Yel, Ransalu Senanayake, Mykel J. Kochenderfer
Autonomous Vehicle Particle Filter Latent State Observable Markov Decision Process Online Merging Smart Transportation

June 23, 2022

June 20, 2022

Intention-Aware Navigation in Crowds with Extended-Space POMDP Planning
Himanshu Gupta, Bradley Hayes, Zachary Sunberg
Crowded Environment Observable Markov Decision Process Partially Observable Markov Decision Process Intention Aware

Observable Markov Decision Process

Papers

Deep reinforcement learning for the olfactory search POMDP: a quantitative benchmark

Learning in POMDPs is Sample-Efficient with Hindsight Observability

Game Theoretic Decision Making by Actively Learning Human Intentions Applied on Autonomous Driving

Semantic and Effective Communication for Remote Control Tasks with Dynamic Feature Compression

On the Challenges of using Reinforcement Learning in Precision Drug Dosing: Delay and Prolongedness of Action Effects

Comparison of Model-Free and Model-Based Learning-Informed Planning for PointGoal Navigation

Reinforcement Learning Methods for Wordle: A POMDP/Adaptive Control Approach

GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond

A POMDP Model for Safe Geological Carbon Sequestration

Safe Reinforcement Learning From Pixels Using a Stochastic Latent Representation

Partially Observable Markov Decision Processes in Robotics: A Survey

Macro-Action-Based Multi-Agent/Robot Deep Reinforcement Learning under Partial Observability

Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning

Partial Counterfactual Identification for Infinite Horizon Partially Observable Markov Decision Process

AG2U -- Autonomous Grading Under Uncertainties

Towards Using Fully Observable Policies for POMDPs

Uncertainty-Aware Online Merge Planning with Learned Driver Behavior

Formalizing the Problem of Side Effect Regularization

Reinforcement Learning under Partial Observability Guided by Learned Environment Models

Intention-Aware Navigation in Crowds with Extended-Space POMDP Planning