Partially Observable Markov Decision Process

Partially Observable Markov Decision Processes (POMDPs) model sequential decision-making under uncertainty where the current state is not fully known, aiming to find optimal policies maximizing cumulative rewards. Current research emphasizes efficient algorithms for solving POMDPs, particularly those with continuous action spaces and sparse rewards, often employing techniques like Monte Carlo tree search, policy gradient methods, and neural network approximations of value functions and policies. These advancements are driving progress in diverse fields, including robotics (e.g., robust control, task planning), healthcare (e.g., optimized diagnosis and treatment), and resource management (e.g., carbon storage optimization), by enabling more effective decision-making in complex, uncertain environments.

Papers

November 11, 2024

Scaling Long-Horizon Online POMDP Planning via Rapid State Space Sampling
Yuanchu Liang, Edward Kim, Wil Thomason, Zachary Kingston, Hanna Kurniawati, Lydia E. Kavraki
Open Sampling State Space Partially Observable Markov Decision Process POMDP Planning POMDP Solver Online POMDP Solver

November 8, 2024

Enhancing Robustness in Language-Driven Robotics: A Modular Approach to Failure Reduction
Émiland Garrabé, Pierre Teixeira, Mahdi Khoramshahi, Stéphane Doncieux
Large Language Model Native Robustness Natural Language LLM Based Partially Observable Markov Decision Process Physical Robot Modular Approach

October 23, 2024

POMDP-Driven Cognitive Massive MIMO Radar: Joint Target Detection-Tracking In Unknown Disturbances
Imad Bouhou, Stefano Fortunati, Leila Gharsalli, Alexandre Renaux
Wave Over the Horizon Radar Observable Markov Decision Process Tracking by Detection Partially Observable Markov Decision Process Multiple Target Unknown Disturbance Radar Signal Cognitive Radar

October 10, 2024

Simplified POMDP Planning with an Alternative Observation Space and Formal Performance Guarantees
Da Kong, Vadim Indelman
Observable Markov Decision Process Partially Observable Markov Decision Process C POMDPs Objective Space

September 22, 2024

Adaptive Compensation for Robotic Joint Failures Using Partially Observable Reinforcement Learning
Tan-Hanh Pham, Godwyll Aikins, Tri Truong, Kim-Doang Nguyen
Reinforcement Learning Robot Person Robotic System Robotic Manipulator Partially Observable Markov Decision Process Machine Learning Based Compensation Robot Failure

September 21, 2024

R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models
Viet Dung Nguyen, Zhizhuo Yang, Christopher L. Buckley, Alexander Ororbia
Markov Decision Process World Model Tetromino Pixel Active Inference Partially Observable Markov Decision Process Sparse Reward Environment

August 11, 2024

Leveraging Knowledge Graph-Based Human-Like Memory Systems to Solve Partially Observable Markov Decision Processes
Taewoon Kim, Vincent François-Lavet, Michael Cochez
Markov Decision Process Knowledge Based Long Term Memory Belief State Partially Observable Markov Decision Process Human Like Memory Memory System

July 24, 2024

Toward an Integrated Decision Making Framework for Optimized Stroke Diagnosis with DSA and Treatment under Uncertainty
Nur Ahmad Khatim, Ahmad Azmul Asmar Irfan, Amaliya Mata'ul Hayah, Mansur M. Arief
High Uncertainty Anticipation Decision Making Observable Markov Decision Process Partially Observable Markov Decision Process Accurate Treatment Stroke Detection Stroke Patient Digital Subtraction Angiography

June 5, 2024

Sound Heuristic Search Value Iteration for Undiscounted POMDPs with Reachability Objectives
Qi Heng Ho, Martin S. Feather, Federico Rossi, Zachary N. Sunberg, Morteza Lahijanian
Markov Decision Process Sequential Decision Partially Observable Markov Decision Process Heuristic Search Reachability Objective

May 28, 2024

Recurrent Natural Policy Gradient for POMDPs
Semih Cayci, Atilla Eryilmaz
Recurrent Neural Network Partially Observable Markov Decision Process Temporal Difference Learning C POMDPs Natural Policy Gradient Non Markovian

April 5, 2024

POMDP-Guided Active Force-Based Search for Robotic Insertion
Chen Wang, Haoxiang Luo, Kun Zhang, Hua Chen, Jia Pan, Wei Zhang
Observable Markov Decision Process Proprioceptive Sensing Partially Observable Markov Decision Process Insertion Task Active Search

March 16, 2024

Inducing Individual Students' Learning Strategies through Homomorphic POMDPs
Huifan Gao, Yifeng Zeng, Yinghui Pan
Markov Decision Process Partially Observable Markov Decision Process Learning Method

February 14, 2024

Learning Interpretable Policies in Hindsight-Observable POMDPs through Partially Supervised Reinforcement Learning
Michael Lanier, Ying Xu, Nathan Jacobs, Chongjie Zhang, Yevgeniy Vorobeychik
Deep Reinforcement Learning Partially Observable Markov Decision Process Observable Environment Interpretable Policy Supervised Reinforcement Learning

January 15, 2024

Learning Explainable and Better Performing Representations of POMDP Strategies
Alexander Bork, Debraj Chakraborty, Kush Grover, Jan Kretinsky, Stefanie Mohr
Markov Decision Process Partially Observable Markov Decision Process Better Representation Product Automaton Star Algorithm Automaton Model

December 1, 2023

Safe Reinforcement Learning in Tensor Reproducing Kernel Hilbert Space
Xiaoyuan Cheng, Boli Chen, Liz Varga, Yukun Hu
Markov Decision Process Safe Reinforcement Learning State Space Reproducing Kernel Hilbert Space Partially Observable Markov Decision Process Reachability Based Safety Analysis

November 20, 2023

Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning
Hongming Zhang, Tongzheng Ren, Chenjun Xiao, Dale Schuurmans, Bo Dai
Markov Decision Process Partial Observability Partially Observable Markov Decision Process Efficient Planning Reliable Representation

October 3, 2023

Rollout Heuristics for Online Stochastic Contingent Planning
Oded Blumenthal, Guy Shani
Markov Decision Process Partially Observable Markov Decision Process C POMDPs Probabilistic Planning Model Rollouts Observable Non Deterministic Planning

July 6, 2023

Sample-Efficient Learning of POMDPs with Multiple Observations In Hindsight
Jiacheng Guo, Minshuo Chen, Huan Wang, Caiming Xiong, Mengdi Wang, Yu Bai
Reinforcement Learning Markov Decision Process Sample Efficient Partially Observable Markov Decision Process Hindsight Instruction

July 1, 2023

Provably Efficient UCB-type Algorithms For Learning Predictive State Representations
Ruiquan Huang, Yingbin Liang, Jing Yang
Markov Decision Process Sequential Decision Making Problem Partially Observable Markov Decision Process Near Optimal Policy Upper Confidence Bound Predictive State Representation

June 21, 2023

Provably Efficient Representation Learning with Tractable Planning in Low-Rank POMDP
Jiacheng Guo, Zihao Li, Huazheng Wang, Mengdi Wang, Zhuoran Yang, Xuezhou Zhang
Representation Learning Markov Decision Process Partially Observable Markov Decision Process POMDP Policy

Partially Observable Markov Decision Process

Papers

Scaling Long-Horizon Online POMDP Planning via Rapid State Space Sampling

Enhancing Robustness in Language-Driven Robotics: A Modular Approach to Failure Reduction

POMDP-Driven Cognitive Massive MIMO Radar: Joint Target Detection-Tracking In Unknown Disturbances

Simplified POMDP Planning with an Alternative Observation Space and Formal Performance Guarantees

Adaptive Compensation for Robotic Joint Failures Using Partially Observable Reinforcement Learning

R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models

Leveraging Knowledge Graph-Based Human-Like Memory Systems to Solve Partially Observable Markov Decision Processes

Toward an Integrated Decision Making Framework for Optimized Stroke Diagnosis with DSA and Treatment under Uncertainty

Sound Heuristic Search Value Iteration for Undiscounted POMDPs with Reachability Objectives

Recurrent Natural Policy Gradient for POMDPs

POMDP-Guided Active Force-Based Search for Robotic Insertion

Inducing Individual Students' Learning Strategies through Homomorphic POMDPs

Learning Interpretable Policies in Hindsight-Observable POMDPs through Partially Supervised Reinforcement Learning

Learning Explainable and Better Performing Representations of POMDP Strategies

Safe Reinforcement Learning in Tensor Reproducing Kernel Hilbert Space

Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning

Rollout Heuristics for Online Stochastic Contingent Planning

Sample-Efficient Learning of POMDPs with Multiple Observations In Hindsight

Provably Efficient UCB-type Algorithms For Learning Predictive State Representations

Provably Efficient Representation Learning with Tractable Planning in Low-Rank POMDP