Observable Markov Decision Process

Observable Markov Decision Processes (POMDPs) provide a powerful framework for modeling and solving sequential decision-making problems under uncertainty, where the complete state of the environment is not always known. Current research focuses on developing efficient algorithms to solve POMDPs, particularly for complex applications like autonomous driving, robotics, and human-robot interaction, often employing hierarchical planning, deep reinforcement learning, and Monte Carlo tree search. These advancements are significantly impacting various fields by enabling more robust and adaptable autonomous systems capable of operating effectively in uncertain and dynamic environments.

Papers

November 11, 2024

OCMDP: Observation-Constrained Markov Decision Process
Taiyi Wang, Jianheng Liu, Jiaye Li, Zhihao Wu, Yu Wu
Observable Markov Decision Process Model Free Deep Reinforcement Learning Full Observability

November 5, 2024

When to Localize? A Risk-Constrained Reinforcement Learning Approach
Chak Lam Shek, Kasra Torshizi, Troi Williams, Pratap Tokekar
Reinforcement Learning Observable Markov Decision Process Underwater Robot Risk Aware Reinforcement Learning Active Localization

October 23, 2024

POMDP-Driven Cognitive Massive MIMO Radar: Joint Target Detection-Tracking In Unknown Disturbances
Imad Bouhou, Stefano Fortunati, Leila Gharsalli, Alexandre Renaux
Wave Over the Horizon Radar Observable Markov Decision Process Tracking by Detection Partially Observable Markov Decision Process Multiple Target Unknown Disturbance Radar Signal Cognitive Radar

October 10, 2024

Simplified POMDP Planning with an Alternative Observation Space and Formal Performance Guarantees
Da Kong, Vadim Indelman
Observable Markov Decision Process Partially Observable Markov Decision Process C POMDPs Objective Space

September 27, 2024

September 24, 2024

Dynamic Game-Theoretical Decision-Making Framework for Vehicle-Pedestrian Interaction with Human Bounded Rationality
Meiting Dang, Dezong Zhao, Yafei Wang, Chongfeng Wei
Autonomous Vehicle Observable Markov Decision Process Pedestrian Vehicle Interaction Bounded Rational Behavioral Game

September 3, 2024

Situation-aware Autonomous Driving Decision Making with Cooperative Perception on Demand
Wei Liu
Autonomous Driving Decision Making Long Range Observable Markov Decision Process Cooperative Perception Video Game Demand Vehicle Decision Making

July 24, 2024

Toward an Integrated Decision Making Framework for Optimized Stroke Diagnosis with DSA and Treatment under Uncertainty
Nur Ahmad Khatim, Ahmad Azmul Asmar Irfan, Amaliya Mata'ul Hayah, Mansur M. Arief
High Uncertainty Anticipation Decision Making Observable Markov Decision Process Partially Observable Markov Decision Process Accurate Treatment Stroke Detection Stroke Patient Digital Subtraction Angiography

May 30, 2024

From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems
Jianliang He, Siyu Chen, Fengzhuo Zhang, Zhuoran Yang
Word List Hierarchical Reinforcement Learning Sublinear Regret Observable Markov Decision Process Past Action Theoretical Foundation Greedy Exploration

May 1, 2024

ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraints
Robert J. Moss, Arec Jamgochian, Johannes Fischer, Anthony Corso, Mykel J. Kochenderfer
Observable Markov Decision Process Policy Iteration C POMDPs Probabilistic Surrogate Unknown Constraint POMDP Planning

April 5, 2024

POMDP-Guided Active Force-Based Search for Robotic Insertion
Chen Wang, Haoxiang Luo, Kun Zhang, Hua Chen, Jia Pan, Wei Zhang
Observable Markov Decision Process Proprioceptive Sensing Partially Observable Markov Decision Process Insertion Task Active Search

April 2, 2024

Distributed Autonomous Swarm Formation for Dynamic Network Bridging
Raffaele Galliera, Thies Möhlenhof, Alessandro Amato, Daniel Duran, Kristen Brent Venable, Niranjan Suri
Multi Agent Reinforcement Learning Dynamic Network Observable Markov Decision Process Graph Reinforcement Learning Autonomous Robotic Swarm Swarm Confrontation

March 15, 2024

Belief Aided Navigation using Bayesian Reinforcement Learning for Avoiding Humans in Blind Spots
Jinyeob Kim, Daewon Kwak, Hyunwoo Rim, Donghan Kim
Reinforcement Learning Autonomous Vehicle Human Robot Interaction Real Human Robot Navigation Observable Markov Decision Process Blind Spot Navigation Strategy

February 28, 2024

Provably Efficient Partially Observable Risk-Sensitive Reinforcement Learning with Hindsight Observation
Tonghe Zhang, Yu Chen, Longbo Huang
Reinforcement Learning High Efficiency Efficient Reinforcement Learning Observable Markov Decision Process Risk Sensitive Reinforcement Learning Hindsight Instruction

February 8, 2024

Offline Risk-sensitive RL with Partial Observability to Enhance Performance in Human-Robot Teaming
Giorgio Angelotti, Caroline P. C. Chanel, Adam H. M. Pinto, Christophe Lounis, Corentin Chauffaut, Nicolas Drougard
System Performance Physiological Signal Observable Markov Decision Process Partial Observability Human Robot Teaming Human Operator Offline Safe Reinforcement Learning Human State Estimation

February 5, 2024

Solving Hierarchical Information-Sharing Dec-POMDPs: An Extensive-Form Game Approach
Johan Peralez, Aurélien Delage, Olivier Buffet, Jilles S. Dibangoye
Part Whole Hierarchy Observable Markov Decision Process Extensive Form Game

January 29, 2024

Effective Communication with Dynamic Feature Compression
Pietro Talli, Francesco Pase, Federico Chiariotti, Andrea Zanella, Michele Zorzi
Deep Reinforcement Learning Mobile Robot Observable Markov Decision Process Feature Compression Vector Quantized Variational Effective Communication Remote Control

December 10, 2023

Multi-Agent Reinforcement Learning for Multi-Cell Spectrum and Power Allocation
Yiming Zhang, Dongning Guo
Multi Agent Reinforcement Learning Task Scheduling Observable Markov Decision Process Interaction Graph Multi Agent Proximal Policy Optimization

October 29, 2023

Spacecraft Autonomous Decision-Planning for Collision Avoidance: a Reinforcement Learning Approach
Nicolas Bourriez, Adrien Loizeau, Adam F. Abdin
Collision Avoidance Observable Markov Decision Process Reinforcement Learning Approach Autonomous Spacecraft Space Domain Awareness Free Space Detection

Observable Markov Decision Process

Papers

OCMDP: Observation-Constrained Markov Decision Process

When to Localize? A Risk-Constrained Reinforcement Learning Approach

POMDP-Driven Cognitive Massive MIMO Radar: Joint Target Detection-Tracking In Unknown Disturbances

Simplified POMDP Planning with an Alternative Observation Space and Formal Performance Guarantees

A POMDP-based hierarchical planning framework for manipulation under pose uncertainty

BoT-Drive: Hierarchical Behavior and Trajectory Planning for Autonomous Driving using POMDPs

Dynamic Game-Theoretical Decision-Making Framework for Vehicle-Pedestrian Interaction with Human Bounded Rationality

Situation-aware Autonomous Driving Decision Making with Cooperative Perception on Demand

Toward an Integrated Decision Making Framework for Optimized Stroke Diagnosis with DSA and Treatment under Uncertainty

From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems

ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraints

POMDP-Guided Active Force-Based Search for Robotic Insertion

Distributed Autonomous Swarm Formation for Dynamic Network Bridging

Belief Aided Navigation using Bayesian Reinforcement Learning for Avoiding Humans in Blind Spots

Provably Efficient Partially Observable Risk-Sensitive Reinforcement Learning with Hindsight Observation

Offline Risk-sensitive RL with Partial Observability to Enhance Performance in Human-Robot Teaming

Solving Hierarchical Information-Sharing Dec-POMDPs: An Extensive-Form Game Approach

Effective Communication with Dynamic Feature Compression

Multi-Agent Reinforcement Learning for Multi-Cell Spectrum and Power Allocation

Spacecraft Autonomous Decision-Planning for Collision Avoidance: a Reinforcement Learning Approach