Partially Observable Markov Decision Processes

Overview

Partially Observable Markov Decision Processes (POMDPs) model agents that must choose actions without directly observing the true state of a changing environment. A belief state—a probability distribution over possible states—turns hidden-state uncertainty into a basis for planning, using methods such as point-based value iteration, Monte Carlo tree search, and finite-state controllers. Learning approaches estimate models or beliefs from data, or use recurrent networks and world models; multi-agent and safety-critical settings add challenges such as communication limits, constraints, and risk. A central tension is making belief-space planning scalable while retaining reliable performance and guarantees, especially when observations are noisy, models are misspecified, or available histories do not identify the hidden state.

Latest papers

CardsList

99 papers9 papers in the last four weeks, down 18% on the four weeks before. 0.1% of all new papers.

  1. OICycle: Observe, Infer, and Intervene Cycle for Belief-State Learning in Longitudinal Clinical Decision Making

    Nov 17, 2026Jitindra Fartiyal, Pedro Freire, Sergei K. Turitsyn +1Clinical Decision-MakingLongitudinal EHR Modeling

  2. Hilbert States for Decision-Making under Partial Observability

    Nov 16, 2026Zibo Meng, Andrew C. M. Austin, Alin AchimDecision-Making under UncertaintySequential Decision Making

  3. DECODE: Decomposable Environments for COntrolled Sequential-Decision Evaluation

    Nov 13, 2026Diego Silva Prado, Bruno Caldas de Souza, Tiago Amaral Penteado de Jesus +3Sequential Decision MakingPartially Observable Markov Decision Processes

  4. Racing the Clock: Teaching VLNs Budget-Conditioned Object Search

    Nov 12, 2026Maitree Meher, Siming He, S. Shankar Sastry +3Vision-Language NavigationPartially Observable Markov Decision Processes

  5. COMBS: Concealed Object Multi-modal Bayesian Search

    Nov 11, 2026Taylor Bergeron, Kevin LeahyRobotic Object SearchBelief-Space Planning

  6. From Likelihood Convergence to Parameter Convergence in POMDPs

    Nov 9, 2026Jack Zhang, Saurabh Amin, Jiawei Zhang +1Partially Observable Markov Decision Processes

  7. K-PWM: Control-Oriented Structured Bilinear World Models under Partial Observation

    Nov 8, 2026Santosh Mohan Rajkumar, Sriram Narayanan, Samuel E. Otto +1World Model LearningPartially Observable Markov Decision Processes

  8. Quasipolynomial-Time Planning for Decodable POMDPs

    Nov 6, 2026Oliver Dobson, Varun Jog, Chun Hei Michael ShiuPartially Observable Markov Decision Processes

  9. Finite-Memory Control of POMDPs: Fundamental Limits and Efficient Design

    Nov 2, 2026Emrecan Kutay, Atilla Eryilmaz, Ness ShroffPartially Observable Markov Decision ProcessesRecurrent-State Quantization

  10. Performance Gaps Between Open-Loop, Certainty-Equivalent, and Optimal Control Under Action-Dependent Observations

    Oct 24, 2026Leo Maynard-Zhang, Sunmook Choi, Yahya Sattar +3Dual ControlStochastic Optimal Control

  11. Reward Objectives for Robust (PO)MDPs: A Law-Preserving Equivalence with (Partially Observable) Games

    Oct 19, 2026Durgam Latha, Dion Reji, Dorde Zikelic +1Partially Observable Stochastic GamesRobust Partially Observable Markov Decision Processes

  12. Finite-Time Guarantees for Constrained Reinforcement Learning under Partial Observability

    Oct 15, 2026Valentina Zaccaria, Philip Jordan, Tingting Ni +2Primal-Dual RLReinforcement Learning

  13. On Symmetry for Characterizing Agent Heterogeneity in Multi-Agent Systems

    Oct 13, 2026Jianhong Wang, Jonathan LawryPartially Observable Stochastic GamesMulti-Agent Systems

  14. Belief Embedding Tree Search

    Oct 10, 2026Christopher Solinas, Radovan Haluška, Martin Schmid +3Partially Observable RLBelief-Space Planning

  15. On the Complexity Gap Between POMDPs and Multi-Environment POMDPs

    Oct 10, 2026Eline M. Bovy, Caleb Probine, Marnix Suilen +2Robust Partially Observable Markov Decision ProcessesPartially Observable Markov Decision Processes

  16. Revealing After Overwriting: An Exponential POMDP OPE Lower Bound under History-Dependent Logging

    Oct 4, 2026Youyu Luo, Pengzhan Zhou, Zhida Qin +4Off-Policy EvaluationPartially Observable Markov Decision Processes

  17. Risk-Averse Online POMDP Planning via CVaR of the Immediate Cost with Performance Guarantees

    Sep 26, 2026Yaacov Pariente, Vadim IndelmanRisk-Sensitive PlanningPartially Observable Markov Decision Processes

  18. A Decentralized Partially Observable Team Decision Methodology with Delayed Information Sharing

    Sep 22, 2026Xiaoxing Ren, Thomas Parisini, Andreas A. MalikopoulosMarkov Decision ProcessesLinear Markov Decision Processes

  19. Reinforcement Learning under State and Outcome Uncertainty: A Foundational Distributional Perspective

    Sep 21, 2026Larry Preuett, Qiuyi Zhang, Muhammad Aurangzeb AhmadDistributional RLMarkov Decision Processes

  20. SPARROW: Survival-POMCP for Adaptive Robot Routing, Observation, and Waiting

    Sep 17, 2026Hshmat Sahak, Aoran Jiao, Nicholas Rhinehart +1Robot NavigationPartially Observable Markov Decision Processes

  21. Adaptive Agent Design

    Sep 11, 2026Raj Kiriti Velicheti, Subhonmesh Bose, Tamer BaşarReinforcement LearningQ-Learning

  22. Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability

    Sep 9, 2026Arnab Chattopadhayay, Debdipta HalderBelief-Space PlanningLLM Agents

  23. Scalable Rao-Blackwellized Online Planning for High-Dimensional POMDPs

    Sep 1, 2026Jiho Lee, Nisar Ahmed, Kyle Hollins Wray +1Belief-Space PlanningPartially Observable Markov Decision Processes

  24. Solving Robust POMDPs with Omega-regular Objectives via Partially Observable Stochastic Games

    Aug 25, 2026Durgam Latha, Dion Reji, S. Akshay +2Robust Markov Decision ProcessesPartially Observable Stochastic Games

  25. The Curious Case of Exploding DecPOMDPs: Containing the Fire through Policy Counting

    Aug 18, 2026Nazlı Nur Karabulut, Tanya BraunPartially Observable Markov Decision ProcessesMulti-Agent Systems