Partially Observable Markov Decision Processes

Overview

Partially Observable Markov Decision Processes (POMDPs) model agents that must choose actions without directly observing the true state of a changing environment. A belief state—a probability distribution over possible states—turns hidden-state uncertainty into a basis for planning, using methods such as point-based value iteration, Monte Carlo tree search, and finite-state controllers. Learning approaches estimate models or beliefs from data, or use recurrent networks and world models; multi-agent and safety-critical settings add challenges such as communication limits, constraints, and risk. A central tension is making belief-space planning scalable while retaining reliable performance and guarantees, especially when observations are noisy, models are misspecified, or available histories do not identify the hidden state.

Latest papers

CardsList

99 papers9 papers in the last four weeks, down 18% on the four weeks before. 0.1% of all new papers.

  1. QANTIS: A Hardware-Validated Quantum Platform for POMDP Planning and Multi-Target Data Association

    Feb 28, 2026Bayram Yüksel Eker, Suayb S. Arslan, Özgür Nazlı +2Partially Observable Markov Decision ProcessesMulti-Object Tracking

  2. Planning under Distribution Shifts with Causal POMDPs

    Feb 26, 2026Matteo Ceriscioli, Karthika MohanPartially Observable Markov Decision ProcessesBelief-Space Planning

  3. POMDPPlanners: Open-Source Package for POMDP Planning

    Feb 24, 2026Yaacov Pariente, Vadim IndelmanPartially Observable Markov Decision ProcessesDecision-Making under Uncertainty

  4. Probing Dec-POMDP Reasoning in Cooperative MARL

    Feb 24, 2026Kale-ab Tessera, Leonard Hinckeldey, Riccardo Zamboni +2Cooperative MARLMulti-Agent Reasoning

  5. Approximating the Reachability Value of Posterior-Deterministic POMDPs

    Feb 7, 2026Nathanaël Fijalkow, Arka Ghosh, Roman Kniazev +2Partially Observable Markov Decision Processes

  6. Toward Learning POMDPs Beyond Full-Rank Actions and State Observability

    Jan 26, 2026Seiji Shaw, Travis Manderson, Chad Kessens +1Partially Observable Markov Decision ProcessesPredictive State Representations

  7. How Well Do Latent World Models Understand Partially Observable Safety Constraints?

    Oct 7, 2025Matthew Kim, Kensuke Nakamura, Andrea BajcsyLatent World ModelsRobot Safety

  8. Stackelberg POMDP: Learning to Lead via Reinforcement Learning

    Oct 7, 2022Matthias Gerstgrasser, Gianluca Brero, Alon Eden +6Game-Theoretic RLReinforcement Learning