Partially Observable Markov Decision Processes

Ask about this topic

Overview

Partially Observable Markov Decision Processes (POMDPs) model agents that must choose actions without directly observing the true state of a changing environment. A belief state—a probability distribution over possible states—turns hidden-state uncertainty into a basis for planning, using methods such as point-based value iteration, Monte Carlo tree search, and finite-state controllers. Learning approaches estimate models or beliefs from data, or use recurrent networks and world models; multi-agent and safety-critical settings add challenges such as communication limits, constraints, and risk. A central tension is making belief-space planning scalable while retaining reliable performance and guarantees, especially when observations are noisy, models are misspecified, or available histories do not identify the hidden state.

Latest papers

CardsList

89 papers10 papers in the last four weeks, up 25% on the four weeks before. 0.1% of all new papers.

  1. POMDP-based Object Search with Growing State Space and Hybrid Action Domain

    Apr 16, 2026Yongbo Chen, Hesheng Wang, Shoudong Huang +1Mobile RoboticsMonte Carlo Tree Search

    School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, 200240, People’s Republic of China · School of Computing, Australian National University (ANU), Canberra, ACT, 2601, Australia (The main part of this work is completed at ANU) · Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai, 200240, People’s Republic of China +1

  2. Olfactory pursuit: catching a moving odor source in complex flows

    Apr 13, 2026Maurizio Carbone, Lorenzo Piro, Robin A. Heinonen +3Policy OptimizationPartially Observable Markov Decision Processes

    Istituto dei Sistemi Complessi, CNR, Via dei Taurini 19, 00185 Rome, Italy and INFN “Tor Vergata", Via della Ricerca Scientifica 1, 00133 Rome, Italy · Department of Physics & INFN, Tor Vergata University of Rome, Via della Ricerca Scientifica 1, 00133 Rome, Italy · Machine Learning Genoa Center (MaLGa) & Department of Civil, Chemical and Environmental Engineering, University of Genoa, Genoa, Italy +2

  3. Semantic Rate-Distortion for Bounded Multi-Agent Communication: Capacity-Derived Semantic Spaces and the Communication Cost of Alignment

    Apr 10, 2026Anthony T. NixonMulti-Agent CommunicationTask-Oriented Semantic Communication

  4. Risk-seeking conservative policy iteration with agent-state based policies for Dec-POMDPs with guaranteed convergence

    Apr 10, 2026Amit Sinha, Matthieu Geist, Aditya MahajanPartially Observable Markov Decision ProcessesPolicy Iteration

    Department of Electrical and Computer Engineering, McGill University, Montreal, Quebec · Earth Species Project

  5. PriPG-RL: Privileged Planner-Guided Reinforcement Learning for Partially Observable Systems with Anytime-Feasible MPC

    Apr 9, 2026Mohsen Amiri, Mohsen Amiri, Ali Beikmohammadi +2Partially Observable RLLearning Using Privileged Information

    The authors are with the Department of Computer and System Science, Stockholm University, 11419 Stockholm, Sweden · The authors are with the School of Mechanical and Material Engineering, Washington State University, Pullman, WA 99164, USA

  6. Generating Local Shields for Decentralised Partially Observable Markov Decision Processes

    Apr 8, 2026Haoran Yang, Nobuko YoshidaPartially Observable Markov Decision ProcessesFormal Verification

    University of Oxford

  7. Cross-fitted Proximal Learning for Model-Based Reinforcement Learning

    Apr 6, 2026Nishanth Venkatesh, Andreas A. MalikopoulosCross-FittingReinforcement Learning

    Department of Systems Engineering, Cornell University, Ithaca, NY, USA · Applied Mathematics, Systems Engineering, Mechanical Engineering, Electrical & Computer Engineering, and School of Civil & Environmental Engineering, Cornell University, Ithaca, NY, USA

  8. Communication Gain and Delay Cost Under Cross-Timestep Delays in Cooperative Multi-Agent Reinforcement Learning

    Apr 4, 2026Zihong Gao, Hongjian Liang, Lei Hao +1Partially Observable Markov Decision ProcessesMulti-Agent Communication

    The State Key Laboratory for Manufacturing Systems Engineering School of Automation Science and Engineering, Xi’an Jiaotong University

  9. Belief-State RWKV for Reinforcement Learning under Partial Observability

    Apr 1, 2026Liu XiaoReinforcement LearningPartially Observable RL

  10. Leveraging the Value of Information in POMDP Planning

    Apr 1, 2026Zakariya Laouar, Qi Heng Ho, Zachary SunbergPartially Observable Markov Decision ProcessesBelief-Space Planning

    Department of Aerospace Engineering Sciences, University of Colorado Boulder · Kevin T. Crofton Department of Aerospace and Ocean Engineering, Virginia Tech

  11. Open-loop POMDP Simplification and Safe Skipping of Replanning with Formal Performance Guarantees

    Apr 1, 2026Da Kong, Vadim IndelmanPartially Observable Markov Decision ProcessesModel-Based Planning

    Technion - Israel Institute of Technology

  12. Model-Based Learning of Near-Optimal Finite-Window Policies in POMDPs

    Apr 1, 2026Philip Jordan, Maryam KamgarpourPartially Observable Markov Decision ProcessesPolicy Learning

    SYCAMORE group, Institute of Mechanical Engineering, EPFL Switzerland.

  13. Internal State-Based Policy Gradient Methods for Partially Observable Markov Potential Games

    Apr 1, 2026Wonseok Yang, Thinh T. DoanNatural Policy GradientMulti-Agent Reinforcement Learning

    University of Texas at Austin, Austin, TX, 78712 USA

  14. The Myhill-Nerode Theorem for Bounded Interaction: Canonical Abstractions via Agent-Bounded Indistinguishability

    Mar 22, 2026Anthony T. NixonPartially Observable Markov Decision Processes

    DefSig

  15. Fundamental Limits for Sensor-Based Control via the Gibbs Variational Principle

    Mar 19, 2026Vincent Pacelli, Evangelos A. TheodorouFeedback ControlPartially Observable Markov Decision Processes

    Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA, 30308

  16. A Semi-Decentralized Approach to Multiagent Control

    Mar 12, 2026Mahdi Al-Husseini, Mykel J. Kochenderfer, Kyle H. WrayMulti-Agent ControlMulti-Agent Communication

    Stanford University Stanford, United States · Northeastern University Boston, United States

  17. Safety-critical Control Under Partial Observability: Reach-Avoid POMDP meets Belief Space Control

    Mar 11, 2026Matti Vahs, Joris Verhagen, Jana TumovaControl Lyapunov FunctionsPartially Observable Markov Decision Processes

    Division of Robotics, Perception and Learning, KTH Royal Institute of Technology, Stockholm, Sweden · Digital Futures

  18. LEGS-POMDP: Language and Gesture-Guided Object Search in Partially Observable Environments

    Mar 5, 2026Ivy Xiao He, Stefanie Tellex, Jason Xinyu LiuPartially Observable Markov Decision ProcessesUncertainty-Aware Multimodal Fusion

    Brown University Providence, USA

  19. Principled Learning-to-Communicate with Quasi-Classical Information Structures

    Mar 4, 2026Xiangyu Liu, Haoyi You, Kaiqing ZhangMulti-Agent CommunicationDecentralized MARL

    University of Maryland, College Park, MD, USA, 20742

  20. A Covering Framework for Offline POMDPs Learning using Belief Space Metric

    Mar 3, 2026Youheng Zhu, Yiping LuOff-Policy EvaluationOffline RL

    Northwestern University

  21. Strategic Shaping of Human Prosociality: A Latent-State POMDP Framework

    Mar 2, 2026Zahra Zahedi, Xinyue Hu, Shashank Mehrotra +2Human-Robot CollaborationPartially Observable Markov Decision Processes

    Honda Research Institute USA, Inc. · Department of Cognitive Science, University of California, Irvine.

  22. QANTIS: A Hardware-Validated Quantum Platform for POMDP Planning and Multi-Target Data Association

    Feb 28, 2026Bayram Yüksel Eker, Suayb S. Arslan, Özgür Nazlı +2Partially Observable Markov Decision ProcessesMulti-Object Tracking

    Neura Parse Ltd., London, United Kingdom · Department of Computer Engineering, Boğaziçi University, Istanbul, Turkey · İzmir Ekonomi Üniversitesi, İzmir, Turkey +2

  23. Planning under Distribution Shifts with Causal POMDPs

    Feb 26, 2026Matteo Ceriscioli, Karthika MohanPartially Observable Markov Decision ProcessesBelief-Space Planning

    Causal Intelligence and Reasoning Lab, Oregon State University, Corvallis, OR 97331, USA

  24. Accelerated Online Risk-Averse Policy Evaluation in POMDPs with Theoretical Guarantees and Novel CVaR Bounds

    Feb 26, 2026Yaacov Pariente, Vadim IndelmanPartially Observable Markov Decision ProcessesPolicy Evaluation

    Technion - Israel Institute of Technology, Israel

  25. POMDPPlanners: Open-Source Package for POMDP Planning

    Feb 24, 2026Yaacov Pariente, Vadim IndelmanPartially Observable Markov Decision ProcessesDecision-Making under Uncertainty

    Faculty of Mathematics, Technion - Israel Institute of Technology, Haifa 3200003, Israel · Stephen B. Klein Faculty of Aerospace Engineering, Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology, Haifa 3200003, Israel

  26. Probing Dec-POMDP Reasoning in Cooperative MARL

    Feb 24, 2026Kale-ab Tessera, Leonard Hinckeldey, Riccardo Zamboni +2Cooperative MARLMulti-Agent Reasoning

    University of Edinburgh, Edinburgh, United Kingdom · Politecnico di Milano, Milan, Italy

  27. Toward Learning POMDPs Beyond Full-Rank Actions and State Observability

    Jan 26, 2026Seiji Shaw, Travis Manderson, Chad Kessens +1Partially Observable Markov Decision ProcessesPredictive State Representations

    MIT Computer Science and Artificial Intelligence Lab, 51 Vassar St, Cambridge, MA 02139, USA · DEVCOM Army Research Laboratory 2800 Powder Mill Road, Adelphi, MD 20783, USA

  28. How Well Do Latent World Models Understand Partially Observable Safety Constraints?

    Oct 7, 2025Matthew Kim, Kensuke Nakamura, Andrea BajcsyLatent World ModelsRobot Safety

    UC San Diego · Carnegie Mellon University

  29. Stackelberg POMDP: Learning to Lead via Reinforcement Learning

    Oct 7, 2022Matthias Gerstgrasser, Gianluca Brero, Alon Eden +6Game-Theoretic RLReinforcement Learning

    John A. Paulson School of Engineering and Applied Sciences, Harvard University · Data Science Initiative, Brown University · School of Computer Science and Engineering, Hebrew University of Jerusalem +2