Belief-Space Planning

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 47

Oct 6, 2026cs.RO

Belief-Space Planning with Planner-Conditioned Estimator Error under Intermittent Observations

Belief-space planners with separately designed or off-the-shelf estimators may have access to state-relevant information the estimator does not observe. Consequently, even an estimator that minimizes mean-squared error under its own information can have a nonzero error mean when conditioned on planner information. When future corrections are intermittent and stochastic, differences between accepted and rejected error means introduce additional uncertainty terms to correction events that zero-mean assumptions ignore. In this paper, we propose a belief-space planning approach that predicts, propagates, and penalizes planner-conditioned estimator error along evaluated trajectories. A planner-conditioned estimator-error model combines affine error dynamics with a moment recursion that preserves stochastically-induced correction uncertainty. We describe a method by which to predict estimator error dynamics within specified operating regimes and incorporate predicted error moments into a task-weighted quadratic risk objective. We evaluate the approach in simulation for a tilt-rotor VTOL landing on a ship deck using receding-horizon planning. We find that conditioning on planner information reduced error-prediction loss for a command-blind EKF by 13.4% relative to a planner-ignorant model, with strongly regime and horizon-dependent forecasting abilities. In a small set of 16 paired closed-loop trials, the planner lowered the median terminal task gauge from 1.60 to 0.99, and predicted estimator error beyond 2s within a factor of 1.3, versus a 5.7-fold underprediction by covariance-only planning.

Peter M. Donley, Joseph L. Moore

Johns Hopkins University Whiting School of Engineering

Belief-Space Planning Robot State Estimation

Oct 6, 2026eess.SY

Belief-Informed Hybrid Control with Almost-Sure Target-Set Convergence

Controlling a hybrid system to a target set under parameter uncertainty can require informative actions that temporarily drive the system state away from the specified set. We propose a belief-informed dual-control algorithm that combines belief-space receding-horizon selection with an expected-decrease constraint on a nonnegative target-set progress function. To accommodate exploratory deviations, a scalar controller state bounds the constraint's cumulative slack. Our algorithm's two-step lookahead selection uses predicted observations to update the parameter belief before evaluating the subsequent action's admissibility. Under distance-comparison bounds, correct conditional prediction, and recursive feasibility, we prove almost-sure target-set convergence at decision times and bound both the sum of expected progress-function values and the expected neighborhood-entry time. Upper confidence bounds extend these convergence guarantees to bounded model samples with summable error probabilities. In planar regulation with an unknown control direction, we verify recursive feasibility: the proposed two-step selection reduces the Euclidean state norm below 0.01 within 11 decisions for either sign, whereas a myopic one-step selection loses admissibility immediately after zero input. In a simulated bimanual assembly task, our algorithm's one-step implementation uses clearance feedback to complete the assembly under nominal friction, yielding a 96.4% lower infinity-norm relative-position error than the reference-tracking baseline at the method's completion time. At lower friction, its posterior conditioning successfully detects an empty admissible set, whereas a fixed-prior alternative admits an action that violates the conditional decrease constraint. Project page: https://clintonenwerem.com/belief-hybrid-control/.

Clinton Enwerem, Saleh Kemal, John S. Baras +1

Institute for Systems Research, University of Maryland, College Park, MD, USA

Belief-Space Planning Adaptive Control

Oct 5, 2026cs.RO

Entropy-Gated Belief Coordination for Decentralized Multi-Agent Search Under Intermittent Communication

We study decentralized multi-agent target search where homogeneous agents communicate intermittently at Poisson-distributed times. Standard unconditional belief fusion wastes communication opportunities by synchronizing agents during high-entropy exploration, when diverse independent beliefs provide better coverage than a premature consensus. We introduce \emph{entropy-gated belief coordination}, in which agents skip fusion while their collective entropy ratio exceeds a threshold~

θ

and merge only during exploitation, consistent with the bifurcation structure of nonlinear opinion dynamics and the submodular structure of the per-step information gain objective. We further derive

\Istep(α,β)

, the expected mutual information per observation step between two agents' binary sensors, as an interpretable, communication-free measure of sensor informativeness that motivates the gating design and guides system-level analysis. Experiments across 103{,}680 trials (nine grid sizes up to

100{\times}100

, Poisson communication timing, four target movement patterns) show that the Entropy-Gated Trust-Decay Planner (\textsc{EG-TDP}), which adds a detection-probability planner switch in exploitation mode, achieves mean belief quality

\bar{Q}=0.300

(mean belief mass at the true target cell, averaged across all trials and steps), a

58.9\%

gain over arithmetic mean and a

37.4\%

gain over visit-weighted fusion. On representative configurations, EG-TDP also outperforms a joint-Bayesian reference that uses all agents'~observations at every step, despite operating under random intermittent contact only.

Mohamed Abdelnaby, Kevin Leahy

Department of Robotics Engineering, Worcester Polytechnic Institute

Belief-Space Planning Multi-Agent Coordination Multi-Agent Systems Multi-Agent Consensus Multi-Agent Communication

Oct 1, 2026cs.RO

BLT*: Informed Belief Localization Trees for Uncertainty-Aware Planning on Digital Twins

We present Informed Belief Localization Trees* (Informed BLT*), a sampling-based belief space planning (BSP) algorithm that scales to large outdoor digital twins with point-cloud observations. We adapt RRT* and Informed RRT* to belief space using the

2

-Wasserstein (

W_2

) metric. Assuming isotropic Gaussian beliefs, sampled belief states can be connected efficiently while accounting for available information and probabilistic collision constraints. This enables steering and rewiring without repeatedly propagating observations, and allows previously computed measurement information to be reused. We present a framework to generate semantically labelled digital twins for planning in real-world environments with point-cloud-based localization. Experiments in simulated environments and digital twins show faster initial solution discovery in most maps with competitive cost convergence.

Elliot Preston-Krebs, Abhishek Goudar, Timothy D. Barfoot

University of Toronto Institute for Aerospace Studies (UTIAS), 4925 Dufferin St, Ontario, Canada.

Belief-Space Planning Digital Twins Robot Localization Sampling-Based Motion Planning

Sep 30, 2026cs.AI

Belief-Aware Multi-Agent Path Finding under Map Uncertainty

Multi-Agent Path Finding (MAPF) aims to find collision-free paths for multiple agents in a shared environment. Classical MAPF assumes that all static obstacles are known in advance, but real-world environments can change unexpectedly due to fallen objects, spills, or other local disturbances. When such changes are spatially correlated, an observation can inform traversability estimates beyond the observed location. Prior approaches address uncertainty in traversability through contingent plans or replanning based on direct observations, but do not leverage this spatial dependence to infer the traversability of nearby unobserved locations. As a result, they cannot use one observation to anticipate nearby unobserved obstacles that may cause costly rerouting later. We focus on Belief-Aware MAPF, where map discrepancies are fixed during execution but initially unknown, and observations can be informative beyond the observed location. We propose Multi-Agent Gaussian belief Inference for Coordination (MAGIC), a framework that updates a shared belief about traversability online based on agents' observations. MAGIC uses a Gaussian Markov Random Field and Gaussian Belief Propagation to approximately infer traversability and construct detour-aware costs for standard MAPF planners. Our experiments on MAPF benchmarks show that MAGIC reduces the executed sum of costs compared to existing approaches on 96.3% of instances, across several planner families and teams of up to 800 agents, demonstrating its applicability to large-scale MAPF problems.

Viraj Parimi, Shao-Hung Chan, Han Zhang +2

Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA 01239. · Symbotic Inc., Wilmington, MA 01887, USA

Belief-Space Planning Multi-Agent Path Finding

Sep 30, 2026q-bio.NC

Belief-Based Maximum Occupancy Principle and Active Inference

Intrinsic motivation plays a central role in adaptive and goal-directed behavior by conferring agents reward-independent objectives and biases useful to act in noisy and uncertain environments. Active Inference addresses the problem of acting in a partially observable environment through a principled framework for belief updating and action selection. A key component of Active Inference is the specification of prior preferences, which shapes behavior by encoding desirable future outcomes. An intrinsic motivation approach called the Maximum Occupancy Principle (MOP) proposes that agents act so as to maximize occupancy over future paths of states and actions, with no preferences or epistemic targets. Despite its simple formulation, MOP gives rise to rich and adaptive behaviors that combine exploratory variability with goal-directed dynamics. In this work, we extend MOP to partially observable environments and introduce a Bellman reformulation of the Expected Free Energy for Active Inference, both incorporating belief-based inference over hidden states as part of the agent state. The Bellman formulation enables tractable offline computation via value iteration over the full belief-state space. We compare the resulting behaviors in a set of minimal experimental settings with uncertain food sources. We find that MOP agents switch between goal-directed (food seeking) behavior and exploration between different food sources, depending on their energy available and their belief state. In contrast, Active Inference agents mostly inhabit regions around a single food source, a strategy having both high pragmatic and epistemic value. We finally compare with Empowerment, which is shown to be qualitatively similar to Active Inference.

Manolis Mylonas, Rubén Moreno Bote

Center for Brain and Cognition & Department of Engineering Universitat Pompeu Fabra Barcelona, Spain

Belief-Space Planning Intrinsic Reward Methods Active Inference Partially Observable RL Free Energy Principle

Sep 30, 2026cs.AI

Beyond the Remembered World: Predictive 4D Belief for Persistent Navigation in Evolving Worlds

Persistent spatial memory enables embodied agents to navigate familiar environments across repeated visits. However, targets may move while unobserved, including during navigation, making remembered locations unreliable by the time an agent arrives. Despite advances in memory retrieval and state prediction, accounting for continued hidden world evolution and revising beliefs under limited visibility remain challenging. We study Evolving-World Navigation, where agents infer target locations from intermittent observations, predict their states at inspection time, and revise beliefs using visual evidence. We propose EvolvingNav, which constructs a time-indexed belief from timestamped 3D object histories through a structured persistence-relocation model. The belief distinguishes persistence at the last observed location from relocation to alternative locations and retains probability mass outside the known candidate set. An event-driven filter propagates the current belief as time elapses, forecasts target occupancy at candidate inspection times, and incorporates new RGB-D evidence. Negative observations downweight location hypotheses according to calibrated, visibility-conditioned detection probabilities, while evidence tracking prevents repeated use of the same observations. A frozen, zero-shot vision-language controller uses the updated belief to choose actions and replan. We further introduce EvoWorld-Bench, a benchmark grounded in human activity traces, comprising 54 scenes and 803,680 tasks with controlled changes before and during navigation. In simulation and real-robot experiments, EvolvingNav improves navigation success and search efficiency over the evaluated baselines. Paired experiments show the clearest gains under learnable temporal patterns, while ablations demonstrate the value of preserving uncertainty and incorporating visibility-aware evidence.

Mingjian Gao, Zhaocheng Li, Haoyang Huang +7

Zhejiang University · University of California, San Diego · Deeprobotics +1

Belief-Space Planning Embodied Navigation Robot Navigation

Sep 24, 2026cs.RO

AquaMend: Minimal Re-probing and Conditional Rollback for Latent-Belief Failures in Embodied Agents

Physical changes or sensing errors can invalidate embodied agents' task-relevant beliefs. AquaMend compares re-probing, rollback, and supported continuation on a probe-belief-action graph under an expected-loss objective covering sensing, physical recovery, and uncorrected failures. A joint posterior guides a one-step policy with conditional detection-power screening. The per-belief three-way optimum requires independence, separability, and fully resolving probes; the general policy has no global optimality guarantee. Across 32 paired scenarios in a self-constructed simulation benchmark, AquaMend recovers in 28/32 cases and reduces mean complete loss by 21.6% versus restart. Its paired loss difference from decision-theoretic troubleshooting (DTT) is not statistically significant after Holm correction. Against the all-candidate ablation, online decision time decreases by 12.3% overall but increases by 3.4% in the uncovered late stage.

Yufan Liu, Shang Luo, Yang Liu +9

University College London · Peking University · School of Electrical and Electronic Engineering, Nanyang Technological University +4

Belief-Space Planning AI Agent Reliability Active Perception Embodied AI

Sep 17, 2026cs.RO

Spatial-Semantic Uncertainty in VLM-Based Target Search: Balancing Exploration and Identification

Robots searching for a target from a natural-language description must determine not only where to search, but also which observed candidate is the desired target. These decisions reflect two distinct sources of uncertainty - spatial uncertainty over candidate locations and semantic uncertainty over target identity - that are often conflated in VLM-based search systems. We introduce a spatial-semantic uncertainty formulation that maintains separate beliefs over each component and integrates probabilistic VLM evidence into a global target-identity posterior, including probability mass for undiscovered targets. This decomposition allows an information-theoretic planner to independently value candidate discovery and target disambiguation through spatial and semantic expected information gain (EIG), providing an explicit mechanism for trading broader exploration against earlier identification. We evaluate six VLM uncertainty-elicitation interfaces on 500 synthetic targets and show that similar recognition accuracy can conceal substantial differences in calibration and false confidence. In degraded-observation search-and-identify experiments, EIG-based planners reach confident decisions in 75.0%-92.5% of trials, compared with 20.0% for Random search, while different spatial-semantic weightings achieve comparable identification accuracy once confidence is attained. Increasing semantic emphasis reduces unnecessary exploration and VLM queries, demonstrating that explicitly planning over semantic uncertainty can accelerate target resolution without sacrificing decision quality. These results highlight the distinct roles of uncertainty representation and uncertainty-driven planning in embodied VLM systems.

Alkesh K. Srivastava, Jonathan Diller, Vijay Kumar +1

Temple University, Philadelphia, PA, 19122 USA · University of Pennsylvania, Philadelphia, PA, 19104 USA

Belief-Space Planning Vision-Language Models Informative Path Planning Robotics VLMs for Robotics

Sep 17, 2026cs.RO

Imagine-TAMP: Imagination-Guided Task and Motion Planning in Partial Observability

Robots operating in cluttered environments must often manipulate objects whose locations are only partially observable. A central challenge is deciding whether to acquire another observation or to first manipulate objects that may occlude the target. Conventional task and motion planning (TAMP) approaches typically make this decision using symbolic action costs or expensive geometric planning, neither of which adequately captures how likely an observation is to reveal an occluded target. We introduce Imagine-TAMP, an interleaved planning and execution framework that uses semantic and geometric imagination to compare alternative task-level strategies under partial observability before committing to expensive motion planning. A vision-language model shapes a particle belief over target locations using commonsense relationships between the target and visible objects, while a generative scene model estimates plausible geometry in unobserved regions. Given a target hypothesis and imagined scene, Imagine-TAMP generates multiple symbolic plan skeletons and assigns non-unit costs that approximate both manipulation effort and target visibility from sensing actions, distinguishing a short but poorly informative observation strategy from a longer strategy that first manipulates an occluder to better expose the target. The selected skeleton is then refined into a feasible continuous plan and executed, with new observations updating the belief and triggering replanning when necessary. Experiments show that imagination-guided evaluation improves observation-versus-manipulation decisions: in viewpoint-constrained shelf scenes, non-unit geometric evaluation increases success from 46.0% to 84.0%, while semantic belief shaping further reduces manipulation and replanning. On a real robot, the complete system reduces planning time by 32% relative to a geometry-only ablation.

Antareep Singha, Shivaram Kumar, Yoonwoo Kim +1

Nanyang Technological University · The University of Texas at Austin

Belief-Space Planning Active Perception Robotic Manipulation Motion Planning

Sep 16, 2026cs.RO

GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning

Large language models (LLMs) provide a flexible interface for long-horizon robot planning, but generated plans often fail to respect embodiment constraints, recover from planning errors, or reason effectively under partial observability. We present GAVEL, a framework for verifying and repairing long-horizon LLM planning built around an explicit graph world model. The graph represents relevant object-relations, action pre-conditions and effects, and probabilistic beliefs over unobserved object locations. This model can predict the consequences of LLM-generated actions before execution, detect violations, and repair those whose corrections follow directly from the world model. This method also reserves LLM replanning solely for errors requiring semantic reasoning. For multi-task instructions, GAVEL reasons over distributions of possible object locations to reorder remaining subtasks and minimize expected search cost. We evaluate GAVEL on BEHAVIOR-1K across 100 single long-horizon tasks and 500 multi-task instructions. With Qwen3-8B, GAVEL improves single-task success from 41.2% to 91.8% and multi-task success from 19.9% to 92.6%. Distributional belief reasoning also reduces travel distance by approximately 5.4% compared with a static variant. These improvements show that an explicit graph world model harness can substantially improve the reliability and efficiency of long-horizon embodied planning across compact and frontier hosted LLM capabilities.

Ruiyang Wang, Hao-Lun Hsu, Swarajh Mehta +3

Belief-Space Planning Large Language Model-Based Robot Planning World Model-Based Planning Long-Horizon LLM Agents Embodied World Models

Sep 15, 2026cs.AI

Bridging Learned Visual Perception and Symbolic Belief-Space Planning

In partially observable settings, agents must act without full knowledge of the world state and rely on uncertain state-estimation pipelines. Obtaining grounded and verifiable symbolic plans under such uncertainty remains a key challenge. Recent work has integrated Vision-Language Models (VLMs) to bridge perception and symbolic reasoning, following two main paradigms. The first, VLM-as-planner, maps images directly to action sequences, and the second, VLM-as-grounder, grounds observations into symbolic predicates used as the initial state by off-the-shelf planners. Both approaches ignore uncertainty in the planning process, compromising robustness. We introduce a third paradigm, VLM-as-probabilistic-grounder, a novel approach that captures the uncertainty of VLM predicate groundings as a probability distribution over symbolic states. This enables planning in belief space and producing robust plans under uncertainty. Experiments in simulated household robot settings show improved robustness and task success over deterministic grounding, underscoring how our approach leverages foundation models for reliable planning under uncertainty.

Guy Azran, Michael Navat, Sarah Keren

Taub Faculty of Computer Science, Technion - Israel Institute of Technology · Faculty of Mathematics, Technion - Israel Institute of Technology

Belief-Space Planning Symbolic Planning VLMs for Robotics

Sep 9, 2026cs.AI

Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability

Large language model agents produce fluent action sequences across a wide range of tasks, yet they fail in characteristic ways once the environment becomes partially observable. Ambiguous feedback pushes them into premature commitments. A single informative observation can collapse their uncertainty onto the wrong hypothesis. Policies drift as the history grows. We trace these symptoms to a common structural cause. An LLM agent, as commonly deployed, is a history-conditioned policy with no explicit belief over hidden state. We propose an architectural fix. The Belief-State Engine (BSE) is an inference module placed outside the LLM. It maintains a Bayesian posterior over the latent states of a given POMDP (Partially Observable Markov Decision Process) model, and at each decision step it exposes only that posterior to the LLM. The raw action-observation log is not shown. We set out a minimal four-axiom specification of what a belief-consistent internal state must satisfy, and prove that the LLM paired with the BSE is a sound Markov policy on the belief MDP induced by the underlying POMDP. It therefore inherits the Bellman optimality guarantees of classical POMDP theory, provided the LLM is never exposed to the raw history. We evaluate the architecture on the Tiger POMDP and a red-team attack-graph task, against six baselines: a reactive LLM, Chain-of-Thought, ReAct, a natural-language belief tracker, QMDP, and POMCP. Across both domains, the BSE-augmented agent improves task return, belief calibration, and decision consistency. Ten targeted ablations isolate the contribution of each architectural choice confirms that the effect is not specific to any one model. Code, environment specifications, prompt templates, and seed logs accompany this paper.

Arnab Chattopadhayay, Debdipta Halder

Independent Researcher (UCL Alumni), Bangalore, India · Independent Researcher (IIT-Kharagpur Alumni), Bangalore, India

Belief-Space Planning LLM Agents Belief Updating in LLMs Partially Observable Markov Decision Processes

Sep 1, 2026cs.RO

Scalable Rao-Blackwellized Online Planning for High-Dimensional POMDPs

Online planning under uncertainty remains a fundamental challenge for robotic systems operating in partially observable environments with high-dimensional state spaces. While sampling-based POMDP solvers enable approximate decision-making in large or continuous domains, their performance degrades as belief dimensionality increases due to the high variance inherent in Monte Carlo-based estimation. In this work, we extend the Rao-Blackwellized online POMDP (RB-POMDP) framework to improve its generalizability in high-dimensional settings through hybrid continuous-discrete belief representations. By analytically propagating uncertainty associated with marginalized state components during tree-based planning, the proposed approach reduces sampling-induced variance in value estimation. We demonstrate the effectiveness of this framework in a robotic search-and-rescue task by integrating it with FastSLAM 2.0. Experimental results show that the proposed planner achieves higher cumulative rewards using significantly fewer particles and planning simulations than purely sampling-based methods under equivalent computational budgets. These results suggest that structured high-dimensional robotic problems admitting tractable sufficient statistics can be effectively leveraged within the RB-POMDP framework for computationally feasible online decision-making.

Jiho Lee, Nisar Ahmed, Kyle Hollins Wray +1

Ann And H.J. Smead Aerospace Engineering Sciences Department, University of Colorado Boulder, Boulder, CO, USA · Manning College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, USA

Belief-Space Planning Partially Observable Markov Decision Processes Decision-Making under Uncertainty

Aug 31, 2026cs.AI

Towards a Belief-Based World Model for LLM Agents

Large language models (LLMs) are being used as policies for autonomous decision-making and planning in many domains. Despite their strong reasoning capabilities, LLMs struggle with long-horizon tasks, especially under partial observability. World models are a promising way to enhance policy performance, both during training and inference. During inference, agents currently use world models to simulate the consequences of candidate actions before choosing an action, which can improve decision-making. However, we argue that simulation alone is an incomplete interface for decision-making under partial observability: simulation does not adequately capture uncertainty about the current state, which agents may need for accurate decision-making. We address this limitation with Belief-Based World Models (BB-WMs), which maintain a belief that LLMs can query to access information on what is known and uncertain about the current state. Before developing methods to learn accurate BB-WMs, this paper focuses on a more fundamental question: does exposing a world model's belief directly to an LLM policy improve decision-making? Our results show that giving LLM agents access to beliefs improves task performance under partial observability, while remaining complementary to existing simulation-based world models. Code: https://github.com/skumar-ml/belief-world-models.

Shubham Kumar, Harshit Kumar, Narendra Ahuja +1

UIUC · IBM

Belief-Space Planning Partially Observable RL World Models World Model-Based Planning Long-Horizon LLM Agents

Aug 5, 2026cs.LG

Capability-Gated Planning: Cost-to-Goal Discovery and the Limits of Myopic Experiment Selection

Systems that automate scientific discovery must repeatedly decide which experiment to run, which hypothesis to test, which tool to build, and when to stop. Many systems make these decisions by maximizing a myopic score such as expected information gain per unit cost or a learned plausibility score. We identify a structural limitation of this approach. Some actions are constructive: they acquire an epistemic capability (an instrument, assay, pipeline, simulator, or abstraction) whose value lies not in the information returned immediately but in the future actions it makes available. When the least-cost route to a confident answer requires a chain of such constructions, a planner that scores actions only by information obtainable within a bounded horizon cannot value the first construction: it yields no information within the horizon and is dominated by any measurement with positive information, however small. We formulate goal-directed discovery as a stochastic shortest-path problem in belief space in which constructive experiments change the downstream action graph, and prove that for every lookahead depth d there is an instance on which every myopic information-maximizing planner has an unbounded approximation ratio, and a related instance on which it never reaches the goal. The mechanism is a capability-indistinguishability lemma: within the horizon, acquiring a capability can be observationally indistinguishable from paying for a null action. This establishes capability gating as a reachability axis of difficulty distinct from curvature (submodularity) and information order (adaptivity gaps). We introduce CG-Plan, an incremental replanner with a capability-aware cost-to-go heuristic h = h_cap + h_exp. In a controlled testbed, the performance gap appears only under gating, persists for every fixed horizon, and arises when near-miss hypotheses come from a data-consistent proposer.

Ahmed Hassoon, Mark Dredze

Johns Hopkins University

Belief-Space Planning Cost-Aware Inference Autonomous Scientific Discovery Autonomous Experimentation

Aug 4, 2026cs.RO

POMDPs for Autonomous Science Exploration

Autonomous exploration missions require decision-making under sensor uncertainty and computational constraints, yet integrating scientific representations into POMDP planning has remained intractable due to high-dimensional observation spaces. Information-theoretic planners overcome this by assuming deterministic observations, sacrificing the principled uncertainty quantification that POMDPs provide. We introduce the Science Hypothesis Map POMDP (SHM-POMDP), which makes science-driven belief-space planning more tractable by branching on inferred physical properties rather than raw sensor data. This preserves full sensor information through learned observation models while enabling the planner to reason jointly about navigation and scientific properties under uncertainty. On an extended RockSample domain with 50-dimensional observations, SHM-POMDP achieves 18.6% higher rewards and 32.9% reduced computation time per step than continuous-observation baselines. On realistic geologic exploration using Cuprite hyperspectral data, SHM-POMDP achieves 2.5

\times

higher information gain than the best information-theoretic baseline by maintaining beliefs and replanning adaptively---reaching 80% of oracle performance using only uniform priors. These results demonstrate that integrating hierarchical probabilistic models into belief-space planning enables tractable, principled autonomous science that outperforms both traditional POMDP methods and science-aware information-theoretic approaches.

Daniel Guirguis, Nathan Wallace, Hanna Kurniawati +1

Belief-Space Planning AI for Science Autonomous Scientific Discovery Partially Observable Markov Decision Processes

Jul 31, 2026cs.AI

NeSyFS: A Neuro-symbolic Fast-Slow Thinking Framework for LLM Agent under Partial Observability

Recently Large Language Models (LLMs) have been increasingly deployed as autonomous agents in applications such as self-reflection, retrieval-augmented generation, and scientific discovery. In these settings, agents must act based on limited observations rather than full environmental states, leading to partial observability. This introduces several key challenges: belief state inference, task objective misalignment, and planning under uncertainty. Prior approaches typically condition actions on full or summarized action-observation histories whose redundant and irrelevant information can mislead the decision making of LLM agent. Inspired by human cognition, we propose a novel neuro-symbolic fast-slow thinking (NeSyFS) framework for LLM agent, addressing the challenges introduced by partial observability in a unified approach. We use a knowledge graph (KG) to represent the belief state, providing triplets as context for every module of NeSyFS. The fast-thinking module performs reactive action, while slow-thinking conducts a new uncertainty-aware planning by following the high-level structure of twisted sequential Monte Carlo (TSMC) algorithm. To mitigate the misalignment of task objective, a reflection module is used to reflect fast-thinking actions, and also switches to the slow-thinking module whenever reactive actions repeatedly fail. Experiments on three representative benchmarks, i.e. ALFWorld, Webshop, and ScienceWorld, demonstrate significant advantages over previous methods.

Duo Xu, Faramarz Fekri

Georgia Institute of Technology 225 North Ave NW, Atlanta, GA 30332

Belief-Space Planning LLM Agents Neuro-Symbolic Reasoning LLM Agent Planning

Jul 15, 2026cs.RO

Merging Reaction to Cognition: A Hybrid Cognitive Strategy for Odour Source Localisation in Natural Environments

Chemical pollutants released into the environment are transported by turbulent flows, generating complex, intermittent plume structures that threaten ecosystems and human health. Rapid localisation of emission sources is critical, and field robots equipped with chemical sensors provide a viable means to perform this task. However, inferring source location from sensor readings remains difficult due to sparse detections and the absence of reliable concentration gradients. Existing approaches fall into two paradigms. Bio-inspired strategies rely on reactive behaviours triggered by detections, such as surge-casting, offering efficiency but requiring scenario-specific tuning. Cognitive strategies integrate observations into a probabilistic belief over source location. While more robust, they suffer from excessive exploration and strong dependence on belief accuracy. The Fast-Cognitive algorithm reduced this computational burden but preserved the fundamental limitations. Previous Markov chain analysis revealed that source-directed motions occur roughly twice as often following odour detections, indicating that reactive behaviours naturally emerge within cognitive frameworks. This work proposes a hybrid strategy that explicitly incorporates bio-inspired reactivity into belief-dependent motion planning. It introduces a detection-triggered switching mechanism formalising transitions between crossflow exploration and source-directed motion, prioritising source proximity over information gain. Behavioural parameters are derived directly from belief metrics, enabling adaptive reactivity without manual tuning. The approach is validated through simulations under three turbulence conditions and field experiments with an autonomous surface vehicle in the Mondego River, Portugal. Results show up to 50% reduction in travelled distance, 86% success rate, and 3.2m average localisation error.

Hugo Magalhães, Rui Baptista, Lino Marques

Institute of Systems and Robotics · Department of Electrical and Computer Engineering · University of Coimbra, 3030-290, Coimbra, Portugal

Belief-Space Planning Robot Navigation

Jul 11, 2026cs.RO

Interleaved POMDP Planning for Multi-Object Search in Unknown Multi-Room Household Environments

Multi-object search in unknown household environments requires planning under extensive uncertainty - from unknown object locations to cluttered spaces with unobserved obstacles. POMDPs offer a principled framework for such problems but remain intractable in large domains. We propose Inter-POMDP, a novel interleaved POMDP planning algorithm that decomposes this challenge into two interacting levels: a high-level POUCT planner reasons over object distributions using LLM-informed histogram beliefs, while a low-level motion planner models navigation uncertainty with obstacle-aware particle beliefs as domain knowledge to guide high-level POUCT. This interleaved design balances planning quality and efficiency despite the large search space across unknown multi-room environments. Both simulation and real-world experiments show that our Inter-POMDP algorithm reduces collision counts by up to 63%, navigation steps by up to 35%, and detection counts by up to 32% compared with baseline methods. Full videos are https://sites.google.com/view/inter-pomdp

Ruochu Yang, Ziyi Xia, Huibo Zhang +6

Georgia Institute of Technology, USA · The Hong Kong University of Science and Technology, China · Ising AI, China +2

Belief-Space Planning Partially Observable Markov Decision Processes Robot Motion Planning

Jul 7, 2026cs.AI

QANTIS: Hardware-Calibrated Sequential POMDP Belief Updates on IBM Heron

Autonomous systems under partial observability act on beliefs, not raw sensor events. QANTIS treats the quantum processor as a calibrated belief-update service in that loop: it receives a prior and an observation model, estimates the rare-event evidence term, and returns an ordinary posterior to a classical planner. This paper asks whether that service can be reused across a sequential Tiger POMDP horizon on present IBM Heron hardware without corrupting the planner-facing posterior. We answer with a controlled hardware case study rather than an end-to-end autonomy or wall-clock speedup claim. The study compares no amplification, guarded Grover amplification, and all-step fixed-point amplification on the same trajectory, then checks whether the returned posterior would change the downstream action. All-step FPAA preserves the Tiger posterior across the reported 8-step and 12-step primary runs, and the 20-step and 32-step controls remain inside the same operating band. In every reported decision check, the hardware posterior and the exact Bayes posterior select the same immediate action. Boundary-aware BIQAE stabilizes amplitude estimation near zero and near one, while a rare-event sweep maps the logical sample-complexity envelope for one-in-a-million evidence. The result is an operating envelope for a hardware-calibrated belief-update primitive, not a standalone hardware-advantage claim.

Bayram Yuksel Eker, Suayb S. Arslan, Ozgur Nazli +2

Neura Parse Ltd. London, United Kingdom · Dept. of Computer Engineering Boğaziçi University Istanbul, Turkey · Dept. of Physics Izmir University of Economics Izmir, Turkey +2

Belief-Space Planning Bayesian Filtering Partially Observable Markov Decision Processes

Jul 4, 2026eess.SY

Finite Reliability Representations: Noise-Calibrated Belief-Space Covers for Reliable Decision-Making

Physical sensing and actuation noise floors should inform how much belief resolution a decision-making system can reliably use. We introduce Finite Reliability Representations (FRR), a framework for covering belief spaces by reliability cells: regions within which the optimal action-value function Q*(b,u) varies by at most a tolerance epsilon, uniformly over actions. The framework is formulated on beliefs rather than states and uses a cover rather than an equivalence quotient, because approximate decision-closeness is not transitive in general. A central technical point is that noisy Bayesian updates should not be treated as globally contractive on arbitrary beliefs. We therefore separate three objects: the fixed-observation filter map, the predictive observation law, and the controlled belief-transition kernel. For nonlinear continuous-state systems, FRR is obtained under a reachable-set Lipschitz modulus for the belief-transition kernel. For finite-state POMDPs, the same construction becomes exact on the belief simplex: prediction is linear, Bayesian correction is a normalized positive linear map, sensor noise enters through observation-distribution distinguishability, and actuation uncertainty enters through an action-execution channel. Under the corresponding action-value Lipschitz condition, an FRR cover supports a cell-constant policy whose suboptimality is bounded by 2 epsilon/(1 - gamma). We also introduce reliability entropy, the logarithm of the minimal number of reliability cells, as a measure of certified decision-relevant belief complexity. The framework distinguishes representation sufficiency from fundamental performance floors imposed by sensing, process, and actuation noise. It applies to finite POMDPs, linear-Gaussian filters, locally linearized nonlinear filters, and particle-filter implementations through analytic or empirical certification of reliability cells.

Hyung-Jin Yoon, Hunmin Kim

Belief-Space Planning Partially Observable Markov Decision Processes Decision-Making under Uncertainty

Jul 3, 2026cs.RO

Continuous-Time Gaussian Belief Trees for Motion Planning

We address sampling-based motion planning for continuous-time stochastic systems under process and measurement uncertainty, with probabilistic guarantees on safety and performance. The robot dynamics are modeled as a continuous-time linear stochastic differential equation, while sensor measurements arrive at discrete time instants. We derive an offline hybrid belief propagation model in which the belief evolves according to continuous-time ODEs between measurements and undergoes discrete Kalman filter update jumps at measurement times. To ensure safety, we introduce a belief-barrier-function-based safety checker for segment-level probabilistic verification. This enables the planner to certify safety over entire continuous trajectory segments and detect inter-sample chance-constraint violations that are missed by conventional node-based checks. Together, these components provide a principled framework for sampling-based belief planning that accounts for both continuous-time uncertainty propagation and continuous-time safety requirements. We integrate the method with RRT and SST planners and evaluate it across multiple benchmark environments. The results show that the proposed method achieves high success rates and robust enforcement of chance constraints, including in narrow-passage scenarios where discrete-time counterparts fail due to missed inter-sample unsafe behavior.

Rayan Mazouz, Qi Heng Ho, Zachary N. Sunberg +1

Department of Aerospace Engineering Sciences at the University of Colorado Boulder, CO, USA · Department of Aerospace and Ocean Engineering at Virginia Tech, Blacksburg, VA, USA

Belief-Space Planning Constrained Motion Planning Safe Motion Planning Motion Planning Sampling-Based Motion Planning

Jun 30, 2026cs.AI

Ask the World Before Acting: Environment Probing for Calibrated Agent World Models

Language agents acting over long horizons must maintain beliefs about tool states, object locations, graph edges, and subgoal dependencies. When these beliefs drift, failures can be fixed neither by longer reasoning traces nor by ordinary self-reflection, since the missing evidence lies in the environment. We formulate environment probing as a budgeted decision problem for structured agent world models: before acting, the agent may query the current value of one belief field, update its table, and pay one interaction step. We introduce EnvProbe, a simple scoring policy that combines task criticality, staleness, verbalized uncertainty, and dependency role. A type-stratified analysis separates the benefit of belief repair from the cost of displaced task actions and predicts different behavior for procedural and spatial beliefs. In three controlled environments with gold belief states, EnvProbe improves terminal world-state accuracy over periodic probing by 11.76 percentage points on procedural tool-dependency tasks, 3.79 points on spatial tasks, and 6.45 points overall. Ablations show that task-structural terms are the main source of the gains, while self-reported uncertainty is unreliable under confident wrong beliefs. The results suggest that agent calibration should be treated as an action-selection problem over environment evidence, not only as a model-internal reasoning problem.

Xinyuan Song, Zekun Cai

Emory University, Atlanta, GA, USA · The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan

Belief-Space Planning World Model Learning Long-Horizon Agent Tasks Uncertainty Calibration

Jun 30, 2026cs.CV

Hierarchical 3D Scene Graph Construction and Belief-based Planning for Semantic Navigation

Semantic navigation is a fundamental task for embodied agents operating in unseen environments, requiring both semantic understanding and long-term decision-making. Recent foundation models have empowered agents with rich semantic priors for this task. However, without structured global representations, decision-making often falls back on local observations and greedy strategies, resulting in inefficient exploration and myopic behaviors, especially in long-distance navigation. To address these challenges, we propose a zero-shot semantic navigation framework. Our method incrementally maintains an online Hierarchical 3D Scene Graph (HSG) to form a multi-granular semantic topology over objects, zones, and regions, serving as a compact state abstraction for global planning. Building on this memory, we introduce a hierarchical belief-based planning framework that fuses semantic priors with exploration evidence on the HSG, and performs finite-horizon rollouts on an HSG-based simulator to explicitly estimate the long-term expected returns of candidate macro-actions. This enables globally consistent decisions and reduces redundant backtracking. Extensive experiments in high-fidelity simulation environments across multiple tasks and datasets demonstrate that our method outperforms existing state-of-the-art methods, particularly in long-distance scenarios, where our approach improves SR and SPL by an average of 9.4% and 5.0%, respectively.

Bing Wu, Zuyao Chen, Changwen Chen

The Hong Kong Polytechnic University

Belief-Space Planning Embodied Navigation 3D SGG Long-Horizon Agent Tasks

Jun 29, 2026cs.MA

ECHO: Learning Epistemically Adaptive Language Agents with Turn-Level Credit

What does it mean for a language agent to be adaptive? Effective multi-turn agents must decide what information to seek, how to use new evidence, and when they are certain enough to act. We introduce Epistemic Decision Processes (EDPs), a belief-state formulation of multi-turn information seeking in which actions produce external observations that update the agent's posterior over a latent task variable. EDPs make epistemic adaptivity explicit: good policies choose actions that are useful under the current belief, not merely those that correlate with eventual success. We prove that belief-agnostic policies can suffer errors that compound exponentially over the horizon, and that aggregate trajectory returns can fail to identify the per-turn Bayesian advantage needed for epistemic credit. We then introduce ECHO (Epistemic Credit for History-Conditioned Optimization), a practical clipped policy-gradient objective that assigns turn-level credit using posterior-sensitive rewards. In the Clue Selector Game, a novel controlled evidence-seeking benchmark, we show that ECHO substantially improves resolution, information gain, and efficiency over trajectory-level GRPO, and matches or exceeds frontier baselines on epistemic metrics such as grounding, recovery, and calibration while producing almost no visible reasoning text.

Abhijnan Nath, Nikhil Krishnaswamy

Situated Grounding and Natural Language (SIGNAL) Lab Department of Computer Science, Colorado State University Fort Collins, CO 80523 USA

Belief-Space Planning Credit Assignment in RL AI Agent Benchmarks RL for LLM Agents

Jun 24, 2026eess.SY

PRISM: Efficient and Locally Optimal Probabilistic Planning with Reachability Guarantees

Belief-space planning under motion uncertainty and state and control constraints remains a fundamental challenge, largely due to the difficulty of establishing reachability guarantees in constrained belief spaces. Existing constrained belief-space planners rely on sampling to construct multi-query belief roadmaps and explicitly find feasible trajectories between sampled nodes to establish reachability. These methods often struggle to cover the belief space or use robust control techniques that improve coverage at the cost of indirect, high-cost trajectories; they also lack finite-time or finite-memory completeness guarantees. We propose PRISM, a multi-query motion planning algorithm for belief spaces with state and control constraints that targets both high coverage and low cost. We present a new result on controllability of the state covariance under constraints, which is used by PRISM to decompose belief-space planning into deterministic mean planning and covariance shrinking. PRISM further includes an online local optimization method that reduces the cost of feasible belief-space trajectories. Under mild assumptions on the start and goal distributions, we prove that PRISM guarantees full coverage (i.e. completeness) despite actuator and obstacle constraints. In challenging simulated scenarios, PRISM achieves substantially higher roadmap coverage than state-of-the-art belief-space planning methods while producing trajectories with lower mean cost and cost variance. For example, PRISM achieves 100% coverage in easy and medium-difficulty scenarios, and, in the hardest scenario, which violates PRISM's coverage assumptions, it still achieves 97-100% coverage, while all other methods achieve less than 45%.

Alex Rose, Christopher Jewison, Jonathan P. How

Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge MA 02139 · Draper, Cambridge MA 02139

Belief-Space Planning Constrained Motion Planning Trajectory Optimization Reachability Analysis Sampling-Based Motion Planning

Jun 20, 2026cs.AI

REBA: A Revealed Belief Automaton Framework for Online Planning in Continuous POMDPs

Online planning in continuous partially observable Markov decision processes (POMDPs) using

ω

-regular specifications requires handling continuous belief dynamics within the finite symbolic memory in order to track temporal progress. Existing methods based on either direct search in belief space or predefined discrete abstractions suffer from drawbacks, e.g., lack of symbolic memory for long-horizon logical progress or difficult to certify from noisy online beliefs. As such, obtaining reliable symbolic states online from continuous observations remains a challenge. To address this issue, we introduce the Revealed Belief Automaton (REBA), an event-driven framework that advances the research from global belief-space discretization to a fundamental new way of thinking, namely online certification of revelation events. Specifically, we propose an online revelation method that, through information-theoretic gates, can dynamically analyse and establish belief abstraction from the continuous belief space by discovering reliable anchors among noisy beliefs. We then develop an incremental topology adaptation mechanism over the certified anchors to realise the online finite Belief Automaton. By combining with the

ω

-regular specification, REBA is able to support formal parity policy synthesis without a predefined discrete abstraction, which in turn can guide the Monte Carlo Tree Search process to perform online search beyond its local horizon. In addition, we design an error decomposition analysis which can assess the effectiveness and reliability of this discrete guidance for the underlying continuous POMDP. Empirical evaluations in patrolling and navigation scenarios show that REBA matches or exceeds all evaluated baselines, with primary metric gains of +17.0% to +47.4% over state-of-the-art approaches.

Xiangwei Chen, Lingling Fang, Andreas Holzinger +1

School of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China · School of Computer Science and Artificial Intelligence, Liaoning Normal University, Dalian 116029, China · Human-Centered AI Lab, Department of Ecosystem Management, Climate and Biodiversity, BOKU University, Vienna, 1190, Austria +1

Belief-Space Planning Partially Observable Markov Decision Processes

Jun 18, 2026cs.RO

UNSEEN: Uncertainty-aware Navigation via Sparse Estimation in Unknown Environments

Visual navigation in unknown environments remains a core challenge in mobile robotics, especially for resource-constrained platforms. Most existing approaches rely on loosely coupled modular pipelines and strong assumptions on perception quality or environmental structure, often resorting to multi-modal sensor suites that increase system complexity and deployment cost. Vision-only navigation offers a lightweight alternative, but its performance degrades severely under motion blur, low texture, and illumination changes, largely because they neglect the tight coupling between commanded motion and perception. While perception-aware methods partially address this issue, they typically optimize individual modules and fail to propagate uncertainty consistently across the navigation stack. In this paper, we present UNSEEN, a unified uncertainty- and perception-aware navigation framework that explicitly couples localization, mapping, and planning using only a front-mounted camera. UNSEEN estimates sparse maps and robot poses with associated uncertainties at 6Hz, and leverages them to plan trajectories that jointly optimize task progress and estimation accuracy in receding-horizon. Simulations and extensive real-world experiments in unknown environments demonstrate the robustness of the proposed approach, with UNSEEN-SLAM reducing absolute translational error by 9.8% and UNSEEN-Plan improving estimation accuracy by up to 45% compared to state-of-the-art methods, while achieving a 100% task success rate.

Tommaso Faraci, Marco Camurri, Daniele Fontanelli +1

University of Trento, Italy

Belief-Space Planning Simultaneous Localization and Mapping Visual Navigation Robot Navigation

Jun 18, 2026cs.RO

VOiLA: Vectorized Online Planning with Learned Diffusion Models for POMDP Agents

Planning under uncertainty is an essential capability for autonomous robots. The Partially Observable Markov Decision Process (POMDP) provides a powerful framework for such a capability. Although POMDP-based planning has advanced significantly, its application to real-world problems is often limited by the difficulty of obtaining faithful POMDP models. We present Vectorized Online planning wIth Learned diffusion model for POMDP Agents (VOiLA), a framework that learns task-agnostic POMDP models for online planning under uncertainty. VOiLA learns transition and observation samplers using conditional diffusion models and learns observation-likelihood models for particle-based belief updates. To enable efficient online planning, the diffusion samplers are distilled into compact feedforward generators and integrated with Vectorized Online POMDP Planner (VOPP), an online POMDP planner designed to leverage GPU parallelization. Experimental results indicate the distillation strategy reduces sampling cost by up to nearly three orders of magnitude, making learned generative POMDP models practical for online planning. Evaluation of VOiLA on three benchmark problems indicate that VOiLA achieves equal or better performance than Recurrent Soft Actor Critic while using less than 10% training data, and generalizes much better to unseen environment configurations. Physical robot evaluation indicates VOiLA uses the models learned using only simulated data and generates a policy that successfully accomplish the task in 10 of 10 runs.

Marcus Hoerger, Rishikesh Joshi, Rahul Shome +2

Australian National University, Canberra, ACT, Australia · The University of Sydney, Sydney, NSW, Australia

Belief-Space Planning Diffusion-Based Planning Model-Based Planning Partially Observable Markov Decision Processes