Belief-Space Planning

Follow this topic Ask about this topic

Belief-space planning studies how to act when an agent cannot directly observe the true state, as in robot navigation, manipulation, search, and autonomous driving. Methods maintain beliefs with Bayesian filters or learned latent, symbolic, and spatial world models, then plan using tools such as Monte Carlo tree search, model-predictive control, and trajectory trees. Many approaches choose actions not only for immediate progress but also for the information they reveal, while risk-sensitive and safety-certified planners constrain possible failures. A central challenge is making these beliefs accurate and decision-relevant without making online planning too costly; learned models can scale to complex settings, but reliable calibration and formal guarantees remain difficult.

Latest papers

65 papers13 papers in the last four weeks, up 18% on the four weeks before. 0.1% of all new papers.

Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration
Mar 4, 2026Danish Rizvi, David BoyleUAV Communication Reinforcement Learning
Imperial College London, UK
Planning under Distribution Shifts with Causal POMDPs
Feb 26, 2026Matteo Ceriscioli, Karthika MohanPartially Observable Markov Decision Processes Belief-Space Planning
Causal Intelligence and Reasoning Lab, Oregon State University, Corvallis, OR 97331, USA
POMDPPlanners: Open-Source Package for POMDP Planning
Feb 24, 2026Yaacov Pariente, Vadim IndelmanPartially Observable Markov Decision Processes Decision-Making under Uncertainty
Faculty of Mathematics, Technion - Israel Institute of Technology, Haifa 3200003, Israel · Stephen B. Klein Faculty of Aerospace Engineering, Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology, Haifa 3200003, Israel
Recursive Belief Vision Language Action Models
Feb 24, 2026Vaidehi Bagaria, Bijo Sebastian, Nirav Kumar PatelVision-Language-Action Models Long-Horizon Robotic Manipulation
Department of Engineering Design, Indian Institute of Technology, Madras, Chennai, India
Multimodal Belief-Space Covariance Steering with Active Probing and Influence for Interactive Driving
Feb 16, 2026Devodita Chakravarty, John Dolan, Yiwei LyuRisk-Sensitive Planning Autonomous Driving Planning
Department of Mechanical Engineering, Indian Institute of Technology Kharagpur, India · Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA

Mar 4, 2026J/K move · Enter open · S save

Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration

Danish Rizvi, David Boyle

Imperial College London, UK

UAV Communication Reinforcement Learning Multi-Robot Exploration Cooperative MARL Belief-Space Planning

Open paper AskPDF arXiv

Coordinating multiple autonomous agents to explore and serve spatially heterogeneous demand requires jointly learning unknown spatial patterns and planning trajectories that maximize task performance. Pure model-based approaches provide structured uncertainty estimates but lack adaptive policy learning, while deep reinforcement learning often suffers from poor sample efficiency when spatial priors are absent. This paper presents a hybrid belief-reinforcement learning (HBRL) framework to address this gap. In the first phase, agents construct spatial beliefs using a Log-Gaussian Cox Process (LGCP) and execute information-driven trajectories guided by a Pathwise Mutual Information (PathMI) planner with multi-step lookahead. In the second phase, trajectory control is transferred to a Soft Actor-Critic (SAC) agent, warm-started through dual-channel knowledge transfer: i) belief state transfer supplies spatial uncertainty, and ii) replay buffer seeding provides demonstration trajectories generated during LGCP exploration. A variance-normalized overlap penalty enables coordinated coverage through shared belief state, permitting cooperative sensing in high-uncertainty regions while discouraging redundant coverage in well-explored areas. The framework is instantiated in two scenarios: multi-Unmanned Aerial Vehicle (UAV) wireless service provisioning, and mobility-on-demand sensing over measured New York City taxi demand. In the wireless scenario it achieves 10.8% higher cumulative reward and takes 38% fewer episodes to reach the reward level of the baselines, with ablation studies confirming that dual-channel transfer outperforms either channel alone; the taxi scenario yields consistent gains with all algorithmic components unchanged.

Latest papers

Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration

Planning under Distribution Shifts with Causal POMDPs

POMDPPlanners: Open-Source Package for POMDP Planning

Recursive Belief Vision Language Action Models

Multimodal Belief-Space Covariance Steering with Active Probing and Influence for Interactive Driving