Learning from Hallucinating Critical Points for Navigation in Dynamic Environments

Saad Abdul Ghani; Kameron Lee; Xuesu Xiao

cs.ROSep 30, 2025

Learning from Hallucinating Critical Points for Navigation in Dynamic Environments

Authors: Saad Abdul Ghani, Kameron Lee, Xuesu Xiao

Organizations: George Mason University · Thomas Jefferson High school intern affiliated with the RobotiXX Lab, George Mason University

Abstract

Generating large and diverse obstacle datasets to learn motion planning in environments with dynamic obstacles is challenging due to the vast space of possible obstacle trajectories. Inspired by hallucination-based data synthesis approaches, we propose Learning from Hallucinating Critical Points (LfH-CP), a self-supervised framework for creating rich dynamic obstacle datasets based on existing optimal motion plans without requiring expensive expert demonstrations or trial-and-error exploration. LfH-CP factorizes hallucination into two stages: first identifying when and where obstacles must appear in order to result in a near-optimal motion plan, i.e., the critical points, and then procedurally generating diverse trajectories that pass through these points while avoiding collisions. This factorization avoids generative failures such as mode collapse and ensures coverage of diverse dynamic behaviors. We further introduce a diversity metric to quantify dataset richness and show that LfH-CP produces substantially more varied training data than existing baseline. Experiments in simulation demonstrate that planners trained on a LfH-CP generated dataset achieves higher success rates compared to a prior hallucination method.

Figures & tables

Figure 1 : A single hallucinated critical point (dotted circle) — a location and time at which an obstacle must appear for the robot’s plan (black) to be near-optimal. Any obstacle trajectory passing through that point (orange arrows) leaves the same plan valid, so one critical point yields many distinct training scenarios.

Panel of Figure 2 — Figure 2 : 100 obstacle trajectories are generated from each critical point. The robot trajectory (green) remains collision-free and near-optimal, regardless of the variations and combinations of obstacle trajectories. That is, the same robot trajectory is near-optimal and collision-free for any single combination of these $100^{N}$ possible obstacle trajectory sets. Fig. 2(a) shows a total of 200 obstacle trajectories generated from $N=2$ critical points, while Fig. 2(b) shows a total of 300 obstacle trajectories generated from $N=3$ critical points.

Panel of Figure 3 — Figure 3 : The LfH-CP method visualized for motion plan (green) and one obstacle (red). We first learn where the obstacle should be to make our plan optimal using Eqn. 6 (Fig. 3(a) ). Then using Eqn. 9 , we learn where the obstacle should be temporally to make the plan near-optimal (Fig. 3(b) , 3(c) ). Finally using Eqn. 10 , numerous obstacle trajectories can be generated from the obstacle’s critical point (Fig. 3(d) ).

Figure 4 : A. The encoder-decoder architecture learns the critical point configuration $\mathcal{K}^{*}$ over two phases. Phase 1 ( where ) learns a distribution over obstacle locations $\{\mathbf{o}_{i}\}^{N}_{i=1}$ that make $\mathbf{p}\in\mathcal{P}$ optimal. Phase 2 ( when ) additionally learns a temporal mask per obstacle via Gumbel–Softmax, whose peak gives the critical time step $t^{*}_{i}$ . B. The generation function $g(\cdot)$ generates $S_{2}$ collision-free obstacle trajectories per critical point over horizon $H$ . C. A motion planner $f_{\theta}(\cdot)$ is trained on $\mathcal{D}_{\mathrm{train}}$ : $M_{l}$ past LiDAR scans paired with the motion plans $\mathbf{p}$ .

Figure 5 : Reconstructed trajectories from Phase 1 (P1) and Phase 2 (P2) using obstacles at their critical points. Trajectories remain visually close to each other, with only minor variations in reconstruction loss.

Component (unit)	Lower bound	Upper bound	Resolution
$r$ (m)	0.15	2	0.1
$\beta$ (deg)	-180	180	5
$v_{\mathrm{o}}$ (m/s)	1	2	0.1
$\phi_{\mathrm{o}}$ (deg)	-180	180	5

Table I : Bounds and Resolution for Dataset Coverage.

Component $\mathcal{S}$	Coverage LfH-CP(%)	Coverage Dyna-LfLH (%)
$r,\beta,v_{\mathrm{o}},\phi_{\mathrm{o}}$	62.21	11.05
$r,\beta,v_{\mathrm{o}}$	100.0	74.61
$\beta,v_{\mathrm{o}},\phi_{\mathrm{o}}$	99.18	49.30
$r,v_{\mathrm{o}},\phi_{\mathrm{o}}$	100.0	69.6
$r,\beta,\phi_{\mathrm{o}}$	93.20	40.49
$\beta,v_{\mathrm{o}}$	100.0	99.13

Table II : DCS for all component combinations for our LfH-CP and comparative approach.

Figure 6 : DCS increases as the number of samples per critical point,

S_{2}

, increases.

Difficulty	# Worlds	Number of obstacles	Range of speed
Easy	20	[5, 10]	[0.5, 1.0]
Medium	10	[5, 10]	[1.0, 2.0]
Medium	10	[10, 20]	[0.5, 1.0]
Hard	20	[10, 20]	[1.0, 2.0]

Table III : DynaBARN Parameters.

Figure 7 : Simulation Results of 120 trials in DynaBARN.

Explore similar work

Learning-Based Motion Planning for Dynamic Environments: From Foundational Algorithms to Emerging Paradigms
Aug 1, 2026Zongyuan Shen, Shalabh Gupta, Shancheng Zhao +7Motion Planning Dynamic Environments
College of Information Science and Technology, Jinan University, Guangzhou 510632, China. · Department of Electrical and Computer Engineering, University of Connecticut, Storrs, CT 06269, USA. · Guangzhou Maritime University, Guangzhou 510725, China. +4
Sum of Costs Diffusion with Dynamic Guidance for Motion Planning
May 23, 2026Aysu Aylin Kaplan, Özgür ErkentCollision-Free Trajectories Motion Planning
Computer Engineering Department, Hacettepe University, Ankara, Turkey
Time-Efficient Iterative Learning Planning for Safety-Critical Dynamic Obstacle Avoidance
Sep 17, 2026Zhiyi Chen, Shuli Lv, Chen Min +3Obstacle Avoidance Control Barrier Functions
National Key Laboratory of Autonomous Intelligent Unmanned Systems, School of Automation, Beijing Institute of Technology, Beijing 100081, China · School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China

Aug 1, 2026, cs.ROJ/K move · Enter open · S save

Learning-Based Motion Planning for Dynamic Environments: From Foundational Algorithms to Emerging Paradigms

Zongyuan Shen, Shalabh Gupta, Shancheng Zhao, Dehua Zhou+6

College of Information Science and Technology, Jinan University, Guangzhou 510632, China. · Department of Electrical and Computer Engineering, University of Connecticut, Storrs, CT 06269, USA. · Guangzhou Maritime University, Guangzhou 510725, China.+4

Motion Planning Dynamic Environments Planning Classical Planning Human-Robot Collaboration

Open paper AskPDF arXiv

Motion planning in dynamic environments is a fundamental problem in robotics, aiming to generate safe and efficient paths, trajectories, or control actions in the presence of moving obstacles, uncertain predictions, and multi-agent interactions. It has broad applications in autonomous driving, service robotics, warehouse logistics, human-robot collaboration, crowd navigation, and multi-robot systems. This survey reviews representative works published primarily between 2015 and 2025, with a particular focus on how recent learning-based advances extend, complement, or interact with classical planning foundations. We first revisit classical planning methods as algorithmic foundations and reference frameworks for learning-based extensions. We then propose a role-of-learning taxonomy that categorizes existing methods according to how learning participates in the planning pipeline, including direct policy learning, learning-augmented classical planning, hybrid planning, and training enhancement methods. For each category, we summarize the main problem settings, representative algorithms, key ideas, integration mechanisms, strengths, and limitations. We further analyze how observation representations, prediction uncertainty, interaction modeling, planner integration, safety constraints, and training strategies shape learning-based motion planning in dynamic environments. Finally, we discuss open challenges and future directions, including sim-to-real gap, safe and certifiable planning, dense crowd navigation, perception-planning coupling, and embodied AI.

Abstract

Explore similar work

Learning-Based Motion Planning for Dynamic Environments: From Foundational Algorithms to Emerging Paradigms

Sum of Costs Diffusion with Dynamic Guidance for Motion Planning

Time-Efficient Iterative Learning Planning for Safety-Critical Dynamic Obstacle Avoidance