CAVEAT: Recurrent Multimodal Diffusion Planning for Mapless Aerial Exploration

Steven Visch; Nicolò Botteghi; Antonio Franchi; Barbara Bazzana

cs.ROSep 6, 2026

CAVEAT: Recurrent Multimodal Diffusion Planning for Mapless Aerial Exploration

Authors: Steven Visch, Nicolò Botteghi, Antonio Franchi, Barbara Bazzana

Organizations: Robotics and Mechatronics group, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7500AE Enschede, The Netherlands. · Modelling and Scientific Computing Laboratory (MOX), Politecnico di Milano. · Department of Computer, Control and Management Engineering, Sapienza University of Rome, 00185 Rome, Italy.

Abstract

Can exploratory UAV waypoint sequences be generated from multimodal onboard observations and a fixed-dimensional recurrent internal state without maintaining a persistent global map in the deployed policy? We investigate this question through CAVEAT, a diffusion policy conditioned on a recurrent internal state updated from fused LiDAR, visual, and pose features and trained from trajectories generated by the map-based FUELv2 expert. Rolling inference partially warm-starts consecutive predictions, while a temporary local signed distance field provides heuristic obstacle guidance. Simulation results evaluate both inference mechanisms and compare CAVEAT with its demonstration-generating expert. Proof-of-concept experiments on a Flyability Elios 3 demonstrate partial exploration of a previously unseen indoor environment and target-directed visual servoing using a separately trained policy.

Figures & tables

Fig. 1 : Overview of CAVEAT. Multimodal observations update a recurrent internal state that conditions diffusion-based waypoint generation. Rolling inference partially warm-starts consecutive predictions, while a temporary local SDF provides heuristic obstacle guidance.

Fig. 2 : Architecture of the CAVEAT exploration policy. Modality-specific encoders process the current LiDAR, visual, and pose observations. Their fused features update a recurrent internal state that conditions diffusion-based waypoint generation through feature-wise linear modulation.

Panel of Figure 3 — Fig. 3 : Gazebo environment families used to collect the expert exploration demonstrations. Environments $A_{\mathrm{train}}$ – $D_{\mathrm{train}}$ correspond to panels (a)–(d), respectively.

Panel of Figure 4 — Fig. 4 : Gazebo test environments. Environments $A_{\mathrm{test}}$ – $B_{\mathrm{test}}$ correspond to panels (a)–(b), respectively.

Trajectory statistic	Steps	Environment	Trajectories
Mean	74.65	$A_{\mathrm{train}}$	1000
Median	79.00	$B_{\mathrm{train}}$	700
Standard deviation	22.64	$C_{\mathrm{train}}$	400
Minimum	1	$D_{\mathrm{train}}$	574
Maximum	114	Total	2674

TABLE I : Exploration-demonstration dataset, comprising 2,674 trajectories and 199,603 recorded time steps.

Panel of Figure 6 — Fig. 5 : Occupancy coverage versus distance traveled in simulation. Panels (a)–(b) compare independent and rolling initialization; panels (c)–(d) compare SDF-guidance thresholds $d_{\min}=0.5\,\mathrm{m}$ and $d_{\min}=1.0\,\mathrm{m}$ ; panels (e)–(f) compare CAVEAT and FUELv2. Curves represent the average exploration trajectory across three to four runs; shaded regions represent $\pm 1$ standard deviation across the evaluation trajectories.

Fig. 6 : Comparison of CAVEAT and FUELv2, over the training environment $B_{\mathrm{train}}$ .

Panel of Figure 8 — Fig. 7 : Two views of the previously unseen real-world test environment.

Panel of Figure 9 — Fig. 8 : Real-world exploration trajectories obtained with DDPM and DDIM sampling. Black lines indicate the principal environment boundaries, colored curves show the UAV trajectories, triangles indicate heading, and green and red markers denote the start and end positions.

Fig. 9 : Target-directed visual-servoing proof of concept. Three trajectories approach the wall-mounted marker from different initial positions. Triangles indicate UAV heading, black points show the LiDAR-observed wall, and the inset images show representative camera observations.

UAV Navigation

Unmanned aerial vehicle (UAV) navigation develops methods for locating and guiding drones safely through known, unknown, and changing environments.

See all 273 papers

WOLF: World Model Guided LiDAR Exploration with Predictive Frontiers
Sep 20, 2026Yuyang Tian, Penghui Yang, Pengyuan Wu +7World Model Learning UAV Navigation
University of Science and Technology of China, Hefei, China. · Shanghai Artificial Intelligence Laboratory, Shanghai, China. · East China University of Science and Technology, Shanghai, China. +3
Guided Diffusion with Distilled Vision-Language Reliability for Aerial Navigation
Jun 11, 2026Ivan Valuev, Iana Zhura, Valerii Serpiva +2Diffusion-Based Planning Robot Navigation
AgenticDiffusion: Multi-View Reasoning with View-Conditioned Diffusion Planning for Vision-Based UAV Navigation
Jun 2, 2026Faryal Batool, Muhammad Ahsan Mustafa, Fawad Mehboob +2Aerial Robotics Trajectory Generation

Sep 20, 2026J/K move · Enter open · S save

WOLF: World Model Guided LiDAR Exploration with Predictive Frontiers

Yuyang Tian, Penghui Yang, Pengyuan Wu, Haoran Yang+6

University of Science and Technology of China, Hefei, China. · Shanghai Artificial Intelligence Laboratory, Shanghai, China. · East China University of Science and Technology, Shanghai, China.+3

World Model Learning UAV Navigation Frontier-Based Exploration

Open paper AskPDF arXiv

LiDAR-based unmanned aerial vehicle (UAV) exploration builds maps by continually selecting where to observe next. However, decisions based on the measured map provide limited foresight into spatial continuations behind occlusions, leaving potentially informative directions unrecognized. We present WOLF, a world-model-guided framework that predicts future observations to enhance autonomous exploration. In the training stage, a recurrent world model learns observation dynamics from exploration trajectories, with recurrent memory retaining the spatial context needed to interpret partial observations across successive views. Building on this context, the model combines observation history with candidate motions during exploration to predict local occupancy and visibility. To guide further sensing, a predictive frontier generation mechanism then aligns and fuses these predictions using confidence, branch agreement, and observation quality to identify promising regions. The resulting predictive frontiers join measured ones to guide geometric viewpoint selection and trajectory generation, while new scans update subsequent predictions. In simulations, our method reduces mean terminal time by 10.9% relative to EPIC in Garage at comparable coverage and increases mean coverage from 42.12% to 98.35% in Tunnel. Real-world experiments further demonstrate onboard deployment of the learned model for online inference during physical flight.

Abstract

UAV Navigation

Explore similar work

WOLF: World Model Guided LiDAR Exploration with Predictive Frontiers

Guided Diffusion with Distilled Vision-Language Reliability for Aerial Navigation

AgenticDiffusion: Multi-View Reasoning with View-Conditioned Diffusion Planning for Vision-Based UAV Navigation