DroneWAM: Efficient World Action Model for Drone Visual Navigation

Liang Yao; Fan Liu; Hongbo Lu; Wei Xu; Jianyu Jiang; Yijun Shen; Chuanyi Zhang; Pai Peng

cs.CVSep 27, 2026

DroneWAM: Efficient World Action Model for Drone Visual Navigation

Authors: Liang Yao, Fan Liu, Hongbo Lu, Wei Xu, Jianyu Jiang, Yijun Shen, Chuanyi Zhang, Pai Peng

Organizations: Hohai University · Key Laboratory of Water Big Data Technology of Ministry of Water Resources · Shanghai Jiao Tong University · CowaRobot Co. Ltd. ·

Abstract

World-action models give visual navigation agents a way to anticipate how candidate actions will change future observations and to act from the predicted consequences. For drones, this capability must operate under tight accuracy and efficiency constraints. We present DroneWAM, an efficient world-action model for drone visual navigation. DroneWAM adopts a JEPA-based architecture to model future states directly in representation space, avoiding the cost of explicit future image generation. A pretrained Resampler further compresses dense encoder features into fewer latent tokens, reducing the computation repeated at each imagined step. We also introduce adaptive rollout, where a preference-trained Gate adaptively allocates prediction depth according to the current scene. To support learning under richer aerial motion, we construct DroneNav-6D, a simulated visual navigation dataset with synchronized RGB observations, 6-DoF flight trajectories, control commands, and randomized wind disturbances. On DroneNav-6D, DroneWAM achieves the best trajectory accuracy among the compared methods. Adaptive rollout further reduces the average prediction depth from 8 to 4.58 while improving trajectory accuracy, demonstrating that predictive computation can be allocated more effectively across scenes. Codes and data will be released.

Figures & tables

Figure 1: Compared to prior approaches.

Figure 2: Examples of goal-conditioned navigation samples in DroneNav-6D. Each row shows the flight trajectory together with the sampled goal image and current observation.

Figure 3: Overview of DroneWAM and DroneNav-6D. (a) DroneNav-6D provides diverse simulated environments with synchronized visual observations and 6-DoF flight trajectories. (b) DroneWAM is trained in three stages: Resampler pretraining, world-action model training with latent prediction and planning objectives, and preference learning for the Gate. (c) During inference, the Predictor and Planner iteratively imagine future latent states and action plans, while the Gate determines when to stop and returns the highest-scoring plan for efficiency.

Method	ATE $\downarrow$	Rel.ATE $\downarrow$	RPE $\downarrow$	Rel.RPE $\downarrow$	Infer. (ms) $\downarrow$
DINO-WM ( Zhou et al., 2024 )	9.0923	0.4459	3.0681	0.1502	29,622.24
V-JEPA 2-AC ( Assran et al., 2025 )	27.4317	0.6621	9.0439	0.2190	64,617.72
LeWM ( Maes et al., 2026 )	4.2259	0.4164	1.0924	0.1080	654.37
FastWAM ( Yuan et al., 2026 )	1.8366	0.1919	0.5086	0.0427	712.33
NWM ( Bar et al., 2025 )	3.7580	0.3740	0.8051	0.0802	403,827.78
Ours	1.1062	0.0578	0.3470	0.0180	483.29

Table 1: Open-loop evaluation on DroneNav-6D.

Method	ATE $\downarrow$	Rel.ATE $\downarrow$	RPE $\downarrow$	Rel.RPE $\downarrow$	Prog. $\uparrow$	MSE $\downarrow$	Infer. (ms) $\downarrow$
FastWAM ( Yuan et al., 2026 )	119.8063	0.7410	2.7309	0.0176	0.510	124.30	712.33
NWM ( Bar et al., 2025 )	88.6305	0.5980	2.7245	0.0171	0.500	178.55	403,827.78
Ours	83.4461	0.5184	2.7194	0.0168	0.774	91.96	483.29

Table 2: Closed-loop navigation results on DroneNav-6D.

PLA	RS	AR	Rel.ATE $\downarrow$	Rel.RPE $\downarrow$	Infer. (ms) $\downarrow$
			0.6621	0.2190	64,617.72
✓			0.0584	0.0189	656.12
✓	✓		0.0603	0.0192	582.65
✓	✓	✓	0.0578	0.0180	483.29

Table 3: Component ablation. PLA, RS, and AR denote Planner, Resampler, and Adaptive Rollout, respectively.

Method	Rel.ATE $\downarrow$	Rel.RPE $\downarrow$	Avg. $K$ $\downarrow$
Fixed	0.0688	0.0221	8.000
Random	0.0722	0.0224	4.581
Latent Margin	0.0887	0.0309	5.625
Ours	0.0578	0.0180	4.578

Table 4: Comparison of rollout strategies. Avg.

K

denotes the average rollout depth.

#Tok.	Rel.ATE $\downarrow$	Rel.RPE $\downarrow$	Infer. (ms) $\downarrow$
512	0.0574	0.0179	591.28
128	0.0578	0.0180	483.29
16	0.0659	0.0198	405.16

Table 5: Effect of Resampler size. #Tok. denotes the number of output latent tokens.

Figure 4: Accuracy at different rollout depths. Samples are grouped by the rollout depth, and red boxes indicate the corresponding selected depth.

Method	N.RMSE $\downarrow$	Grip.(%) $\uparrow$	Latency $\downarrow$
$K=8$	0.2685	89.83	890.8
Adaptive	0.2652	90.35	411.9

Table 6: Open-loop results on LIBERO. Full action metrics are reported in the Appendix.

Figure 5: Generalization of the preference-trained Gate across unseen environments. Seen and unseen results are compared under environment-level cross-validation.

Figure 6: Qualitative comparison of predicted trajectories across diverse aerial scenes. Green, yellow, and red denote the ground-truth trajectory, FastWAM, and DroneWAM, respectively. DroneWAM consistently follows the reference trajectory more closely, while FastWAM exhibits larger directional and accumulated drift.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Split	Clips	Frames	Hours	Distance (km)	20-frame windows
Training	3,453	1,427,016	56.49	1,795.42	1,361,409
Validation	384	165,154	6.54	208.69	157,858
Total	3,837	1,592,170	63.03	2,004.11	1,519,267

Appendix

Table 7: Size of DroneNav-6D. Window counts enumerate overlapping sequences of 20 consecutive frames within individual clips. Flight duration and distance are summed over clips.

Figure 7: Dataset composition across the five worlds. The panels show (a) training and validation clip counts, (b) recorded flight duration, and (c) the distribution of individual clip durations. CityBIM, Rural, Roads, and Snow denote CITYBIM, RuralAustralia, SnappyRoads, and Snowy Mountain, respectively. These abbreviations are used throughout the dataset figures.

Component	Unit	P10	Median	P90
$\Delta x$	m/frame	0.408	1.138	1.346
$\Delta y$	m/frame	$-0.943$	0.000	0.945
$\Delta z$	m/frame	$-0.199$	0.000	0.193
$\Delta\phi$ (roll)	deg/frame	$-1.395$	0.000	1.392
$\Delta\theta$ (pitch)	deg/frame	$-1.252$	0.011	1.376
$\Delta\psi$ (yaw)	deg/frame	$-4.406$	0.000	4.376

Appendix

Table 8: Distribution of pose increments between consecutive frames. Translation uses yaw-aligned horizontal axes and the NED vertical axis; angular increments are wrapped Euler-angle differences. P10 and P90 denote the 10th and 90th percentiles. Values are rounded to three decimal places.

Figure 8: Distributions of six-dimensional pose increments between consecutive frames, shown separately for each world using clip-balanced samples. The top row shows translation in meters per frame, and the bottom row shows roll, pitch, and yaw changes in degrees per frame. Violin plots depict the distributions; boxes indicate the interquartile range, central lines mark the medians, and whiskers span the 5th–95th percentiles.

World	Clips	Frames	Duration (s)	Speed (m/s)
CITYBIM	943	373,702	58.44	9.357
NYC	1,159	603,705	66.57	9.298
RuralAustralia	518	185,346	53.14	9.319
SnappyRoads	611	217,483	53.58	9.329
Snowy Mountain	606	211,934	53.16	9.325

Appendix

Table 9: Per-world data and motion statistics. Duration is the median clip duration; speed is the per-world median in the dataset statistics.

Figure 9: Trajectory characteristics across worlds. Violin and box plots summarize (a) clip duration, (b) three-dimensional path length, (c) cumulative vertical travel, and (d) the 95th-percentile linear speed within each clip.

Figure 10: Motion over prediction horizons of one to eight frame intervals. The panels show translational displacement and quaternion-based geodesic rotation. Colored curves are per-world medians; the black curve is the pooled median, and the shaded region spans the pooled 10th–90th percentiles.

Figure 11: Translation and rotation in the recorded motion. (a) Spearman correlations among the six pose-increment components. (b) Each point shows the 95th-percentile linear and angular speeds of one clip, colored by world. The vertical axis uses a logarithmic scale.

Figure 12: Visual features of 1,000 sampled RGB observations. Each world contributes 100 training and 100 validation images. Frozen ImageNet-pretrained AlexNet fc7 features are projected with PCA to 50 dimensions and then with t-SNE to two dimensions. The same coordinates are colored by (a) world and (b) split. The reported trustworthiness at 10 neighbors is 0.990; the visualization characterizes local feature neighborhoods.

Method	Success $\uparrow$	rel.ATE $\downarrow$	rel.RPE $\downarrow$	Task Progress $\uparrow$	Latency (ms) $\downarrow$
NWM	74%	0.31	0.012	0.79	$1.3\times 10^{5}$
FastWAM	72%	0.38	0.014	0.76	700ms
DroneWAM	88%	0.15	0.008	0.95	460ms

Appendix

Table 10: Global visual closed-loop results on easy routes.

Case	Gate depth	D1 rel.ATE	Gate rel.ATE	D8 rel.ATE	D1 rel.RPE	Gate rel.RPE	D8 rel.RPE
Case 1	4	0.1879	0.1083	0.1064	0.0627	0.0366	0.0322
Case 2	1	0.0142	0.0142	0.0192	0.0064	0.0064	0.0073
Case 3	2	0.0543	0.0423	0.0514	0.0164	0.0127	0.0150
Case 4	1	0.2160	0.2160	0.1285	0.0924	0.0924	0.0668

Appendix

Table 11: Case studies at different rollout depths. Each row evaluates the same sample at depth 1, the Gate-selected depth, and depth 8.

Method	Mean depth	Norm. RMSE $\downarrow$	Norm. L1 $\downarrow$	EEF RMSE $\downarrow$	EEF L1 $\downarrow$	Gripper acc. $\uparrow$	Latency (ms) $\downarrow$
Base WAM	–	0.3636	0.2466	0.2647	0.1631	85.42%	957.1
Fixed D1	1.00	0.2735	0.1874	0.1932	0.1232	89.79%	129.7
Fixed D2	2.00	0.2680	0.1844	0.1906	0.1218	90.21%	237.7
Fixed D4	4.00	0.2667	0.1839	0.1890	0.1212	90.35%	451.3
Fixed D8	8.00	0.2685	0.1848	0.1905	0.1223	89.83%	890.8
Ours	3.56	0.2652	0.1825	0.1881	0.1206	90.55%	411.9

Appendix

Table 12: Full open-loop action-prediction results.

Suite	Episodes	Base WAM $\downarrow$	Fixed D8 $\downarrow$	Ours $\downarrow$	Mean depth
Spatial	44	0.3479	0.2321	0.2275	3.43
Object	46	0.2609	0.1857	0.1831	4.02
Goal	44	0.4461	0.3478	0.3441	3.09
Long	39	0.4094	0.3177	0.3156	3.69

Appendix

Table 13: Normalized action RMSE by LIBERO suite.

Method	Spatial	Object	Goal	Long	Avg.
DroneWAM	52.26	64.32	54.27	10.08	45.23
Improved-DroneWAM	62.89	90.22	70.22	16.44	59.94

Appendix

Table 14: Closed-loop success rates (%) on LIBERO.

UAV Navigation

Unmanned aerial vehicle (UAV) navigation develops methods for locating and guiding drones safely through known, unknown, and changing environments.

See all 272 papers

DiffWAM: A Fast and Efficient Navigation World Action Model
Sep 30, 2026Mo Zhu, Yuze Wu, Xijie Huang +3Trajectory Generation World Models for Robotics
Zhejiang University · Differential Robotics
NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation
Jun 11, 2026Daichi Azuma, Taiki Miyanishi, Koya Sakamoto +6World Models for Robotics Robot Navigation
The University of Tokyo · National Institute of Informatics · AIRoA +1
RAE-NWM: Navigation World Model in Dense Visual Representation Space
Mar 10, 2026Mingkun Zhang, Wangtian Shen, Fan Zhang +3Visual Navigation World Model Learning
Department of Precision Instrument, Tsinghua University, Beijing, China · University of Rochester, Rochester, NY, USA · Beijing Information Science and Technology University, Beijing, China

Sep 30, 2026J/K move · Enter open · S save

DiffWAM: A Fast and Efficient Navigation World Action Model

Mo Zhu, Yuze Wu, Xijie Huang, Xiao Cui+2

Zhejiang University · Differential Robotics

Trajectory Generation World Models for Robotics UAV Navigation Robot Motion Generation

Open paper AskPDF arXiv

Pretrained video foundation models encode rich semantic and spatiotemporal priors for embodied navigation, yet converting these priors into UAV motion typically requires expensive future-video synthesis and geometric reconstruction. We investigate whether the motion implicit in future visual prediction can instead be recovered directly from the predictive representations of a frozen video model. To this end, we present DiffWAM, a geometry-conditioned navigation world-action model that directly transforms multi-level predictive features into continuous camera trajectories. Its Grid-Motion module preserves spatial-temporal motion associations, while Latent2Pose grounds them with first-frame geometry to recover metrically meaningful 3D motion. Complete video rollouts and geometric reconstruction are required only for offline supervision, eliminating future-video decoding and multi-frame reconstruction during deployment. We further introduce FastDreamer, which overlaps predictive and geometric computation with ongoing flight and performs timestamp-aware asynchronous trajectory handoff for continuous UAV execution. DiffWAM achieves a trajectory RMSE of 0.3492 m and an endpoint success rate of 74.40% on the 1,000-sample DiffWAM-1000 benchmark, while representative real-world experiments demonstrate complex behaviors including constrained traversal, orbiting, S-shaped flight, and multi-stage navigation. An onboard DiffWAM-Flash implementation further reaches 1.08 s model-pipeline latency on NVIDIA Jetson AGX Thor. These results demonstrate that predictive video representations can be efficiently grounded into continuous 3D motion, providing a direct alternative to generate-then-reconstruct navigation pipelines. Project page: https://zzmmzzm.github.io/diffwam.github.io/.

Abstract

UAV Navigation

Explore similar work

DiffWAM: A Fast and Efficient Navigation World Action Model

NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation

RAE-NWM: Navigation World Model in Dense Visual Representation Space