Organizations: State Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China
Agile flight in unknown, cluttered, and dynamic environments requires a planner that knows where and when danger will appear, not only that a trajectory is dangerous. One-stage learning-based planners trained with differentiable privileged costs are fast and expert-free, but the only signal reaching their encoder is a scalar trajectory cost with no spatial or temporal structure, so avoidance degrades into late reactive maneuvers. We present RiskFly, a one-stage planner that predicts risk in the same space in which it acts. A dual-stream observation pairs a short depth sequence with a frustum-aligned inverted spherical range-map sequence, whose angular cells match the end-state proposals one to one. An auxiliary head regresses a frustum-aligned spatio-temporal risk field, supervised by a privileged closest-point-of-approach (CPA) target. This self-predicted field is queried differentiably along the instantiated quintic trajectory at its own arrival times, and also enters the training objective, so representation supervision and planning gradients meet in a single space. Privileged signals are discarded at deployment, and the planner runs map-free from onboard depth and proprioception. Extensive simulation and zero-shot real-world flights on resource-constrained platforms show higher success rates at comparable end-to-end latency.
Figures & tables
Fig. 1: Snapshot of real-world flight experiments. Zero-shot sim-to-real deployment of the proposed one-stage planner in a previously unseen forest with two walking pedestrians, using only onboard depth and proprioception. White arrows give the flight direction and the red ellipse marks the quadrotor; the bottom strip shows the onboard depth frames 1 – 4 taken at the instants the dashed lines indicate.
Fig. 2: System overview. A depth sequence Dt , a frustum-aligned range-map sequence Lt , the proprioceptive state ot and the goal direction are encoded on the shared V×H frustum grid: a CoAtNet branch and a range-map CNN are concatenated, fused with the state by a 1×1 convolution, and aggregated by a per-cell TCN into G . A risk head regresses the spatio-temporal risk field R^∈[0,1]V×H×B indexed by bearing and arrival time, and a planning head regresses one scored terminal-state proposal per cell. During training, the ESDF and ground-truth actor states supply the CPA target for Lrf and the trajectory costs, while R^ is queried along the instantiated trajectory at its own arrival times as Jrf . At inference the privileged modules are detached and rˉij modulates proposal selection.
Fig. 3: Construction of the frustum-aligned range map. Min-pooled depth is spherically projected onto Nh×Nv angular bins that subdivide the planning frustum, and optical-axis depths are converted to Euclidean ranges and inverted so that proximity yields a high response. Average pooling with (Nv,Nh)=(αV,αH) maps the result exactly onto the V×H grid, and the Lr most recent maps are stacked into Lt .
Fig. 4: Spatio-temporal risk field and arrival-time query. (a) The CPA of the relative motion defines t^k and dk∗ . (b) A bivariate hazard maps (t^k,dk∗/rsafe) to a continuous risk score. (c) Hazards are rasterized onto the frustum-time grid Rgt∈[0,1]V×H×B , indexed by bearing and arrival time. (d) The instantiated trajectory queries the predicted field at its own bearing and arrival time, so the cost is evaluated where and when the vehicle will actually be.
Fig. 5: Simulation environments and performance of our method. Four 30×60m scenarios keep the obstacle count comparable while shifting the static-to-dynamic ratio: (a) 90 Static, (b) 40 Static & 20 Dynamic, (c) 20 Static & 40 Dynamic, (d) 60 Dynamic. Trajectories are color-coded by UAV speed ( m/s ), with a scale that differs across scenarios. Static obstacles are magenta; dynamic obstacles are red and move randomly at [1,4]m/s .
TABLE I: Baseline Comparison in Cluttered Environments with Different Densities of Static and Dynamic Obstacles
Fig. 6: Generalization to unseen environments. Three held-out environments, (a) Inclines, (b) Cave, (c) Forest, differ from the training scenes in geometry, obstacle shape, and free-space topology. Top: success rate over 50 trials against the achieved average speed, obtained by sweeping the reference speed from 1.0 to 7.0m/s . Bottom: trajectories of all five methods for one start–goal pair; grey dot is the start, yellow star the goal, dashed circles mark collisions. Dynamic obstacles are red, static obstacles blue to magenta.
TABLE II: Ablation study on the explicit range-map stream, the spatio-temporal risk field, and the arrival-time risk coupling
Fig. 7: Real-world experiments. (a) Our quadrotor platform: a Rockchip RK3588 onboard computer, an Intel RealSense D435i, a CUAV V5+ flight controller, the motors, and the battery. (b)–(d) Static flights through a dense forest, with (d) crossing a sloped section of irregular trunk spacing. (e)–(g) Dynamic flights in the same forest with two walking pedestrians; the red ellipse in (g) marks the quadrotor. Every sequence is composited from a fixed viewpoint, so the repeated silhouettes trace the executed trajectory.
System Modules
RTX 3090 Ti (ms)
RK3588 (ms)
Orin NX (ms)
Depth Processing
1.02
1.71
1.19
Frustum Range Map
0.58
1.09
0.79
Data Preparation
0.55
1.26
0.82
Network Inference
4.81
9.02
5.94
Proposal Selection
0.41
0.91
0.57
Total Latency
7.37
13.99
9.31
TABLE III: Latency of each component in the proposed system
National Key Laboratory of Science and Technology on Multi-spectral Information Processing, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China