Organizations: State Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China
Agile flight in unknown, cluttered, and dynamic environments requires a planner that knows where and when danger will appear, not only that a trajectory is dangerous. One-stage learning-based planners trained with differentiable privileged costs are fast and expert-free, but the only signal reaching their encoder is a scalar trajectory cost with no spatial or temporal structure, so avoidance degrades into late reactive maneuvers. We present RiskFly, a one-stage planner that predicts risk in the same space in which it acts. A dual-stream observation pairs a short depth sequence with a frustum-aligned inverted spherical range-map sequence, whose angular cells match the end-state proposals one to one. An auxiliary head regresses a frustum-aligned spatio-temporal risk field, supervised by a privileged closest-point-of-approach (CPA) target. This self-predicted field is queried differentiably along the instantiated quintic trajectory at its own arrival times, and also enters the training objective, so representation supervision and planning gradients meet in a single space. Privileged signals are discarded at deployment, and the planner runs map-free from onboard depth and proprioception. Extensive simulation and zero-shot real-world flights on resource-constrained platforms show higher success rates at comparable end-to-end latency.
Figures & tables
Fig. 1: Snapshot of real-world flight experiments. Zero-shot sim-to-real deployment of the proposed one-stage planner in a previously unseen forest with two walking pedestrians, using only onboard depth and proprioception. White arrows give the flight direction and the red ellipse marks the quadrotor; the bottom strip shows the onboard depth frames 1 – 4 taken at the instants the dashed lines indicate.
Fig. 2: System overview. A depth sequence Dt , a frustum-aligned range-map sequence Lt , the proprioceptive state ot and the goal direction are encoded on the shared V×H frustum grid: a CoAtNet branch and a range-map CNN are concatenated, fused with the state by a 1×1 convolution, and aggregated by a per-cell TCN into G . A risk head regresses the spatio-temporal risk field R^∈[0,1]V×H×B indexed by bearing and arrival time, and a planning head regresses one scored terminal-state proposal per cell. During training, the ESDF and ground-truth actor states supply the CPA target for Lrf and the trajectory costs, while R^ is queried along the instantiated trajectory at its own arrival times as Jrf . At inference the privileged modules are detached and rˉij modulates proposal selection.
Fig. 3: Construction of the frustum-aligned range map. Min-pooled depth is spherically projected onto Nh×Nv angular bins that subdivide the planning frustum, and optical-axis depths are converted to Euclidean ranges and inverted so that proximity yields a high response. Average pooling with (Nv,Nh)=(αV,αH) maps the result exactly onto the V×H grid, and the Lr most recent maps are stacked into Lt .
Fig. 4: Spatio-temporal risk field and arrival-time query. (a) The CPA of the relative motion defines t^k and dk∗ . (b) A bivariate hazard maps (t^k,dk∗/rsafe) to a continuous risk score. (c) Hazards are rasterized onto the frustum-time grid Rgt∈[0,1]V×H×B , indexed by bearing and arrival time. (d) The instantiated trajectory queries the predicted field at its own bearing and arrival time, so the cost is evaluated where and when the vehicle will actually be.
Fig. 5: Simulation environments and performance of our method. Four 30×60m scenarios keep the obstacle count comparable while shifting the static-to-dynamic ratio: (a) 90 Static, (b) 40 Static & 20 Dynamic, (c) 20 Static & 40 Dynamic, (d) 60 Dynamic. Trajectories are color-coded by UAV speed ( m/s ), with a scale that differs across scenarios. Static obstacles are magenta; dynamic obstacles are red and move randomly at [1,4]m/s .
TABLE I: Baseline Comparison in Cluttered Environments with Different Densities of Static and Dynamic Obstacles
Fig. 6: Generalization to unseen environments. Three held-out environments, (a) Inclines, (b) Cave, (c) Forest, differ from the training scenes in geometry, obstacle shape, and free-space topology. Top: success rate over 50 trials against the achieved average speed, obtained by sweeping the reference speed from 1.0 to 7.0m/s . Bottom: trajectories of all five methods for one start–goal pair; grey dot is the start, yellow star the goal, dashed circles mark collisions. Dynamic obstacles are red, static obstacles blue to magenta.
TABLE II: Ablation study on the explicit range-map stream, the spatio-temporal risk field, and the arrival-time risk coupling
Fig. 7: Real-world experiments. (a) Our quadrotor platform: a Rockchip RK3588 onboard computer, an Intel RealSense D435i, a CUAV V5+ flight controller, the motors, and the battery. (b)–(d) Static flights through a dense forest, with (d) crossing a sloped section of irregular trunk spacing. (e)–(g) Dynamic flights in the same forest with two walking pedestrians; the red ellipse in (g) marks the quadrotor. Every sequence is composited from a fixed viewpoint, so the repeated silhouettes trace the executed trajectory.
System Modules
RTX 3090 Ti (ms)
RK3588 (ms)
Orin NX (ms)
Depth Processing
1.02
1.71
1.19
Frustum Range Map
0.58
1.09
0.79
Data Preparation
0.55
1.26
0.82
Network Inference
4.81
9.02
5.94
Proposal Selection
0.41
0.91
0.57
Total Latency
7.37
13.99
9.31
TABLE III: Latency of each component in the proposed system
Safe quadrotor navigation in cluttered and dynamic environments depends not only on instantaneous geometric perception, but more critically on anticipating collision risks induced by relative motion. Conventional modular pipelines frequently suffer from perception latency, while end-to-end learning methods relying on implicit scalar rewards often struggle to extract reliable spatio-temporal features without physics-grounded supervision. To address this, we propose an anticipatory risk-guided reinforcement learning framework. Leveraging privileged simulator states, we construct a directionally aligned future collision risk map based on the Closest Point of Approach (CPA). Through an asymmetric actor-critic architecture, the network is trained to self-predict this structured risk, which explicitly guides the visual policy during deployment. A lightweight spatio-temporal encoder extracts motion cues directly from onboard depth sequences, bypassing explicit object tracking or optical flow estimation. Extensive simulated and real-world experiments demonstrate that our method effectively improves safety margins and flight efficiency in dense dynamic clutters compared to existing baselines. Furthermore, the learned policy achieves robust zero-shot Sim-to-Real transfer on a physical quadrotor, relying purely on abstracted spatio-temporal depth sequences and its self-predicted risk priors, validating the effectiveness of our approach and its robust generalization from simulation to reality.
Yuchao Mei, Guohao Zhang, Luxia Ai +2
National Key Laboratory of Science and Technology on Multi-spectral Information Processing, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China
We present MC-Risk, a planner-aligned, multi-component risk field on a bird's-eye-view grid that yields early, calibrated, and class-aware risk localization. MC-Risk linearly composes three interpretable modules: (i) a motorized-agent field that fuses a black-box multimodal trajectory predictor with an analytic Gaussian-torus construction whose lateral width grows with speed/curvature and whose height attenuates with look-ahead; (ii) a VRU risk field that replaces isotropic pedestrian blobs with a forward-biased anisotropic kernel aligned to heading and speed; and (iii) a road penalty field that exploits full HD-map topology, imposing an off-road penalty and lane-aware risk exposure for same/opposite directions. We conduct, to our knowledge, the first standardized quantitative evaluation of a risk-field formulation on RiskBench's collision subset. MC-Risk attains the best overall risk localization and the earliest hazard indication. Finally, we demonstrate a plug-and-play planning interface by using the field as an MPC cost density, enabling risk-aware trajectory generation without additional training.
Maximilian Link, Yingjie Xu, Yingbai Hu +1
Technical University of Munich · Munich, Germany · The Chinese University of Hong Kong +3
Agile quadrotor avoidance of fast-moving obstacles requires anticipating collisions and selecting feasible maneuvers within short reaction windows. Reliable predictive avoidance remains challenging because sparse range observations do not directly reveal obstacle motion, while online trajectory optimizers either scale poorly with obstacle count or remain efficient at the expense of reliability in dense, high-speed encounters. We present OmniRisk, an omnidirectional planning framework that learns trajectory-level risk offline for efficient onboard evasion. A fixed-dimensional tensor combines LiDAR range panoramas, dynamic masks, and Cartesian surface velocities to represent geometry and motion jointly. We formulate an asymmetric risk field aligned with obstacle velocity that emphasizes approaching interactions and attenuates receding ones. Accumulating this risk along predicted relative trajectories provides dense supervision and discourages unnecessary hesitation after obstacles pass. A dual-branch circular convolutional network predicts terminal boundary states and dynamic risks for candidate primitives over an omnidirectional anchor lattice in a single forward pass, followed by selection and closed-form reconstruction of the selected candidate primitive. This formulation removes online risk accumulation along trajectories and makes risk-inference cost independent of obstacle count. OmniRisk enables efficient onboard avoidance, with real-world flights demonstrating consecutive evasive maneuvers at relative encounter speeds up to 15 m/s without fine-tuning. Code is available at https://github.com/VANdexj/OmniRisk.
Yifan He, Yang Liu, Wenhao Zhao +8
Differential Robotics, Hangzhou, China · Zhejiang University, Hangzhou, China