Organizations: State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, 430079, China · School of Electrical and Electronic Engineering, Nanyang Technological University, 639798, Singapore
Human-in-the-loop (HITL) UAV operation is essential in complex and safety-critical aerial surveying environments, where human operators provide navigation intent while onboard autonomy must maintain accurate and robust state estimation. A key challenge in this setting is that resource-constrained UAV platforms are often limited to narrow-field-of-view LiDAR sensors. In geometrically degenerate or feature-sparse scenes, limited sensing coverage often weakens LiDAR Inertial Odometry (LIO)'s observability, causing drift accumulation, degraded geometric accuracy, and unstable state estimation, which directly compromise safe and effective HITL operation and the reliability of downstream surveying products. To overcome this limitation, we present AWARE, a bio-inspired whole-body active yawing framework that exploits the UAV's own rotational agility to extend the effective sensor horizon and improve LIO's observability without additional mechanical actuation. The core of AWARE is a differentiable Model Predictive Control (MPC) framework embedded in a Reinforcement Learning (RL) loop. It first identifies the viewing direction that maximizes information gain across the full yaw space, and a lightweight RL agent then adjusts the MPC cost weights online according to the current environmental context, enabling an adaptive balance between estimation accuracy and flight stability. A Safe Flight Corridor mechanism further ensures operational safety within this HITL paradigm by decoupling the operator's navigational intent from autonomous yaw optimization to enable safe and efficient cooperative control. We validate AWARE through extensive experiments in diverse simulated and real-world environments.
Figures & tables
Fig. 1. : Overview of the proposed AWARE framework. Upper left: Bio-inspired active sensing strategy that expands FoV through adaptive yaw control. Lower left: The RL agent learns adaptive cost weights from a diverse scene database, balancing observability and flight stability. Center: System architecture with predictive observability analysis and hybrid MPC, operating under human-in-the-loop operations and safe flight corridor constraints. Right: Real-world validation showing observability-aware yaw planning along the pilot’s intended path.
Fig. 2. : Generation of the Unified Panoramic Representation. Left: The aggregated 3D pointcloud ScanBlock St and the spherical projection model relative to the UAV body frame. Right: The resulting 2D Panoramic Depth Map Dt . Each pixel (u,v) corresponds to an angular direction and stores the minimum depth value, min(∥p∥) , to accurately represent the closest environmental obstacles.
Fig. 3. : The Observability Analysis pipeline. We utilize the panoramic representation as input to simulate virtual scans for candidate yaw angles. By computing the Fisher Information Matrix (FIM) for each candidate, we generate an observability curve Jobs(ψ) .
Fig. 4. : Overview of the Hybrid RL-MPC Control Architecture. The State Encoding module fuses UAV kinematics and observability metrics into sint , and compresses the panoramic depth map into sext via perception encoder Eϕ . The RL Meta-Controller uses an Actor-Critic network to output adaptive MPC cost weights. The MPC Execution module remaps these weights into a constrained QP solver to produce the optimal control command u∗ . The reward rt closes the learning loop.
Fig. 5. : Simulation pipeline for the proposed active sensing framework. The environment integrates real-world point clouds and expert trajectories to drive UAV and LiDAR simulations. Within this closed loop, observability metrics and LIO feedback continuously train the RL policy to output control commands that maximize state estimation robustness.
Fig. 6. : Overview of the simulation dataset and training configuration. (a) Diverse training scenarios spanning structured, degraded, and unstructured environments, from which dense point cloud maps are collected. (b) Trajectory and training setup: expert-demonstrated reference trajectories ( xref ) serve as navigation baselines, with randomized initialization points along the path; the first 50% of each trajectory is used for training and the remainder for evaluation.
Sequence
Scenario Name
Category
Geometric Challenges
Length (m)
Seq01
Spine
Structured
Linear corridor, repetitive vertical columns
952 m
Seq02
Wuhan Subway
Structured
Large-scale indoor, repetitive patterns
344 m
Seq03
Building
Structured
Multi-level complex, partition walls
690 m
Seq04
Istanbul Tunnel
Degraded
Featureless Tunnel, geometric degeneracy
2314 m
Seq05
Wuhan Tunnel
Degraded
Unexposed space, linear structure
378 m
Seq06
Lava Tube
Unstructured
Irregular curvature, severe occlusion
292 m
Table 1: Detailed specification of the diverse simulation scenarios selected for active scanning training. The dataset is categorized into Structured, Degraded, and Unstructured environments, each presenting unique geometric challenges ranging from featureless tunnel to high-frequency clutter, ensuring comprehensive policy generalization.
Fig. 7. : Trajectory errors on the eight sequences of the AWARE dataset evaluated against the ground truth across five independent runs.
Fig. 8. : Estimated trajectories on our simulation datasets aligned with the ground truth. The plots display the single run with the minimum Absolute Pose Error (APE) for each sequence.
Fig. 9. : Quantitative localization results of the proposed AWARE framework across eight diverse testing scenarios. (a) Box plots illustrating the distribution of the Absolute Pose Error (APE) in meters, presented on a logarithmic scale. (b) Cumulative Distribution Function (CDF) of per-frame trajectory error.
Sequence
Scene
Fixed-rate (Fast)
Fixed-rate (Slow)
λobs=100
λobs=500
λobs=1000
λobs=1500
λobs=2000
AWARE
Simu-Seq01
Spine
5.24
6.96
4.16
3.39
3.04
3.49
3.35
2.29
Simu-Seq02
Wuhan Subway
14.80
8.41
5.26
3.39
6.55
4.23
6.27
1.38
Simu-Seq03
Building
2.38
2.03
1.24
1.41
1.34
1.18
1.23
0.71
Simu-Seq04
Istanbul Tunnel
139.31
132.89
21.70
19.36
20.75
19.61
21.70
17.86
Simu-Seq05
Wuhan Tunnel
3.01
2.74
2.02
1.94
1.63
1.42
1.62
1.20
Simu-Seq06
Lava Tube
3.92
3.87
3.05
3.20
3.28
3.00
3.12
1.68
Table 2: APE (in meters) of different scanning control methods across simulated scenes. Lower is better. Bold indicates best performance. Underline indicates second-best.
Fig. 10. : The custom-built quadrotor UAV platform used for real-world experiments. The onboard sensor suite includes a solid-state LiDAR with an IMU. An edge-grade x86 Intel-N305 processor runs the AWARE decision loop in real time.
Scene
Geometric Challenges
Duration (s)
Length (m)
Avg. Vel. (m/s)
Abandoned Athletic Infrastructure
Open-air, intermittent features
568.70
291.67
0.51
Underground Tunnel
Axial degeneracy, branching corridors
607.90
407.17
0.67
Abandoned Bunker
Confined, dusted, heavy occlusion
147.41
49.05
0.33
Dense Forest
Unstructured vegetation, sparse primitives
265.82
110.39
0.42
Total
–
1589.89
858.28
0.54
Table 3: Operational statistics of the real-world flight sequences.
Fig. 11. : Real-world deployment sites and representative field data acquisition. For each environment, the satellite image serves as the basemap, overlaid with the corresponding top-view point-cloud map. The accompanying photographs illustrate representative scene characteristics, TLS-based ground-truth data collection, and UAV flight during deployment. (a) Abandoned Athletic Infrastructure; (b) Underground Tunnel; (c) Abandoned Bunker; (d) Dense Forest.
Fig. 12. : Estimated trajectories aligned with the ground truth and the corresponding Absolute Pose Error (APE) across four real-world environments. (a) Abandoned Athletic Infrastructure; (b) Underground Tunnel; (c) Abandoned Bunker; (d) Dense Forest. Blue solid lines denote the ground truth, and red dashed lines denote the aligned estimates. Insets highlight zoomed-in views of local alignment details.
Scene
APE Mean (m)
APE RMSE (m)
APE Max (m)
Drift Rate (%)
Abandoned Athletic Infrastructure
0.0441
0.0510
0.1589
0.0545
Underground Tunnel
1.2008
1.3516
2.6868
0.6598
Abandoned Bunker
0.0081
0.0091
0.0281
0.0573
Dense Forest
0.0333
0.0355
0.0859
0.0778
Average
0.3216
0.3618
0.7399
0.2124
Table 4: Absolute Pose Error (APE, in meters) of AWARE across real-world environments. The drift rate is computed as the ratio of APE Max to the trajectory length and serves as a normalized measure of accumulated drift over the traveled distance.
Fig. 13. : Scatter plot and statistical distribution of the computing time required by the AWARE decision loop. The mean execution time is 46.90 ms, with a 95th percentile limit of 81.02 ms.
Fig. 14. : Guided by the hybrid RL-MPC framework, AWARE adaptively modulates the commanded yaw rate ψ˙ according to scene features. Left: representative corridor-like (feature-poor) and landmark-rich (feature-rich) subregions. Right: corresponding controller traces, where the corridor-like segment exhibits a higher average ψ˙ , whereas the landmark-rich segment requires a lower average ψ˙ .
Scene
Mean λobs
Abandoned Athletic Infrastructure
1124.14
Underground Tunnel
1585.54
Abandoned Bunker
1544.85
Dense Forest
268.30
Table 5: Average RL modulated observability weight λobs across real world scenes. Larger values indicate stronger emphasis on observability in the MPC objective.
Open-source LiDAR-inertial odometry (LIO) systems have achieved remarkable benchmark accuracy, yet current state-of-the-art implementations are primarily optimized for evaluation performance rather than the requirements of real-time closed-loop aerial control. When deployed onboard UAVs, this can introduce limitations that degrade flight performance. In this work, we identify five architectural deficiencies in a representative tightly coupled IESKF-based LIO implementation: odometry publishing tied to the LiDAR rate (10 Hz instead of the IMU's 200 Hz), missing velocity outputs, execution bottlenecks that block IMU processing, mutex contention, and synchronization race conditions. We introduce corresponding modifications including IMU-rate forward propagation, direct body-frame velocity publishing, SLERP-based smoothing, dual-executor isolation, and explicit synchronization protection. The resulting system increases odometry output from ~10 Hz to a stable 200 Hz, provides a complete Twist state at every IMU sample, and preserves continuity during transient LiDAR loss. Experiments on a Livox Mid-360 / Pixhawk 4 Mini autonomous UAV with motion-capture ground truth validate the approach. Since the underlying estimator (IESKF + ikd-Tree) remains unchanged, the proposed improvements can be directly applied to FAST-LIO2-derived implementations.
Alvaro J. Gaona, David Perez-Saura, Francisco J. Anguita +1
Computer Vision and Aerial Robotics Group at Centre for Automation and Robotics C.A.R. (UPM-CSIC), Universidad Politécnica de Madrid (CVAR-UPM), Calle Jose Gutierrez Abascal 2, 28006 Madrid, Spain
Robust and accurate navigation is critical for Unmanned Aerial Vehicles (UAVs) especially for those with stringent Size, Weight, and Power (SWaP) constraints. However, most state-of-the-art (SOTA) LiDAR-Inertial Odometry (LIO) systems still suffer from estimation inconsistency and computational bottlenecks when deployed on such platforms. To address these issues, this paper proposes a consistent and efficient tightly-coupled LIO framework tailored for UAVs. Within the efficient Multi-State Constraint Kalman Filter (MSCKF) framework, we build coplanar constraints inferred from planar features observed across a sliding window. By applying null-space projection to sliding-window coplanar constraints, we eliminate the direct dependency on feature parameters in the state vector, thereby mitigating overconfidence and improving consistency. More importantly, to further boost the efficiency, we introduce a parallel voxel-based data association and a novel compact cluster-to-plane measurement model. This compact measurement model losslessly reduces observation dimensionality and significantly accelerating the update process. Extensive evaluations demonstrate that our method outperforms most state-of-the-art (SOTA) approaches by providing a superior balance of consistency and efficiency. It exhibits improved robustness in degenerate scenarios, achieves the lowest memory usage via its map-free nature, and runs in real-time on resource-constrained embedded platforms (e.g., NVIDIA Jetson TX2).
Jinwen Zhu, Xudong Zhao, Fangcheng Zhu +4
Meituan UAV, Beijing, China · Dept. of Mechanical Engineering, Computer and Information Sciences, University of Delaware, Newark, DE
LiDAR Inertial Odometry (LIO) is a critical component for many mobile robots that need to navigate without relying on external positioning (e.g., GPS). Platforms that operate autonomously in different environments and with heterogeneous LiDAR sensors require a LIO approach that can adapt to these different scenarios without human intervention. Existing LIO approaches can typically provide reliable and accurate odometry in scenarios with similar environments and sensors when suitably tuned. However, many approaches struggle to retain robust odometry across heterogeneous environments and sensors while using a consistent configuration. This paper presents EllipseLIO, a real-time LIO approach that generalises between scenarios by using methods for LiDAR scan filtering and registration that adapt to the sensor capabilities and environment without requiring scenario-specific tuning. Experiments with EllipseLIO and state-of-the-art LIO approaches on five datasets with diverse and challenging scenarios demonstrate that EllipseLIO is the best-performing approach overall. It achieves a 38% lower odometry error on average than the second-best approach and is the only approach that does not diverge in any experiment. An open-source version of EllipseLIO will be available at github.com/v4rl-ucy/ellipselio.
Rowan Border, Margarita Chli
Vision for Robotics Lab (V4RL), University of Cyprus, Cyprus · ETH Zurich, Switzerland