Robust Tightly-Coupled Filter-Based Monocular Visual-Inertial State Estimation and Graph-Based Evaluation for Autonomous Drone Racing

Maulana Bisyir Azhari; Donghun Han; Sung Jun Park; David Hyunchul Shim

cs.ROMar 3, 2026

Robust Tightly-Coupled Filter-Based Monocular Visual-Inertial State Estimation and Graph-Based Evaluation for Autonomous Drone Racing

Authors: Maulana Bisyir Azhari, Donghun Han, Sung Jun Park, David Hyunchul Shim

Organizations: Unmanned Systems Research Group (USRG), School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea.

Abstract

Autonomous drone racing (ADR) demands state estimation that is simultaneously computationally efficient and resilient to the perceptual degradation experienced during extreme velocity and maneuvers. Traditional frameworks typically rely on conventional visual-inertial pipelines with loosely-coupled gate-based Perspective-n-Points (PnP) corrections that suffer from a rigid requirement for four visible features and information loss in intermediate steps. Furthermore, the absence of GNSS and Motion Capture systems in uninstrumented, competitive racing environments makes the objective evaluation of such systems remarkably difficult. To address these limitations, we propose ADR-VINS, a robust, monocular visual-inertial state estimation framework based on an Error-State Kalman Filter (ESKF) tailored for autonomous drone racing. Our approach integrates direct pixel reprojection errors from gate corners features as innovation terms within the filter. By bypassing intermediate PnP solvers, ADR-VINS maintains valid state updates with as few as two visible corners and utilizes robust reweighting instead of RANSAC-based schemes to handle outliers, enhancing computational efficiency. Furthermore, we introduce ADR-FGO, an offline Factor-Graph Optimization framework to generate high-fidelity reference trajectories that facilitate post-flight performance evaluation and analysis on uninstrumented, GNSS-denied environments. The proposed system is validated using TII-RATM dataset, where ADR-VINS achieves an average RMS translation error of 0.143 m, while ADR-FGO yields 0.060 m as a smoothing-based reference. Finally, ADR-VINS was successfully deployed in the A2RL Drone Championship Season 2, maintaining stable and robust estimation despite noisy detections during high-agility flight at top speeds of 20.9 m/s. We further utilize ADR-FGO for post-flight evaluation in uninstrumented racing environments.

Figures & tables

Fig. 1: Left: The autonomous drone racing platform used in the A2RL Drone Championship Season 2, featuring a low-cost monocular RGB camera and IMU sensors. Right: Autonomous racing flight at high speeds over 20 m/s featuring the proposed ADR-VINS for its real-time state estimation.

Ref

Validation

Sensors

Gate/Corner Detection

State Estimation Pipeline

Top Speed

Real-World

Dataset

Camera

IMU

Additional

Gate

Corner

Type

[ 2 ]

AIRR ∗

Stereo

Yes

1D LiDAR

U-Net

1-stage †

ROVIO + Drift EKF

8 m/s

[ 3 ]

AIRR ∗

Mono

Yes

1D LiDAR

GateNet

Snake-Gate

2-stages

RANSAC + PnP + MHE

9.2 m/s

[ 4 ]

Self-hosted

Stereo

Yes

U-Net

1-stage †

T265-VIO + PnP + Drift KF

22 m/s

[ 5 ]

Self-hosted

Stereo

Yes

YoloV8

U-Net

2-stages

T265-VIO + PnP + Drift KF + EKF

21.1 m/s

[ 6 ]

A2RL S1 ∗

Mono

Yes

U-Net

1-stage †

VINS-Mono/OpenVINS + PnP + Drift KF

12.5 m/s

TABLE I: Comparison of Gate-Based State-Estimation Methods for Autonomous Drone Racing.

Fig. 2: Overview of ADR-VINS. (a) The framework consists of a visual measurement pipeline (including detection, reordering, and association) and a tightly-coupled Error-State Kalman Filter (ESKF) that fuses raw IMU data with direct pixel reprojection errors to provide robust, high-rate state estimation. (b) Illustration of the corner detections (yellow dots), reprojections (red dots), and the corresponding reprojection errors (red lines).

Fig. 3: The proposed factor graph structure in ADR-FGO. The graph integrates IMU pre-integration, gate corner detections, pose prior factors from ADR-VINS, and camera extrinsics priors. Keyframes are maintained even during visual outages to distribute drift and IMU biases across these visual-less sections to provide better interpolation.

Seq		ONLINE METHODS												OFFLINE METHODS
		OpenVINS			PnP-Pose			PnP+EKF			ADR-VINS			MAPLAB			ADR-FGO
		$e_{t}$	$e_{r}$	$e_{v}$	$e_{t}$	$e_{r}$	$e_{v}$	$e_{t}$	$e_{r}$	$e_{v}$	$e_{t}$	$e_{r}$	$e_{v}$	$e_{t}$	$e_{r}$	$e_{v}$	$e_{t}$	$e_{r}$	$e_{v}$
Piloted	05P	.296	1.77	.293	.466	4.05	-	.282	1.59	.388	.130	1.73	.202	.194	1.60	.153	.052	1.16	.090
	06P	.943	2.21	.449	.488	4.17	-	.350	2.64	.505	.134	1.69	.218	.159	1.35	.132	.061	1.47	.100
	11P	.369	3.12	.345	.398	3.39	-	.242	1.76	.371	.132	1.85	.199	.249	2.20	.216	.054	1.25	.110
	12P	.204	2.39	.194	.298	3.87	-	.245	1.62	.355	.169	2.02	.221	.297	1.67	.208	.049	1.39	.090

TABLE II: Results on the TII-RATM Dataset.

Fig. 4: Qualitative comparison of ADR-VINS against baseline online methods across various tracks on the TII-RATM dataset autonomous sequences, including an ellipse (Flight 06A), a lemniscate (Flight 12A), and a complex 3D racing track (Flight 18A).

Fig. 5: Comparison of ADR-VINS with

\#

\#

4, and

\#

6 minimum corners visibility requirements for Flight 18A. Requiring only two corners enables earlier and more frequent updates, significantly reducing drift. ADR-FGO further minimizes this error by globally refining ADR-VINS, even in sections where gates are not visible. G1-G7 is the gate ordering.

	w/o vl KF			w/o ext factor			Full
Seq.	$e_{t}$	$e_{r}$	$e_{v}$	$e_{t}$	$e_{r}$	$e_{v}$	$e_{t}$	$e_{r}$	$e_{v}$
Avg @P	.056	1.51	.111	.067	1.88	.115	.054	1.32	.098
Avg @A	.064	2.37	.177	.070	2.15	.179	.064	2.13	.165

TABLE III: ADR-FGO Ablation Study.

Panel of Figure 9 — Fig. 6: Top: Visualization of reprojection errors for ADR-VINS and ADR-FGO compared to raw detections on flight 18A. Bottom: Statistics of the detection and reprojection errors on TII-RATM dataset.

Panel of Figure 10 — Fig. 7: Top: 3D visualization of ADR-VINS against GT (ADR-FGO) of our fastest flight in A2RL S2 track. Middle: Velocity and acceleration profile. Bottom: Error analysis.

PnP-Pose			PnP+EKF			ADR-VINS
$e_{t}$	$e_{r}$	$e_{v}$	$e_{t}$	$e_{r}$	$e_{v}$	$e_{t}$	$e_{r}$	$e_{v}$
.797	5.97	-	.367	2.31	.717	.152	1.25	.413

TABLE IV: Results on A2RL Season 2 Track.

Panel of Figure 12 — Fig. 8: Top: Visualization of the corner detections and the reprojections of ADR-VINS and ADR-FGO estimates. Bottom: Statistics of the detection and reprojection errors on our fastest flight in A2RL S2.

Fig. 9: Robustness ablation study visualization. Performance comparison of the proposed Huber-based reweighting, a

\chi^{2}

(chi-squared) rejection test, and naive visual measurements fusion. The Huber-based method maintains stability against noisy detections where naive and

\chi^{2}

test variants exhibit significant drift or failure.

	Visual Measurements		ESKF
component:	detection	reorder.+assoc.	propagation	update
time [ms]:	8.165	0.192	0.011	0.242

TABLE V: Timing of ADR-VINS.

Fig. 11: Robustness against added detection noise in TII-RATM Dataset.

Autonomous Drone Racing

See all 12 papers

Dual Pose-Graph Semantic Localization for Vision-Based Autonomous Drone Racing
Apr 16, 2026David Perez-Saura, Miguel Fernandez-Cortizas, Alvaro J. Gaona +1Pose Graph Optimization Autonomous Racing
Computer Vision & Aerial Robotics Group, Universidad Polit´ecnica de Madrid, Madrid, Spain · Automation and Robotics Group (ARG-SnT), University of Luxembourg – SnT, Luxembourg
TinyCVIO: A Constellation-Aided Visual-Inertial Odometry System for Nanodrones
Sep 24, 2026Derin Ozturk, Kaan Akan, Irwin Wang +1Visual-Inertial Odometry UAV Navigation
Department of ECE, Cornell University, Ithaca, NY, USA
AERO-VIS: Asynchronous Event-based Real-time Onboard Visual-Inertial SLAM
May 8, 2026Yannick Burkhardt, Sebastián Barbas Laina, Simon Boche +2Event-Based Vision UAV Navigation
Mobile Robotics Lab, ETH Zürich · Technical University of Munich · Munich Center for Machine Lerning (MCML) +1

Apr 16, 2026J/K move · Enter open · S save

Dual Pose-Graph Semantic Localization for Vision-Based Autonomous Drone Racing

David Perez-Saura, Miguel Fernandez-Cortizas, Alvaro J. Gaona, Pascual Campoy

Computer Vision & Aerial Robotics Group, Universidad Polit´ecnica de Madrid, Madrid, Spain · Automation and Robotics Group (ARG-SnT), University of Luxembourg – SnT, Luxembourg

Pose Graph Optimization Autonomous Racing Robot Localization Autonomous Drone Racing

Open paper AskPDF arXiv

Autonomous drone racing demands robust real-time localization under extreme conditions: high-speed flight, aggressive maneuvers, and payload-constrained platforms that often rely on a single camera for perception. Existing visual SLAM systems, while effective in general scenarios, struggle with motion blur and feature instability inherent to racing dynamics, and do not exploit the structured nature of racing environments. In this work, we present a dual pose-graph architecture that fuses odometry with semantic detections for robust localization. A temporary graph accumulates multiple gate observations between keyframes and optimizes them into a single refined constraint per landmark, which is then promoted to a persistent main graph. This design preserves the information richness of frequent detections while preventing graph growth from degrading real-time performance. The system is designed to be sensor-agnostic, although in this work we validate it using monocular visual-inertial odometry and visual gate detections. Experimental evaluation on the TII-RATM dataset shows a 56% to 74% reduction in ATE compared to standalone VIO, while an ablation study confirms that the dual-graph architecture achieves 10% to 12% higher accuracy than a single-graph baseline at identical computational cost. Deployment in the A2RL competition demonstrated that the system performs real-time onboard localization during flight, reducing the drift of the odometry baseline by up to 4.2 m per lap.

Abstract

Autonomous Drone Racing

Explore similar work

Dual Pose-Graph Semantic Localization for Vision-Based Autonomous Drone Racing

TinyCVIO: A Constellation-Aided Visual-Inertial Odometry System for Nanodrones

AERO-VIS: Asynchronous Event-based Real-time Onboard Visual-Inertial SLAM