SUPER - Sensitivity-based Uncertainty-aware Performance and Risk Assessment in Visual Inertial Odometry
Authors: Johannes A. Gaus, Daniel Häufle, Woo-Jeong Baek
Organizations: Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience, University of Tübingen, Germany. · Intelligent Process Automation and Robotics Lab (IPR), Karlsruhe Institute of Technology (KIT), Germany, Artificial Intelligence Institute (AIIS), Seoul National University, Republic of Korea.
This paper presents SUPER (Sensitivity-based Uncertainty-aware PErformance and Risk assessment), a bounded controller that adapts visual measurement information in visual-inertial odometry at runtime. Building on multi-view landmark dispersion from stereo observations, SUPER projects track consistency into pixel space and combines it with the reprojection residual to gate a temporally smoothed, bounded information update. Because the controller operates inside nondeterministic optimizers, we additionally propose an action-verification protocol based on factor-level logs, inactive-instrumented controls, and same-linearization replay. We evaluate SUPER in Ceres and g2o backends on EuRoC, TUM-VI, UMA-VI, and TartanAir under controlled visual corruptions, with a frozen policy and prospective confirmation. SUPER is computationally efficient and selectively active, but does not establish a general accuracy improvement: all 13 split-level confidence intervals cross zero, and effects are heterogeneous across corruption families. The action-verification protocol reliably distinguishes controller-induced state changes from solver variability.
Figures & tables
Fig. 1: Overview of SUPER: SUPER forms a multi-view dispersion signal on track level on the basis of stereo observations and optimized backend states. These signals are mapped into the pixel space by considering the uncertainties. Next, the obtained information is combined with the residual to activate the temporal update for the ViO optimization in the subsequent timestep.
Fig. 2: Schematic overview of SUPER . (1) Backend optimization provides state estimates with residuals and Jacobians that form (2) The Schur complement marginalizes poses and yields per landmark covariance. (3) Each covariance is projected into the pixel space to obtain per feature uncertainty. (4) Aggregating all pixel level covariances produces a dense map of feature uncertainties across the image. (5) Geometric conditioning is computed from the shape of each pixel covariance, revealing how viewing geometry affects estimation quality. (6) The residual and the feature uncertainty are transformed into an activation signal. (7) the obtained information matrix is integrated into the visual residuals in the Ceres or g2o backend. The frozen Ceres policy initialization is performed at ρt≥1 and avoids the effect of overweighted information.
Fig. 3: Cross-split evaluation. Median paired translation-RMSE change with 95% cluster-bootstrap CIs; negative values favor SUPER . Top: 13 evaluation splits. Bottom: corruption-family decomposition of the held-out Ceres split.
Split
R/P
W/L/T
Med. Δt
Div. F/A / ΔC
Held-out Ceres
57/68
31/20/6
−1.01%
11/11 / −39
Locked Ceres
102/119
50/43/9
0.00%
17/17 / −63
Prospective clean
7/7
5/2/0
−7.42%
0/0 / −3
Prospective blur
14/14
5/9/0
+4.31%
0/0 / +5
Repeated all UMA-VI
10/10
7/2/1
−2.07%
0/0 / –
Repeated held-out UMA
6/6
4/1/1
−2.07%
0/0 / –
TABLE I: Frozen-policy confirmation. R/P denotes robust/planned pairs; the final column reports fixed/adaptive divergences and the Ceres-warning delta ΔC .
Mode
R
W/L/T
Med. Δt
P90 Δt
Scalar controls
SUPER
23
13/7/3
−2.14%
+11.64%
Residual-only
23
14/6/3
−2.03%
+14.70%
Feature count
23
12/8/3
−0.76%
+8.01%
DCS-like
23
8/9/6
0.00%
+100.75%
Random-action (matched duty)
23
9/11/3
+0.00%
+68.11%
TABLE II: Ablations relative to fixed information. R denotes robust pairs.
Fig. 4: Same-linearization replay, MH03 temporal occlusion and repeat. Top: Modified visual factors per sliding-window solve. Bottom: The fixed ↔ active pose separation from an identical shadow solve. The color indicates modifications. The action-zero solves correspond to floor precision while action-positive solves range from sub-millimeter to the 3.43 m outlier.
Backend/contrast
Condition
W/L/T
Med. Δt
Med. ΔR
Ceres A/F
Clean
9/6/0
−1.74%
+1.17%
Temporal blur
6/9/0
+6.05%
+1.13%
Temporal occl.
10/5/0
−0.10%
−3.50%
g2o A/F
Clean
6/9/0
+2.29%
−0.26%
Temporal blur
10/5/0
−4.07%
+0.85%
Temporal occl.
5/10/0
+3.78%
+0.02%
TABLE III: Matched three-repeat confirmation. A/F and A/I denote active/fixed and active/inactive comparisons.
Backend/contrast
Scope
R/P
W/L/T
Med. Δt [95% CI]
Ceres selected/fixed
Severity
19/21
9/10/0
+4.48 [ −7.64,+8.67 ]
g2o active/fixed
Severity
21/21
12/9/0
−1.86 [ −6.70,+2.89 ]
g2o active/inactive
Severity
21/21
10/11/0
+1.04 [ −4.01,+4.16 ]
Ceres strict/fixed
MH04
2/2
2/0/0
−11.13 [ −13.67,−8.59 ]
Ceres mild/fixed
TUM clean
2/2
2/0/0
−6.12 [ −11.97,−0.28 ]
TABLE IV: Preregistered causal/safety suite. R/P denotes robust/planned sequence–condition clusters after repeat aggregation. Negative translation change favors the first mode. TUM guard rows include clean clusters only; two-cluster guard intervals are descriptive.
Condition
Runs
Gate open
Modified factors
Held-out clean
4
0.67%
0.59%
Held-out degraded
64
17.69%
7.06%
Matched Ceres clean
15
0.53%
0.47%
Matched Ceres degraded
30
17.41%
15.57%
Matched g2o clean
15
0.00%
0.00%
Matched g2o degraded
30
0.069%
0.014%
TABLE V: Median intervention duty from factor-level logs.
Visual-inertial odometry (VIO) for aerial robots relies on high rate inertial measurement unit (IMU) propagation between visual updates. However, conventional multi state constraint Kalman filters (MSCKFs) use random walk bias assumptions and fixed noise parameters, which can limit robustness when visual information is unreliable. To address this problem, we propose LBDU-VIO, a learning-augmented MSCKF with learned continuous time bias dynamics and an IMU uncertainty model. A neural ordinary differential equation (ODE) models continuous time bias dynamics to propagate the filter's bias states, replacing their random walk model. The IMU uncertainty model predicts motion adaptive measurement noise covariances for covariance propagation. Both models are trained with pose supervision without direct labels. Experiments on real world EuRoC and TUM-VI benchmarks show lower errors than representative visual-inertial baselines, including a 25.1% reduction in mean relative position error compared with S-MSCKF on EuRoC sequences with 10s visual outage.
Qizhi Guo, Junning Lyu, Defu Lin +1
School of Aerospace Engineering and the Beijing Key Laboratory of UAV Autonomous Control, Beijing Institute of Technology, Beijing 100081, China
This paper presents a geometric observer for monocular visual-inertial odometry driven by individual optical-flow constraints. For each tracked point, the bearing and its rotationally compensated derivative give a depth-free homogeneous constraint on the body-frame velocity. These constraints enter the innovation of an observer on the gravity-augmented group SE3(3) through a Riccati gain, without estimating depths or reconstructing the direction of motion. The auxiliary gravity state makes the six-dimensional body-frame velocity-gravity error dynamics independent of the attitude correction, and their uniform observability is shown to be equivalent to persistent excitation of a matrix formed from the inertial bearings and their derivatives. Under this condition, the body-frame velocity and gravity errors converge globally and uniformly exponentially, and a gravity-based attitude correction renders the combined velocity, gravity, and reduced-attitude error almost globally asymptotically stable and locally uniformly exponentially stable. Simulations illustrate the excitation condition, recovery from initial errors, and sensitivity to perturbed measurements.
Tarek Bouazza, Soulaimane Berkane, Tarek Hamel
I3S, CNRS, Université Côte d’Azur 06900 Sophia Antipolis, France · Département d’informatique et d’ingénierie, Université du Québec en Outaouis Gatineau, QC J8X 3X7, Canada · Department of Electrical Engineering, Lakehead University Gatineau, QC J8X 3X7, Canada +1
In cuVSLAM, visual and inertial measurement unit (IMU) residuals are fused in sparse bundle adjustment (SBA) with a weighting fixed by pre-calibrated noise covariances, leaving no mechanism to adapt the visual-inertial balance at runtime. We introduce a scalar α≥1 that upscales the visual information matrix relative to the IMU preintegration constraint. Across the candidate set {1.0,2.0,5.0,10.0,20.0}, the resulting trajectories form two empirical performance regimes. The IMU-TRUST regime contains α∈{1,2} near the calibrated baseline, while the VISION-TRUST regime contains α∈{10,20}. The intermediate value α=5 is treated as a transition case. Differences within each regime are small relative to those observed between the regimes. This structure motivates reducing α-selection to a binary decision. We train a lightweight Visual Quality Estimator (VQE), implemented as a multilayer perceptron with 914 trainable parameters. It maps a 10-dimensional visual quality feature vector assembled from statistics already maintained by the tracker to a per-SBA regime prediction. The proposed method reduces mean RMSE APE by 27.7% and by 14.9% under leave-one-out cross-validation (LOOCV) on EuRoC MAV and TUM-VI Room, respectively.