cs.ROAug 21, 2026

IMU-Free Body-Frame State Estimation with Sparse Scene Flow for Quadcopters

Authors: Daniel Grønhaug, Sofie Markeset, Mathias Kolberg

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present a vision-only state estimation system for X-configuration quadcopters equipped with a canonical stereo camera pair and no inertial sensors. The system operates entirely in the body frame, requiring only synchronised stereo images and motor thrust commands. A continuous-discrete extended Kalman filter on a composite manifold state ⟨SE(3),R3,…⟩\langle SE(3), \mathbb{R}^3, \ldots \rangle maintains estimates of body-frame pose, velocity, angular velocity, gravity, and disturbances, using stationary scene points as implicit inertial references. Feature points are detected (FAST, Shi-Tomasi), tracked temporally (SSD, Lucas-Kanade) and matched across cameras (NCC), with search regions predicted from filter-derived pose and point uncertainty. Chi-squared gating on the normalised innovation admits only stationary points to the filter. The system also produces a sparse 3D point cloud carrying per-point position, velocity and joint covariance. These come from a 4-view (two stereo pairs at two timestamps) full bundle adjustment that jointly estimates position and velocity from stereo disparity and temporal parallax, with the filter-derived relative pose as a prior. Feature points in the EKF do not enter the solver; their information is reflected through the pose prior. Point cloud density is spatially adaptive: an external focus point directs allocation, producing dense coverage in the region of attention and sparse coverage elsewhere. The output is a body-frame state estimate, a calibrated pose change, and a sparse scene flow. It is intended as a measurement source for a downstream world model anchored in the current body frame, without dependence on GPS, IMU, or any world-frame infrastructure, though the architecture accommodates their future integration.

Figures & tables

Figure 1: Representative left-camera frame from the VID indoor sequence. Hangar interior at the start of the bag, quadcopter stationary on the floor.
Figure 2: All but last 350 frames. Per-frame error magnitudes (translation, rotation, body-frame linear and angular velocity) against Vicon ground truth. Dashed red: RMSE over the full evaluated window.
Figure 3: All frames. Per-frame error magnitudes (translation, rotation, body-frame linear and angular velocity) against Vicon ground truth. Dashed red: RMSE over the full evaluated window.
Figure 4: All but last 350 frames. Estimated vs ground-truth trajectory. Left: 3D path. Right: per-axis position, body-frame velocity, and angular velocity over time.
QuantityRMSEMedianp95
Translation [m]1.1040.2293.964
Rotation [deg]7.552.5828.25
Velocity magnitude [m/s]0.5020.0340.457
∣vx∣|v_{x}| error [m/s]0.0650.0100.065
∣vy∣|v_{y}| error [m/s]0.2730.0190.101
∣vz∣|v_{z}| error [m/s]0.4170.0100.420
Table 1: All frames summary error metrics (4733 frames, 79.0 s), including ground contact, takeoff, flight, and landing. The RMSE figures are dominated by post-landing divergence.
Window [s]Rot [deg] med∣vz∣|v_{z}| med∣vz∣|v_{z}| p95Trans med
0–50.190.0050.0180.002
5–100.520.2770.6890.152
10–202.450.0090.0480.177
20–402.510.0060.0230.154
40–602.740.0070.0320.278
60–702.610.0180.0630.261
Table 2: Error breakdown by time window over all frames. Takeoff occurs at t≈8t\approx 8 s (window 5–10). Landing occurs at t≈72t\approx 72 s (window 70–73). The 73–75 s window contains catastrophic divergence: translation error grows from 0.32 m to 3.4 m and rotation from 2.2° to 20° within 2 s of touchdown.
Figure 5: All frames. Internal state diagnostics. Top: body-frame gravity estimate (per-axis and magnitude; grey dashed: nominal ∥g∥=9.7935\|\mathbf{g}\|=9.7935 m/s 2 ). Middle: body-frame disturbance estimate. Bottom: point-set sizes over time — features in the EKF (blue), pre-admission candidates (green), interest points (red).
More in this topic

Robot State Estimation

Robot state estimation infers a robot’s position, motion, orientation, and internal configuration from noisy sensors and physical constraints.

See all 196 papers

Explore similar work

CardsList
Apr 20, 2026

Relative State Estimation using Event-Based Propeller Sensing

Autonomous swarms of multi-Unmanned Aerial Vehicle (UAV) system requires an accurate and fast relative state estimation. Although monocular frame-based camera methods perform well in ideal conditions, they are slow, suffer scale ambiguity, and often struggle in visually challenging conditions. The advent of event cameras addresses these challenging tasks by providing low latency, high dynamic range, and microsecond-level temporal resolution. This paper proposes a framework for relative state estimation for quadrotors using event-based propeller sensing. The propellers in the event stream are tracked by detection to extract the region-of-interests. The event streams in these regions are processed in temporal chunks to estimate per-propeller frequencies. These frequency measurements drive a kinematic state estimation module as a thrust input, while camera-derived position measurements provide the update step. Additionally, we use geometric primitives derived from event streams to estimate the orientation of the quadrotor by fitting an ellipse over a propeller and backprojecting it to recover body-frame tilt-axis. The existing event-based approaches for quadrotor state estimation use the propeller frequency in simulated flight sequences. Our approach estimates the propeller frequency under 3% error on a test dataset of five real-world outdoor flight sequences, providing a method for decentralized relative localization for multi-robot systems using event camera.

Ravi Kumar Thakur, Luis Granados Segura, Jan Klivan +4

Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Karlovo Namesti 13, 121 35 Prague 2, Czechia

Event-Based VisionMulti-Robot SystemsRobot State EstimationRobot Localization
AskPDF
Jun 8, 2026

Dual Quaternion-Based Unscented Kalman Filter with Visual Inertial Odometry for Navigation in GPS-Denied Environments

Reliable navigation in GPS-denied environments remains a fundamental challenge in robotics, aerospace, and autonomous vehicle applications. This paper presents a Dual Quaternion-Based Unscented Kalman Filter (DQUKF) equipped with a Visual Inertial Odometry (VIO) algorithm for accurate state estimation enabling navigation in GPS denied locations. The proposed framework formulates the DQUKF in an error state manner, where the nominal pose is represented by a unit dual quaternion and the local pose error is represented by a 6-dimensional twistor parameterization used for sigma point generation, covariance propagation, and measurement correction. In parallel, the VIO algorithm tracks features across image frames, synchronizes measurements between the IMU and camera, and provides visual constraints that complement inertial propagation. Simulation results on the EuRoC MAV dataset show that the proposed DQUKF converges under high initialization uncertainty and achieves a position RMSE of 0.2584~m in the difficult flight sequence, outperforming the benchmark filters.

Mohamed Khalifa, Hashim A. Hashim

Department of Mechanical and Aerospace Engineering, Carleton University, Ottawa, Ontario, K1S-5B6, Canada

Kalman FilteringInertial OdometryGNSS-Denied LocalizationUnscented Kalman FilteringVisual-Inertial OdometryRobot State Estimation
AskPDF
Sep 30, 2026

Towards Agile Vision-Based Multi-UAV Flight: Revisiting State Estimation

Agile multi-UAV flight requires accurate and low-latency onboard estimation of the kinematic states of neighboring UAVs for collision avoidance, motion coordination, etc. Most vision-based approaches rely on position-only measurements, inferring velocity and acceleration indirectly from displacement. We show that this introduces a fixed structural delay in the estimation of higher-order states, which limits the achievable agility. To address this, we propose to integrate tilt measurements, provided by a state-of-the-art visual detector, which inform about the thrust direction of co-planar multirotor UAVs. We benchmark four position-only and five pose-aware estimators, including a novel formulation of a linear thrust-constraining Kalman filter, on two real-world and one high-fidelity photorealistic simulated dataset over different levels of agility (3-21 m/s^2). In our setup, pose-aware estimation consistently reduces the average velocity and acceleration estimation errors by 40% and 57% across the three datasets with the proposed KF formulation outperforming the other estimators. Position-only filters exhibit a constant ~300 ms delay in acceleration step response independent of agility, whereas the tilt-constrained estimators operate near the physical response limit given by the camera frame-rate by observing the change in thrust direction before the displacement accumulates. In a closed-loop leader-follower simulated experiment with NMPC control, position-only estimation of the leader's state fails to facilitate stable hovering of the follower, while the proposed estimator enables tracking of lateral maneuvers exceeding 2g of acceleration.

Michal Pliska, Matouš Vrba, Ondřej Víta +3

Multi-robot Systems Group, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic

Aerial RoboticsKalman FilteringRobot State Estimation
AskPDF