Organizations: Dalle Molle Institute for Artificial Intelligence (IDSIA), USI-SUPSI, Lugano, Switzerland · University of Milan, Italy · Integrated Systems Laboratory (IIS), ETHZ, Zürich, Switzerland
We consider the task of visually estimating the relative pose of a drone racing gate in front of a nano-quadrotor, using a convolutional neural network pre-trained on simulated data to regress the gate's pose. Due to the sim-to-real gap, the pre-trained model underperforms in the real world and must be adapted to the target domain. We propose an unsupervised domain adaptation (UDA) approach using only real image sequences collected by the drone flying an arbitrary trajectory in front of a gate; sequences are annotated in a self-supervised fashion with the drone's odometry as measured by its onboard sensors. On this dataset, a state consistency loss enforces that two images acquired at different times yield pose predictions that are consistent with the drone's odometry. Results indicate that our approach outperforms other SoA UDA approaches, has a low mean absolute error in position (x=26, y=28, z=10 cm) and orientation (ψ=13∘), an improvement of 40% in position and 37% in orientation over a baseline. The approach's effectiveness is appreciable with as few as 10 minutes of real-world flight data and yields models with an inference time of 30.4ms (33 fps) when deployed aboard the Crazyflie 2.1 Brushless nano-drone.
Figures & tables
Fig. 1: We estimate the pose ( x , y , z , ψ ) of drone racing gates by transferring a model trained in simulation to a real-world environment and deploying it aboard the Bitcraze Crazyflie 2.1 Brushless nano-UAV.
Fig. 2: Assuming perfect odometry (left), the state consistency loss forces the relative gate poses P^G1 and P^G2 , predicted from the drone’s poses at PR1 and PR2 , to be coherent with the drone’s relative odometry O1→2 between the two poses. In our experiments, the measured odometry O^1→2 is collected by the drone itself and is affected by drift and noise (right).
Fig. 3: The Crazyflie 2.1 Brushless nano-drone used in our experiments.
Fig. 4: Examples of (left) simulated images from Dsimtrain , (middle) real-world robot images in Drealtrain , and (right) after applying the pencil filter [ 6 ] .
Model
Supervision
MSE ↓
MAE↓
ρ↑
Plot for MSExyz [cm] ←
xyz [cm]
x [cm]
y [cm]
z [cm]
ψ [deg]
x [%]
y [%]
z [%]
ψ [%]
Error bars mark 95% CI
Mean Predictor
Drealtest
90.7
57.0
48.7
23.1
20.9
−
−
−
−
0 30 60 90
Zero-Shot
Dsimtrain
79.7
51.6
41.0
24.5
33.5
38
53
40
5
PencilNet [ 6 ]
Dsimtrain
88.3
56.7
46.8
22.5
32.7
4
40
44
2
DA [ 28 ]
Dsimtrain , Drealtrain
103.5
66.0
34.7
33.0
28.8
30
69
52
5
Ours
Drealtrain (Odometry only)
44.7
25.6
28.2
10.5
13.1
88
79
91
82
TABLE I: Mean absolute error MAE and Pearson correlation coefficient ρ on the real-world testing set Drealtest , 3 runs per row.
Fig. 5: Our self-supervised domain adaptation approach vs ground truth on Drealtest for the components of the gate pose ( x,y,z,ψ ).
Fig. 6: Input frames overlayed with gate skeleton (in orange) using the pose predicted by our approach. The model correctly locates the gate (1-3), while failures occur when a large portion of the gate is occluded and for shallow angles (4-5).
Fig. 7: Position (top) and orientation (bottom) performance measure with MSE of our approach with increasing amounts of fine-tune data. DA is trained on the full real-world dataset (60 sequences), while PencilNet and Zero-Shot are trained only on Dsimtrain .
Unsupervised domain adaptation (UDA) enables robust transfer of knowledge from simulated to real environments while exploiting a subset of unlabeled target data to improve real-world performance. Existing UDA methods for Object pose estimation often rely on global feature matching, multi-stage larger frameworks, or image translation pipelines, which tend to overlook the pose-specific information embedded in feature representations. To bridge this limitation, we introduce CAPLR that targets the adaptation of pose-sensitive features in localized regions, ensuring that domain alignment preserves the geometric cues essential for accurate pose estimation. CAPLR achieves UDA with three key components: (1) Efficient Cross-Domain Pairing strategy leveraging intermediate features to identify pose similar image pairs across domains without supervision; (2) Contrastive Alignment to perform feature alignment at localised regions in both intermediate and task-specific representations; and (3) Consistency-Based Pseudo-Label Refinement to improve reliability by encouraging stable target predictions. Extensive experiments demonstrate that CAPLR achieves state-of-the-art performance across multiple well-known object pose estimation benchmarks featuring diverse and challenging scenarios.
Remote pose estimation of noncooperative Unmanned Aerial Vehicles (UAVs) from imagery is critical, as they cannot be influenced or instrumented in advance. Deep-learning-based approaches offer a promising solution; however, their development is constrained by the cost and difficulty of acquiring large-scale real-world datasets with accurate pose labels. Synthetic imagery provides an alternative, but models trained on synthetic data must overcome the synthetic-to-real domain gap to generalize to real-world imagery. This work investigates the inherent synthetic-to-real generalization capability of a Vision Transformer (ViT)-based model for monocular UAV pose estimation. The proposed approach employs a self-supervised DINOv2 backbone and is trained exclusively on labeled synthetic imagery while being evaluated on labeled real-world imagery. Pose ambiguity-aware strategies are incorporated during training and inference to address ambiguities arising from the projection of a three-dimensional target onto a two-dimensional image plane and from target symmetries. An α-β filter is further integrated during inference to improve pose estimations. To assess the model under operational requirements, it is evaluated in terms of Mean Angular Error (MAE) and inference time, both before and after filtering, using a real-world dataset containing 77,077 labeled UAV images. Before filtering, the model achieves an MAE of 19.18∘ and an inference time of 13.25 ms, whereas after filtering, these values are 8.74∘ and 13.42 ms, respectively.
Continuous 6-DoF pose estimation is essential for autonomous UAV operations. Yet, existing visual odometry and SLAM methods accumulate drift and yield only relative, up-to-scale trajectories. Single-frame geo-localization, in turn, discards temporal continuity and remains too slow for real-time use. We present OrthoTrack, a training-free system that estimates continuous 6-DoF UAV trajectories using only publicly available orthophotos and surface models as a map prior. OrthoTrack matches keyframes against the orthophoto and lifts correspondences to metric 3D via the surface model. It then propagates these map-anchored correspondences to intermediate frames with optical flow, producing absolute, metrically scaled poses at every frame without GPS or post-hoc alignment. We also introduce the MovingDrone Dataset, a large-scale benchmark pairing photorealistic UAV sequences with dense 6-DoF ground truth and co-registered multi-modal geodata including multi-temporal orthophotos. On MovingDrone and real-world benchmarks, OrthoTrack runs in real time on a single GPU. It outperforms all baselines by a large margin, even those receiving oracle scale and alignment. By relying on publicly available geodata, OrthoTrack enables deployment to new regions without site-specific adaptation.
Oussema Dhaouadi, Zuria Bauer, Johannes Michael Meier +3
ETH Zurich · TU Munich · University of Cambridge +2