Recurrent Vision Transformers for Event-Based Spacecraft Pose Estimation Under Slow Relative Motions

Sofia McLeod; Mohsi Jawaid; Carl Seubert; Tat-Jun Chin

cs.CVOct 19, 2026

Recurrent Vision Transformers for Event-Based Spacecraft Pose Estimation Under Slow Relative Motions

Authors: Sofia McLeod, Mohsi Jawaid, Carl Seubert, Tat-Jun Chin

Organizations: AI for Space Group, Adelaide University · Australasian Space Innovation Institute

Abstract

Vision-based object pose estimation is vital for conducting spacecraft rendezvous and proximity operations (RPO). Recent research has demonstrated the robustness of event cameras against the harsh lighting conditions that often occur in orbit, thanks to the extremely high dynamic range of event sensors. However, event-based spacecraft pose estimation (EBSPE) suffers when the relative motion between the chaser and target is slow, since the event rate drops significantly, leading to data with low signal-to-noise ratio. Unfortunately, for safety reasons, real RPO trajectories often involve slow relative motions (e.g., down to 1 mm/s), and operating under low illumination exacerbates the problem. Addressing slow relative motions is thus crucial to raise the practicality of EBSPE. To this end, we propose using recurrent vision transformers (RVT) for EBSPE. The key idea is to exploit recurrent memory to capture longer-term information across sparse event streams, and to learn the temporal attention span from full trajectories rather than fixing the time window. We developed an RVT-based heatmap regressor to produce 2D-3D correspondences for pose estimation, which was validated on real event data collected under slow relative motions representative of RPO trajectories. Results indicate the superior pose estimation accuracy and stability of our RVT-based method, relative to previous per-event-frame-based or fixed-time-window EBSPE methods.

Figures & tables

Fig. 1 : Footage of the final docking phase ( $10$ m to target) in Northrop Grumman’s MEV-2 mission [ 9 ] . Circles mark the docking ring of the target satellite (Intelsat 10-02) across sequential frames, illustrating the slow, linear approach.

Panel of Figure 2 — Fig. 2 : Comparing event frames accumulated from event streams generated under two different relative speeds (top: $5$ cm/s, bottom: $1$ mm/s) across increasing batching window sizes. Due to the lower SNR in the slower sequence ( $1$ mm/s), smaller batching windows create frames with little structure, while larger batching windows accumulate significantly more noise.

Fig. 3 : Proposed RVT-P pipeline. S1–S4 output feature maps at 1/4, 1/8, 1/16, and 1/32 input resolution; P1–P3 are the output feature maps of the FPN, in order of lowest to highest resolution. Fig. 4 depicts the RVT block.

Fig. 4 : RVT block as proposed by [ 16 ] .

Panel of Figure 5 — Fig. 5 : (a) DRACO space-grade event camera. (b) Laboratory setup used to collect E-PROSPER. (c) A plot of traj1-6 with example sequence durations in the legend, with a to-scale Tango model shown. The camera orientations were fixed w.r.t. the world frame, and the coloured lines indicate the linear approach trajectories executed. Event data was recorded only when the target was in the FOV (dotted black lines indicate the line of sight of the plotted camera pose).

Panel of Figure 6 — Fig. 6 : Sample $50$ ms event frames from E-PROSPER, with ground-truth (green) and RVT-P (red) poses.

Method	$\uparrow$ $\Xi$	$\uparrow$ $\Xi_{50}$	$\uparrow$ $\Xi_{75}$	$\downarrow$ $e_{t}(\text{m})$		$\downarrow$ $\bar{e}_{t}[\%]$		$\downarrow$ $e_{r}[^{\circ}]$		$\downarrow$ $e_{p}$
				Mean	Median	Mean	Median	Mean	Median	Mean	Median
FRESH
EF	0.202	0.445	0.110	1.86e6	0.152	2.18e6	0.214	91.2	99.0	2.18e6	1.27e2
RVT-P	0.000	0.000	0.000	6.91e9	0.572	1.03e10	0.759	1.27e2	1.34e2	1.03e10	1.37e2
E-PROSPER
EF	0.527	0.999	0.397	0.018	0.011	0.025	0.015	7.905	3.119	7.930	3.137

TABLE I : OD and pose performance of EF and RVT-P on FRESH and E-PROSPER.

Speed (m/s)	Method	$\downarrow$ $e_{t}(\text{m})$	$\downarrow$ $\bar{e}_{t}[\%]$	$\downarrow$ $e_{r}[^{\circ}]$	$\downarrow$ $e_{p}$	$s$
$0.05$	EF	0.009	0.013	4.301	4.314	18
$0.05$	RVT-P	0.007	0.009	3.483	3.492	18
$0.01$	EF	0.010	0.014	4.533	4.546	18
$0.01$	RVT-P	0.005	0.007	3.423	3.431	18
$0.005$	EF	0.011	0.015	4.220	4.235	18
$0.005$	RVT-P	0.005	0.007	3.597	3.604	18

TABLE II : Mean pose performance across approach speeds.

Lighting	Method	$\downarrow$ $e_{t}(\text{m})$	$\downarrow$ $\bar{e}_{t}[\%]$	$\downarrow$ $e_{r}[^{\circ}]$	$\downarrow$ $e_{p}$	$s$
reg	EF	0.009	0.013	3.472	3.485	30
reg	RVT-P	0.007	0.009	2.682	2.692	30
low	EF	0.021	0.029	9.681	9.710	30
low	RVT-P	0.006	0.008	4.037	4.045	30
dark	EF	0.025	0.034	11.379	11.413	30
dark	RVT-P	0.009	0.012	6.468	6.480	30

TABLE III : Mean pose performance across different lighting.

Trajectory	Method	$\downarrow$ $e_{t}(\text{m})$	$\downarrow$ $\bar{e}_{t}[\%]$	$\downarrow$ $e_{r}[^{\circ}]$	$\downarrow$ $e_{p}$	$s$
Traj-seen	EF	0.017	0.024	6.382	6.406	60
Traj-seen	RVT-P	0.004	0.006	0.457	0.462	60
Traj-unseen	EF	0.020	0.028	13.819	13.848	30
Traj-unseen	RVT-P	0.019	0.027	19.802	19.829	30

TABLE IV : Average pose performance on trajectories with different training visibility.

Fig. 7 : Pose performance on traj1-dark-0.001ms .

Method	$\downarrow$ $e_{t}(\text{m})$	$\downarrow$ $\bar{e}_{t}[\%]$	$\downarrow$ $e_{r}[^{\circ}]$	$\downarrow$ $e_{p}$	$s$
EF	0.017	0.026	7.741	7.767	45
RVT-P	0.005	0.008	4.126	4.135	45
RVT-P-FW	0.010	0.016	4.840	4.855	45

TABLE V : Average pose performance on Tango.

Panel of Figure 13 — Fig. 8 : Average pose performance on Tango across speeds.

Spacecraft Pose Estimation

Spacecraft pose estimation recovers a spacecraft’s position and orientation from images and other sensor data for navigation, rendezvous, and servicing.

See all 19 papers

Apr 5, 2026

Efficient Onboard Spacecraft Pose Estimation with Event Cameras and Neuromorphic Hardware

Reliable relative pose estimation is a key enabler for autonomous rendezvous and proximity operations, yet space imagery is notoriously challenging due to extreme illumination, high contrast, and fast target motion. Event cameras provide asynchronous, change-driven measurements that can remain informative when frame-based imagery saturates or blurs, while neuromorphic processors can exploit sparse activations for low-latency, energy-efficient inferences. This paper presents a spacecraft 6-DoF pose-estimation pipeline that couples event-based vision with the BrainChip Akida neuromorphic processor. Using the SPADES dataset, we train compact MobileNet-style keypoint regression networks on lightweight event-frame representations, apply quantization-aware training (8/4-bit), and convert the models to Akida-compatible spiking neural networks. We benchmark three event representations and demonstrate real-time, low-power inference on Akida V1 hardware. We additionally design a heatmap-based model targeting Akida V2 and evaluate it on Akida Cloud, yielding improved pose accuracy. To our knowledge, this is the first end-to-end demonstration of spacecraft pose estimation running on Akida hardware, highlighting a practical route to low-latency, low-power perception for future autonomous space missions.

Arunkumar Rathinam, Jules Lecomte, Jost Reelsen +3

University of Luxembourg · Fortiss GmbH, Munich, Germany · Technical University of Munich, Munich, Germany +1

Spacecraft Pose Estimation Relative Pose Estimation Event-Based Vision Neuromorphic Computing

Ask PDF

Oct 27, 2026

Polarity--Time Interaction and Multi-Level Reliability for Event-Based Spacecraft Pose Estimation

Sparse event responses produce uneven keypoint errors whose effects on spacecraft pose depend on correspondence geometry and the reliability of successive observations. We present a reliability-aware framework connecting polarity--time localization, geometric recovery, event--CAD refinement, and trajectory optimization. A three-branch representation combines positive events, negative events, and recent-event timing through cross-branch attention and gating, followed by spatiotemporal aggregation. An ordered semantic-point head estimates relative correspondence precision for geometrically constrained pose recovery. Event-edge and CAD-projection agreement then guide local pose refinement under keypoint-geometry safeguards. At the sequence level, a ten-dimensional descriptor combines heatmap shape, reprojection consistency, and geometric conditioning to set time-aligned observation scales. On the official SPADES challenge test set, the complete system achieves a combined pose score of 0.0474, approximately 5.4% below PASAN+IRIS. Validation ablations reduce the score from 0.0772 for base localization with uniform PnP to 0.0514 for the complete cascade; on identical single-frame observations, quality weighting reduces the score from 0.0549 with constant-weight optimization to 0.0514.

Yonglong Zhang, Zongwu Xie, Yang Liu

Spacecraft Pose Estimation Camera Pose Estimation Event-Based Vision

Ask PDF

Jun 10, 2026

Precision-Aware Illumination-Disentangled Vision Transformer for Spacecraft 6D Pose Estimation

Vision sensors provide a lightweight solution for spacecraft proximity operations, but monocular spacecraft 6D pose estimation remains difficult under illumination variation, specular reflection, shadowing, weak texture, and background interference. These factors make local visual evidence spatially unreliable and can destabilize pose regression. This article proposes a Precision-Aware Illumination-Disentangled Vision Transformer (PAID-ViT) for robust spacecraft pose estimation.The proposed model separates pose-relevant structure tokens from illumination-sensitive appearance tokens, estimates patch reliability before pose aggregation, and uses foreground mask supervision to preserve silhouette cues. A parameter-free geometric recovery module converts normalized crop coordinates, log-depth, and a continuous 6D rotation representation into camera-frame rotation and translation. Experiments on SPEED+ V2, the SPEED+ validation/lightbox/sunlamp evaluation configuration used in this study, suggest that PAID-ViT reduces translation error and improves robustness in the challenging sunlamp domain, while ablation studies support the complementary roles of illumination disentanglement, reliability-aware token aggregation, mask supervision, and training-side regularization.

Zongwu Xie, Yifan Yang, Yonglong Zhang +3

School of Mechatronics Engineering, Harbin Institute of Technology, Harbin 150001, China

Vision Transformer Object Pose Estimation Spacecraft Pose Estimation Invariant Representation Learning

Ask PDF

Abstract

Spacecraft Pose Estimation

Explore similar work

Efficient Onboard Spacecraft Pose Estimation with Event Cameras and Neuromorphic Hardware

Polarity--Time Interaction and Multi-Level Reliability for Event-Based Spacecraft Pose Estimation

Precision-Aware Illumination-Disentangled Vision Transformer for Spacecraft 6D Pose Estimation