HAT: Hypothesis-Anchored Tracking for Video Monocular Spacecraft Pose Estimation

André Lopo; Atabak Dehban; Rodrigo Ventura

cs.CVSep 18, 2026

HAT: Hypothesis-Anchored Tracking for Video Monocular Spacecraft Pose Estimation

Authors: André Lopo, Atabak Dehban, Rodrigo Ventura

Organizations: ISR, Instituto Superior T´ecnico, Universidade de Lisboa, Portugal

Abstract

Monocular 6-DoF pose estimation of non-cooperative targets is important for on-orbit servicing and debris removal. A single-image estimator can confuse near-symmetric spacecraft orientations, and tracking can preserve an incorrect pose. We present Hypothesis-Anchored Tracking (HAT), a causal framework that uses inter-frame motion to select among competing CAD-based pose hypotheses before alignment and fusion. Rather than independently choosing the highest-scoring hypothesis in each image, HAT retains competing orientation histories and selects a pose to anchor the relative trajectory estimated by monocular SLAM. Sparse anchors and pose fusion provide per-frame estimates after initialization without revising past outputs. The method requires only a calibrated RGB sequence, a metric CAD model, and target image regions, which can be supplied by detection or segmentation. The pretrained pose and SLAM networks require no target-specific training or fine-tuning. We evaluate two versions, Mega-HAT and Pico-HAT, using MegaPose and PicoPose, on SPARK-2024, SwissCube and SHIRT, with YCB-Video assessing performance outside the space domain. Using one temporal configuration per method, the arithmetic means of the four dataset-wise comparisons show 9.4% lower mean pose error and 3.76 times the sustained input FPS for Mega-HAT relative to independent MegaPose, and 23.9% lower mean pose error and 2.42 times the FPS for Pico-HAT relative to independent PicoPose. Mega-HAT ablations on SPARK and an offline reference examine component contributions and the effect of revising past estimates.

Figures & tables

Parameter group	Mega-HAT	Pico-HAT
Absolute anchors	gap 4, $K=10$ , five refiner iterations	gap 6, $K=5$ , PicoPose stages 1–3, no MegaPose refiner
Decoder	$\alpha=3$ , $\mu=0.3$	$\alpha=100$ , $\mu=0.3$
Fusion	$w=0.002$ , $D=75^{\circ}$ warm-up 4, Huber scale 0.3	$w=0.1$ , $D=30^{\circ}$ warm-up 4, Huber scale 0.3

TABLE I: Shared temporal configurations for the four-dataset benchmark. Each method uses the same settings across datasets.

Configuration	$E_{t}\,(\mathrm{m})$	$E_{q}^{\circ}$	$E_{p}$
Mega-HAT , causal	0.442	10.82	0.2355
Mega-HAT , all stages offline	0.430	9.66	0.2125
Mega-HAT , offline + dense refinement	0.348	9.52	0.2016
Pico-HAT , causal, tuned	0.123	4.19	0.0886
Pico-HAT , offline, tuned	0.104	2.81	0.0618
Motion-aware ViT [ 12 ] , offline	0.137	4.69	0.0990

TABLE II: Causal and offline HAT on synthetic SPARK, with tuned temporal settings.

Fig. 2: Rotation error and signed depth error $\hat{T}_{z}-T_{z}$ of appearance-selected MegaPose hypotheses on SPARK-2024, after refinement and before temporal selection. Markers show medians in $2\,\mathrm{m}$ bins of ground-truth depth, and shading shows interquartile ranges. Negative depth errors indicate underestimated depth.

Configuration	$E_{t}\,(\mathrm{m})$	$E_{q}^{\circ}$	$E_{p}$
MegaPose tracking	0.471	28.13	0.5387
Keyframe anchors $+$ SLAM fusion	0.490	18.77	0.3788
$+$ Viterbi basin selection (Stage 3)	0.469	12.20	0.2619
$+$ size-weighted unary	0.451	11.63	0.2504
$+$ disagreement gate ( $D{=}75^{\circ}$ )	0.442	10.82	0.2355

TABLE III: Cumulative component ablations of Mega-HAT on SPARK, causal throughout.

Method	$E_{p}$	SD	$E_{t}$ ( $\mathrm{m}$ )	$E_{q}$ ( ∘ )	$\mathrm{FPS}$
MegaPose (per frame)	0.413	0.808	0.469	21.0	0.82
PicoPose	0.098	0.344	0.112	4.8	2.74
GigaPose + ref.	0.410	0.811	0.333	21.5	1.66
RGBTrack	2.169	1.065	1.804	111.0	2.34
SRT3D + init.	0.650	0.943	1.311	29.1	131.61
Mega-HAT	0.325	0.648	0.463	15.9	2.27

TABLE IV: Means and sample SD summarize each dataset. Bold denotes the best result in each column per dataset (lowest errors and SD, highest FPS). Italics denote video baselines; our methods appear below the divider. “+ ref.” adds five MegaPose refinement iterations.

\mathrm{FPS}

includes decoding after warm-up and excludes detector generation and loading. Mega-HAT uses the shared temporal configuration. Evaluation and timing protocols are specified in Sec. VI .

Fig. 3: Mean

E_{p}

(bars), one sample SD (upper whiskers), and sustained input

\mathrm{FPS}

(diamonds). Our implementations are colored, with larger outlined diamonds. Baselines are gray. Numerical labels mark whiskers above 4 and rates above

20\,\mathrm{FPS}

. Throughput is measured on separate stream subsets. Table IV specifies labels and evaluation scope.

Object Pose Estimation

Object pose estimation recovers an object's 3D position and orientation from images, depth, point clouds, or contact sensing.

See all 187 papers

TALON: Token-Aligned Lightweight Adapters for 6-DoF Spacecraft Pose Estimation
May 29, 2026Abid Ali, Nidhal Eddine Chenni, Arunkumar Rathinam +1Vision Foundation Model Adaptation Object Pose Estimation
DreamSat-Pose: Spacecraft Pose Estimation from Single-View 3D Reconstructions and Learned 2D-3D Feature Matching
Jul 15, 2026Josiane Uwumukiza, Jocelyn Zhao, Giovanni Lavezzi +5Relative Pose Estimation Spacecraft Rendezvous and Proximity Operations
Undergraduate Student, Wellesley College, Wellesley, MA 02481, USA · Undergraduate Student, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Research Scientist, Massachusetts Institute of Technology, Cambridge, MA 02139, USA +5
Vision Foundation Models with Synthetic-Only Training for Monocular Spacecraft Pose Estimation
Sep 22, 2026John Church, Vazghen NikolianVision Foundation Model Adaptation Object Pose Estimation
Both authors contributed equally to this work; author order is alphabetical.

May 29, 2026J/K move · Enter open · S save

TALON: Token-Aligned Lightweight Adapters for 6-DoF Spacecraft Pose Estimation

Abid Ali, Nidhal Eddine Chenni, Arunkumar Rathinam, Djamila Aouada

Vision Foundation Model Adaptation Object Pose Estimation Spacecraft Pose Estimation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Monocular 6-DoF spacecraft pose estimation methods predominantly process individual frames, discarding the temporal information present in an image sequence acquired during spacecraft manoeuvres. Few temporal approaches require full backbone fine-tuning or auxiliary optical flow networks, risking catastrophic forgetting or increasing computational cost, respectively. We propose TALON (Token-Aligned Lightweight adapters for Orbital Navigation): spatiotemporal 3D adapters injected before the self-attention layers of a frozen ViT vision transformer, combined with a patch-token alignment loss that geometrically grounds the adapted features to keypoint structure through a prototype-conditioned KL-divergence objective. Pre-attention placement allows the frozen attention to reason over temporally enriched tokens, achieving stronger performance with a single adapter per block than post-attention alternatives. The alignment loss shapes the intermediate representations so that each keypoint induces a spatially precise activation in the token field, while the framework adds less than 5% parameters to the frozen backbone. On SPADES dataset, TALON reduces the pose error by 50% over the prior state-of-the-art, and on SwissCube dataset it surpasses the prior best by 21.8% in ADD-0.1d accuracy. Zero-shot cross-domain evaluation from sim-to-real on SPARK real data reduces pose error by 4.7x, and ablations characterise the role of adapter depth across in-domain and cross-domain settings.

Abstract

Object Pose Estimation

Explore similar work

TALON: Token-Aligned Lightweight Adapters for 6-DoF Spacecraft Pose Estimation

DreamSat-Pose: Spacecraft Pose Estimation from Single-View 3D Reconstructions and Learned 2D-3D Feature Matching

Vision Foundation Models with Synthetic-Only Training for Monocular Spacecraft Pose Estimation