Organizations: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, 100094, China · Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, Chinese Academy of Sciences, Beijing, 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, 101408, China
Satellite video object tracking (SVOT) remains fundamentally challenging due to texture scarcity, arbitrary rotation, aspect ratio changes, and severe occlusions. While recent state-of-the-art trackers excel in general scenarios, their reliance on rich appearance details or rigid spatial matching mechanisms leads to significant performance degradation in the satellite domain. To bridge this gap, we propose SiamGM, a real-time spatial-temporal unified tracking framework tailored for satellite videos. Instead of conventionally stacking modules, we synergize geometric-topological perception with temporal-kinematic prior, addressing the core limitations of SVOT. Spatially, we closely couple a Topological Attention Module (TAM) with a Geometry-Constrained Label Assignment (GCLA) method during the training phase, where a Local Graph Propagation (LGP) mechanism enforces neighborhood-consistent template-search correspondences. This joint paradigm establishes fine-grained structural correspondences and explicitly suppresses surrounding background noise. Temporally, rather than passively relying on visual cues, we propose an Online Motion Model Refinement (OMMR) strategy during the tracking phase. Gated by an online self-calibrated confidence indicator, OMMR adaptively fuse long-term stable and short-term instantaneous motion estimation for robust tracking. Evaluations on two challenging SatSOT and SV248S benchmarks confirm that SiamGM outperforms not only recent Siamese-based satellite trackers but also state-of-the-art Transformer trackers in both precision and success metrics. Notably, this highly unified architecture introduces only minor computational overhead, enabling real-time tracking at 130 frames per second (FPS).
Figures & tables
Figure 1: Visual illustrations of the inherent challenges in satellite videos. (a) Tiny objects lack distinct texture features. (b) Targets often undergo arbitrary rotations, causing misalignment between search and template features. (c) Non-rigid targets undergo non-linear structural deformations, leading to large aspect ratio change and severe background noise within horizontal bounding boxes. (d) Severe partial or full occlusions easily lead to tracking drift.
Figure 2: Overall architecture of the proposed SiamGM framework. The Topological Attention Module (TAM) is integrated to establish fine-grained, topology-consistent correspondences via the Local Graph Propagation (LGP) mechanism over the feature grid graphs. Geometry-Constrained Label Assignment (GCLA) is employed during the training phase to explicitly supervise spatial ambiguities. The Online Motion Model Refinement (OMMR) strategy is deployed as a post-processing stage to dynamically rectify trajectory bounds based on historical motion information. CGR denotes the combination of convolution, group normalization, and ReLU activation, while PWC denotes point-wise convolution.
Figure 3: Visualization of (a) 1D multi-gamma centerness comparison and (b) 2D centerness distribution in which the aspect ratio is set to 3.
Figure 4: Visualization of nPSR curves for (a) two car targets and (b) two ship targets in SatSOT Dataset. The blue curve represents the nPSR values in normal scenarios, while the red curve corresponds to occlusion scenarios.
Figure 5: Illustration of the OMMR process.
Trackers
Source
Features
SatSOT
SV248S
P-5. (%)
P-20. (%)
S. (%)
P-5. (%)
P-20. (%)
S. (%)
KCF
TPAMI2014
HOG
55.3
62.2
40.0
73.1
76.7
46.2
ECO-HC
CVPR2017
HOG+CN
58.6
66.4
43.8
71.1
77.2
44.0
SiamCAR
CVPR2020
CF
60.0
67.8
42.7
71.8
77.1
41.4
Mixformer-L
CVPR2022
TF
46.7
52.8
37.8
33.9
37.4
14.9
SeqTrack
CVPR2023
CF+TF
52.3
56.6
40.1
41.7
43.4
16.6
Table 1: Overall Performance on SatSOT and SV248S. HOG is Histogram of Oriented Gradients, CN is Color Names, CF is Convolutional Feature, and TF is Transformer Feature. The P-5. is Precision Rate with CLE Threshold of 5 Pixels, P-20. of 20 Pixels, and the S. is Success Rate. The Red Represents the Best Performance, Blue Represents the Second-place, and Green Represents Third-place. The values in parentheses indicate the absolute performance gains over the baseline SiamCAR.
Figure 6: Precision vs. FPS Comparison on SatSOT benchmark. The horizontal axis represents the tracking speed, while the vertical axis represents the precision score.
Method
TAM
GCLA
OMMR
P.
N.P.
S.
FPS
SiamCAR
×
×
×
60.0
70.8
42.7
142
SiamCAR-TAM
✓
×
×
61.9
73.3
44.2
136
SiamCAR-GCLA
×
✓
×
61.1
71.9
43.5
142
SiamCAR-M
×
×
✓
63.3
74.1
44.6
132
SiamCAR-G
✓
✓
×
63.1
74.2
45.2
136
SiamGM
✓
✓
✓
65.9
76.6
47.7
130
Table 2: Comprehensive Ablation Studies on SatSOT. Bold Highlights the Best Results, and Underline Marks the Second-place Results. All three metrics are reported as percentages.
γ
P. (%)
N.P. (%)
S. (%)
0.25
60.5
71.4
43.0
0.5
61.1
71.9
43.5
0.75
59.8
71.5
42.9
1.0
60.2
71.1
42.5
Table 3: Ablation Study on γ Parameter in the GCLA Method on SatSOT.
Metric
P. (%)
N.P. (%)
S. (%)
FPS
Reliability Coeff
63.1
73.7
45.9
128
Location Map
63.0
73.6
44.5
122
nPSR
65.9
76.6
47.7
130
Table 4: Ablation Study on the Gating Confidence of OMMR on SatSOT.
Figure 7: Attribute-based performance comparisons evaluating SiamGM, its ablated variants, and recent SOTA SNN-based tracker on the SatSOT dataset via radar charts. Our SiamGM (in red) exhibits distinct advantages and robust generalization in most challenging scenarios.
Attribute
Description
Sequences
BC
Background Clutter
45
IV
Illumination Variation
3
LQ
Low Quality
13
ROT
Rotation
56
POC
Partial Occlusion
34
FOC
Full Occlusion
12
Table 5: Attributes for Performance Analysis in SatSOT.
Figure 8: Attribute-based performance comparison of SiamGM, its ablated variants, and state-of-the-art tracker on the SatSOT dataset. The twelve sub-figures present precision, normalized precision, and success plots under four representative challenging scenarios (IV+DEF, ROT, POC+FOC, TO) including six attributes. SiamGM consistently demonstrates superior performance, especially under severe deformation, rotation, occlusion, and tiny object scenarios.
Figure 9: Visualization of the template saliency maps for the plane_02 sequence, which contains both IV and ROT attributes. The consistent focus on geometrically salient regions (e.g., wings and fuselage) across different orientations validates the effectiveness of TAM in enhancing geometric structural perception.
Figure 10: Visualization of the classification and centerness maps during the tracking process on the train_08 sequence. The top two rows show the classification map outputs, while the bottom two rows show the centerness map outputs. For each type of map, the first column presents the results from the baseline model, and the second column shows the outputs from SiamCAR-GCLA. The ground truth is highlighted with a green bounding box.
Figure 11: Visualization of the tracking results for three sequences ( car_35 , car_58 , and car_61 ) under occlusion and tiny object scenarios. The first two rows correspond to POC+TO and FOC+TO attributes, while the third row corresponds to FOC. The red and blue bounding boxes represent the predictions with OMMR, while the green box indicates the ground truth.
Visual object tracking follows objects across video frames, maintaining their locations and identities despite motion, occlusion, and changing appearance.
Traditional visual object tracking (VOT) methods typically rely on task-specific supervised training, limiting their generalization to unseen objects and challenging scenarios with distractors, occlusion, and nonlinear motion. Recent vision foundation models, exemplified by SAM 2, learn strong video understanding priors from large-scale pretraining and offer a promising foundation for building more robust and generalizable trackers. However, directly applying SAM 2 to VOT remains suboptimal, as it does not explicitly model target motion dynamics or enforce geometric and semantic consistency across frames, both of which are essential for reliable tracking. To address this issue, we propose SAMOSA, a new tracking framework that adapts SAM 2 to complex VOT scenarios by explicitly leveraging motion, geometry, and semantic cues. Specifically, we introduce a lightweight nonlinear motion predictor to model target dynamics and guide mask selection as well as memory filtering. We further exploit semantic cues to detect target shifts and recover from tracking failures, while geometric cues are incorporated as structural constraints to improve tracking stability. In this way, SAMOSA bridges the gap between the implicit video understanding prior of SAM 2 and explicit tracking-oriented modeling. Extensive experiments show that SAMOSA consistently outperforms state-of-the-art SAM 2--based approaches on general benchmarks, demonstrates stronger generalization than supervised VOT methods, and achieves substantial gains on anti-UAV datasets, which typify complex nonlinear motion scenarios. Our code is available at https://github.com/DurYi/SAMOSA.
Deyi Zhu, Yuji Wang, Yong Liu +4
Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China · Department of Automation, Tsinghua University, Beijing 100084, China
Satellite video object detection (SVOD) for oriented and fine-grained objects plays an important role in satellite applications. Most existing SVOD methods only focus on one or a few coarse-grained categories of moving objects and represent objects with horizontal bounding boxes. They have difficulty extracting complete, accurate, and consistent information about objects in whole satellite videos. In this paper, we propose a satellite video object detection framework based on Temporal Consistency Learning (TCL). TCL adeptly detects oriented and fine-grained objects by leveraging the rich temporal contexts within satellite videos. The framework integrates three key modules: temporal and fine-grained feature aggregation (TFA), structure encoding (SE), and temporal consistency constraint (TCC). TFA and TCC modules facilitate consistent representation learning across frames, while the SE module encodes both appearance and structural information for precise fine-grained recognition. Experimental results on the SAT-MTB benchmark dataset demonstrate TCL's superior performance, achieving a new state-of-the-art oriented and fine-grained detection accuracy of 47.7% mAP--a 4.8% improvement over the baseline. Furthermore, our TCL framework readily accommodates existing image-based detectors, leading to enhanced detection accuracies.
Weilong Guo, Shengyang Li, Yanfeng Gu
Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences, Beijing 100094, China · Key Laboratory of Space Utilization, Chinese Academy of Sciences, Beijing 100094, China · University of Chinese Academy of Sciences, Beijing 100049, China +1
Multi-object tracking in satellite video underpins persistent monitoring, yet offline-tuned trackers lack the onboard intelligence and efficiency to adapt within an observation of tens of seconds. The root is that adaptation still means relabelling and retraining the detector, which cannot run on board in time. Tuning the tracker configuration strongly controls accuracy without touching the network, yet search needs labelled or proxy evaluations, and a language agent costs seconds and tokens per decision. We propose a dual-speed agentic framework pairing a fast instinct compiled offline with a slow reason serving the mission. Its algorithm, Arbitration of Unlabelled Shortlists from Prior Experience, or AUSPEX, learns a covering shortlist and a label-free ranker from labelled source tasks and commits one configuration per sequence. Its system, a Cascaded Onboard Runtime with Verified Uplinked Skills, or CORVUS, cascades reflex, skill, and agent layers, so that a verified skill commits 2.5 s into each observation while a language agent serves natural- language missions. On SatMTB and SatVideoDT with three recent detectors, AUSPEX raises IDF1 by 2.84 to 2.94 points and MOTA by 3.21 to 3.29 points on average without target labels, balances accuracy and labels best among sixteen methods, and loses at most 0.28 IDF1 points at 1.25 tracking passes. CORVUS executes four missions with execution accuracy up to 0.99 and target recall at the perception ceiling, spends no token on the routine decision, runs on Jetson Xavier NX, and hosts fifteen language models from six families without altering any commitment, demonstrating its extensibility and generalization.
Fan He, Baining Wu, Yan Li +1
State Key Laboratory of Integrated Chips and Systems, Fudan University, Shanghai 200433, China