STATrack: A Target-Aware Fully Spiking Neural Network for Efficient UAV Tracking
Authors: Pengzhi Zhong, Jiwei Mo, Dan Zeng, Feixiang He, Shuiwang Li
Organizations: Guilin University of Technology, Guilin 541006, China · School of Artificial Intelligence, Sun Yat-sen University, Zhuhai 510275, China · School of Electronic Information, Central South University, Changsha, Hunan, 410083, China
Spiking Neural Networks (SNNs), characterized by their event-driven computation and low power consumption, have shown great potential for energy-efficient visual tracking on unmanned aerial vehicles (UAVs). However, existing SNN-based trackers often rely on costly event cameras, which limits their deployment on standard RGB-camera UAV platforms. To address this limitation, we propose STATrack, a fully spiking neural network framework for UAV visual tracking using only RGB inputs. To the best of our knowledge, this is the first study to explore fully spiking neural networks for RGB-based UAV visual tracking. Considering the challenge of preserving fine-grained target information in deep spiking representations under a limited number of time steps, we introduce Adaptive Mutual Information Maximization (AMIM). AMIM maximizes the statistical dependency between target-centered templates and their deep spiking representations, encouraging the spiking backbone to learn representations that better distinguish the target from surrounding backgrounds. We further develop a sample-difficulty-aware dynamic weighting strategy that adaptively adjusts the mutual-information constraint according to the relative localization difficulty of each training batch. Extensive experiments on four widely used UAV tracking benchmarks demonstrate that STATrack achieves state-of-the-art tracking performance with low theoretical energy consumption, highlighting its potential for energy-constrained UAV applications. Code is released at: https://anonymous.4open.science/r/STATrack.
Figures & tables
Fig. 1: Accuracy–Energy Comparison on the VisDrone2018 dataset. STATrack achieves 69.4% AUC with only 5.6 mJ theoretical inference energy, demonstrating a favorable accuracy–energy trade-off.
Fig. 2: Overview of the proposed framework. The architecture consists of an efficient SNN-based Transformer backbone for joint template–search feature learning, followed by a spiking head. (Bottom-left) Illustration of the proposed Adaptive Mutual Information Maximization (AMIM) module. (Bottom-right) Detailed structure of the adaptive dynamic weighting strategy used in AMIM. Note that {Z} denotes a batch of input samples, while {Z′} represents a randomly shuffled batch of Z .
Tracker
Source
UAVTrack112
UAVDT
VisDrone2018
UAV123
Average
Power
Param.
Prec.
Succ.
Prec.
Succ.
Prec.
Succ.
Prec.
Succ.
Prec.
Succ.
(mJ)
(M)
KCF [ 14 ]
TPAMI 15
40.2
26.6
57.1
29.0
68.5
41.3
52.3
33.1
54.5
32.5
-
-
fDSST [ 15 ]
TPAMI 17
56.8
39.1
66.6
38.3
69.8
51.0
58.3
40.5
62.9
42.2
-
-
ECO_HC [ 34 ]
CVPR 17
68.6
47.4
69.4
41.6
80.8
58.1
71.0
49.6
72.5
49.2
-
-
MCCT_H [ 41 ]
CVPR 18
63.4
43.6
66.8
40.2
80.3
56.7
65.9
45.7
69.1
46.6
-
-
ARCF [ 1 ]
ICCV 19
67.2
45.6
72.0
45.8
79.7
58.4
67.1
46.8
71.5
49.2
-
-
TABLE I: Comparison of STATrack with lightweight trackers in terms of precision (Prec.), success rate (Succ.), and energy consumption (mJ) on UAVTrack112, UAVDT, VisDrone2018, and UAV123. Red , blue , and green indicate the first, second, and third best results, respectively. Note that percentage symbols (%) are omitted for both precision and success rate values.
Fig. 3: Attribute-based success plots on the UAVDT dataset under two representative challenging attributes: background clutter and large occlusion.
Method
Source
Prec.
Succ.
Method
Source
Prec.
Succ.
Method
Source
Prec.
Succ.
STATrack
Ours
89.1
69.4
EVPTrack [ 47 ]
AAAI 24
84.5
65.8
SparseTT [ 48 ]
IJCAI 22
81.4
62.1
MambaLCT [ 49 ]
AAAI 25
88.1
65.5
ZoomTrack [ 50 ]
NIPS 23
81.4
63.4
OSTrack [ 19 ]
ECCV 22
84.2
64.8
HIPTrack [ 51 ]
AAAI 24
86.7
67.1
SeqTrack [ 52 ]
CVPR 23
85.3
65.8
SAOT [ 53 ]
ICCV 21
76.9
59.1
AQATrack [ 54 ]
CVPR 24
87.2
66.9
MAT [ 55 ]
CVPR 23
81.6
62.2
KeepTrack [ 56 ]
ICCV 21
84.0
63.5
ODTrack [ 57 ]
AAAI 24
85.6
64.8
ROMTrack [ 58 ]
ICCV 23
86.4
66.7
PrDiMP50 [ 59 ]
CVPR 20
79.4
59.7
TABLE II: Comparison results of STATrack with deep learning-based trackers on the VisDrone2018 dataset.
Fig. 4: Qualitative comparison of attention maps with and without AMIM. The top row shows the search images, while the middle and bottom rows show the attention maps generated by the baseline without AMIM and STATrack with AMIM, respectively.
Method
STATrack (Ours)
STATrack-S (Ours)
ORTrack -DeiT [ 7 ]
AVTrack -DeiT [ 3 ]
OSTrack -256 [ 19 ]
MambaLCT -256 [ 49 ]
EVPTrack -224 [ 47 ]
Power (mJ)
5.6
3.2
11.0
4.5–8.8
98.9
119.1
100.3
FPS (GPU)
118.4
156.7
226.4
260.3
65.4
57.4
74.3
FPS (CPU)
24.2
32.1
55.4
59.8
5.9
–
6.7
TABLE III: Evaluation of CPU and GPU Speed for different methods.
Components
UAVTrack112
UAVDT
VisDrone2018
UAV123
Average
MIM
ADW
Prec.
Succ.
Prec.
Succ.
Prec.
Succ.
Prec.
Succ.
Prec.
Succ.
-
-
80.3
67.7
78.5
61.2
82.5
65.0
86.1
67.0
81.9
65.2
✓
-
81.8
68.2
81.5
63.4
85.6
67.6
86.5
67.3
83.9
66.6
✓
✓
83.0
69.7
84.0
65.3
89.1
69.4
86.7
67.6
85.7
68.0
TABLE IV: Ablation results of different methods in STATrack on four UAV benchmarks. MIM refers to mutual information maximization with a fixed weighting scheme, while ADW denotes the proposed adaptive weighting strategy. Best results are shown in bold .
λbase
β
UAVTrack112
UAVDT
VisDrone2018
UAV123
Average
Prec.
Succ.
Prec.
Succ.
Prec.
Succ.
Prec.
Succ.
Prec.
Succ.
0.01
0.0
82.0
68.5
81.1
63.0
85.1
66.0
86.1
67.2
83.6
66.2
0.10
0.0
81.8
68.2
81.5
63.4
85.6
67.6
86.5
67.3
83.9
66.6
1.00
0.0
80.2
67.2
81.0
62.8
84.7
66.3
85.8
66.6
82.9
65.7
0.10
0.01
82.0
68.5
82.5
63.8
86.5
68.2
86.6
67.9
84.4
67.1
0.10
0.05
83.0
69.7
84.0
65.3
89.1
69.4
86.7
67.6
85.7
68.0
TABLE V: Parameter sensitivity analysis on four benchmarks. We investigate the impact of the base weight λbase and the dynamic adjustment amplitude β . The best results in each group are highlighted in bold .
Fig. 5: Qualitative comparison of STATrack with six state-of-the-art UAV trackers on four video sequences from UAVTrack112, UAV123, VisDrone2018, and UAVDT, namely swan , Car7 , S1607 , and uav0000088_00000_s .
Visual object tracking follows objects across video frames, maintaining their locations and identities despite motion, occlusion, and changing appearance.
With sparse and event-driven computation, spiking neural networks show great potential for achieving accurate and power-efficient UAV visual tracking. However, existing SNN-based trackers typically use spike firing rates only for power consumption and lack explicit optimization of actual spike activity. Moreover, regulating spike activity alone does not explicitly encourage stable target representations under partial observations and temporal appearance changes. We propose SBMVTrack, a fully spiking tracking framework that combines spike activity regulation with complementary multi-view representation learning. Specifically, SBMVTrack introduces Energy-Weighted Spike Budgeting (EWSB), which incorporates layer-wise computational costs when regulating spike firing rates and penalizing saturated activations, thereby reducing redundant spike computation. To further improve target representations under the spike budget constraint, we introduce Masked Multi-View Target Modeling (MVTM), which treats the initial template, online template, and search region as temporal views of the same target. By aligning target embeddings between masked and corresponding unmasked views and enforcing cross-view identity consistency, MVTM encourages robustness to missing local cues and temporal appearance changes. Experiments on four UAV benchmarks demonstrate competitive tracking performance with a 24.1% reduction in estimated power consumption relative to the baseline. On VisDrone2018, SBMVTrack achieves a success rate of 70.0%, exceeding SpikeTrack by 9.7 percentage points while reducing estimated power consumption by 45.7%. The source code will be released upon acceptance.
Pengzhi Zhong, Jiwei Mo, Haolun Li +4
College of Computer Science and Engineering, Guilin University of Technology, Guilin 541004, China · School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China
Spiking Neural Networks (SNNs) promise energy-efficient vision, but applying them to RGB visual tracking remains difficult: Existing SNN tracking frameworks either do not fully align with spike-driven computation or do not fully leverage neurons' spatiotemporal dynamics, leading to a trade-off between efficiency and accuracy. To address this, we introduce SpikeTrack, a spike-driven framework for energy-efficient RGB object tracking. SpikeTrack employs a novel asymmetric design that uses asymmetric timestep expansion and unidirectional information flow, harnessing spatiotemporal dynamics while cutting computation. To ensure effective unidirectional information transfer between branches, we design a memory-retrieval module inspired by neural inference mechanisms. This module recurrently queries a compact memory initialized by the template to retrieve target cues and sharpen target perception over time. Extensive experiments demonstrate that SpikeTrack achieves the state-of-the-art among SNN-based trackers and remains competitive with advanced ANN trackers. Notably, it surpasses TransT on LaSOT dataset while consuming only 1/26 of its energy. To our knowledge, SpikeTrack is the first spike-driven framework to make RGB tracking both accurate and energy efficient. The code and models are available at https://github.com/faicaiwawa/SpikeTrack.
Qiuyang Zhang, Jiujun Cheng, Qichao Mao +5
Tongji University · Nova University of Lisbon · Stockholm University +2
Given the real-time demands of UAV tracking, many methods simplify the backbone to reduce computation, but this often weakens feature representation and degrades performance in complex scenarios. To alleviate this issue, we propose EATrack, an efficient and asymmetric UAV tracking framework centered around a teacher-guided dual-branch distillation strategy that enhances the feature expressiveness of the lightweight student model. Specifically, EATrack investigates two complementary perspectives of knowledge transfer: spatially focused feature-level distillation that compensates for weakened representations by guiding the student to learn strong target representations, and prediction-level distillation that enhances spatial localization by learning the teacher's capability for accurate target localization. Furthermore, to enhance robustness against appearance variations, we introduce a fine-grained target-aware distillation strategy that selectively transfers the teacher's target modeling capacity to the student. A temporal adaptation module is incorporated at inference to enhance robustness over time. Experiments on five UAV benchmarks demonstrate that EATrack achieves a favorable balance between accuracy and speed. Code: https://github.com/GXNU-ZhongLab/EATrack
Hongtao Yang, Bineng Zhong, Qihua Liang +4
Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin, 541004, China · Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin, 541004, China · Nanjing University of Science and Technology +1