LoopTrack: A Simple Baseline for Parameter-Efficient Transformer Tracking

Liang Peng; Chenxiao Li; Libo Zhang; Xingping Dong; Heng Fan

cs.CVSep 27, 2026

LoopTrack: A Simple Baseline for Parameter-Efficient Transformer Tracking

Authors: Liang Peng, Chenxiao Li, Libo Zhang, Xingping Dong, Heng Fan

Organizations: School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University · University of North Texas · Institute of Software, Chinese Academy of Sciences

Abstract

Current Transformer-based tracking methods typically stack multiple Transformer blocks with separate parameters to model interactions between the target template and the search region for target localization. These trackers often incur substantial parameter overhead from stacked blocks, making their deployment on resource-limited devices difficult. To address this, we propose a parameter-efficient Transformer tracking framework, dubbed LoopTrack, which repeatedly applies a set of Transformer blocks with shared parameters to interact features in a looped architecture for tracking, significantly reducing the number of parameters. To further exploit target cues, we present two lightweight designs, including target-aware looping (TAL) and gated target memory (GTM). The former applies intermediate target information generated by one loop to guide feature interaction in the subsequent loop, enabling progressive feature refinement, while the latter maintains a compact memory across frames, which is incorporated into the loop process to provide long-term information to the tracker, mitigating temporal drift in tracking. Compared to existing Transformer trackers, LoopTrack enables multiple rounds of feature interaction with fewer model parameters, making it resource-friendly for deployment. In extensive experiments on multiple datasets, LoopTrack shows a favorable accuracy-parameter trade-off. In particular, our LoopTrack $_{\rm One}$ , with a single shared Transformer block, achieves 66.2% SUC score on LaSOT with only 3.4M parameters, while LoopTrack $_{\rm Three}$ , using three shared blocks, achieves 69.3% SUC score with 6.4M parameters, surpassing existing parameter-efficient tracking methods with comparable or larger model size. With LoopTrack, we aim to establish a simple yet strong baseline for parameter-efficient Transformer tracking. Our code and models will be released.

Figures & tables

Figure 1: Overview of LoopTrack. (a) Input template and search region. (b) Target-Aware Looping (TAL) for recurrent feature refinement. (c) Gated Target Memory (GTM) for cross-frame target modeling. (d) Tracking head for target localization.

Model	Blocks	$M$	Params (M)	MACs (G)	FPS
LoopTrack ${}_{\text{One}}$	$[6]$	1	3.4	3.79	153
LoopTrack ${}_{\text{Two}}$	$[5,6]$	2	4.9	6.70	119
LoopTrack ${}_{\text{Three}}$	$[4,5,6]$	3	6.4	9.62	98

Table 1: Model size and efficiency at

K=8

Method	Efficiency		LaSOT			LaSOT ext			TrackingNet			GOT-10k
	Params(M)	FPS	SUC	$P_{\mathrm{N}}$	$P$	SUC	$P_{\mathrm{N}}$	$P$	SUC	$P_{\mathrm{N}}$	$P$	AO	$\mathrm{SR}_{0.5}$	$\mathrm{SR}_{0.75}$
Compression
CompressTracker-4 ( Hong et al., 2025 )	35.4	228	66.1	75.2	70.6	45.7	–	50.8	82.1	87.6	80.1	–	–	–
CompressTracker-6 ( Hong et al., 2025 )	49.8	162	67.5	77.5	72.4	46.7	–	52.5	82.9	87.8	81.5	–	–	–
CompressTracker-8 ( Hong et al., 2025 )	63.7	127	68.4	78.0	73.1	47.2	–	53.1	83.3	88.0	81.9	–	–	–
FARTrack-Pico ( Wang et al., 2026 )	2.8	343	58.6	67.1	59.6	41.8	–	–	75.6	81.3	70.5	62.8	72.6	50.9

Table 2: Comparison with representative trackers on four benchmarks. Gray rows mark the method groups. The best score in each tracking metric is shown in bold , and LoopTrack

{}_{\text{Three}}

is highlighted in light cyan. All tracking scores are percentages. GPU FPS values for other trackers are taken from the original papers. “–” denotes unreported results.

	TAL	GTM	SUC	$P_{\mathrm{N}}$	$P$
❶	–	–	65.8	75.5	70.8
❷	✓	–	68.8	78.6	74.0
❸	✓	✓	69.3	79.2	74.7

Table 3: Ablation of LoopTrack components.

	SR	TU	SUC	$P_{\mathrm{N}}$	$P$
❶	–	–	68.5	78.3	73.7
❷	✓	–	68.8	78.5	74.1
❸	✓	✓	69.3	79.2	74.7

Table 4: Ablation of components in TAL.

	Mem	AP	GU	SUC	$P_{\mathrm{N}}$	$P$
❶	–	–	–	68.8	78.6	74.1
❷	✓	–	–	68.9	78.7	74.2
❸	✓	✓	–	69.1	79.0	74.4
❹	✓	✓	✓	69.3	79.2	74.7

Table 5: Ablation of components in GTM.

	$\mathcal{L}_{\mathrm{box}}$	$\mathcal{L}_{\mathrm{qual}}$	SUC	$P_{\mathrm{N}}$	$P$
❶	–	–	68.8	78.7	74.1
❷	✓	–	69.1	78.9	74.5
❸	–	✓	69.0	79.0	74.3
❹	✓	✓	69.3	79.2	74.7

Table 6: Ablation of supervision in TAL.

Variant	Params (M)	SUC	$P_{\mathrm{N}}$	$P$
UTPTrack-O-256	92.5	67.40	76.30	72.50
w/ LoopTrack ${}_{\text{Three}}$	24.0	67.04	76.36	72.21
OSTrack-256 (w/o CE)	92.52	68.70	78.10	74.60
w/ LoopTrack ${}_{\text{Three}}$	24.1	68.26	77.82	74.21

Table 7: Generality of LoopTrack across different Transformer trackers on LaSOT.

Figure 2: Effect of

K

Model	BS = 1	BS = 32
LoopTrack ${}_{\text{One}}$	46.09	368.28
LoopTrack ${}_{\text{Two}}$	54.00	376.74
LoopTrack ${}_{\text{Three}}$	61.38	383.36

Table 8: GPU memory efficiency of LoopTrack, where BS denotes batch size.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Training distribution	$K=6$	$K=8$	$K=12$
Fixed $K=8$	68.6	68.9	68.7
Uniform $\{6,\ldots,12\}$	69.0	69.2	69.1
$\operatorname{Beta}(2,1)$	68.8	69.1	69.2
$\operatorname{Beta}(1,2)$	69.0	69.3	68.9

Appendix

Table 9: Ablation of loop-count sampling.

Memory confidence supervision	SUC	$P_{\mathrm{N}}$	$P$
Without $\mathcal{L}_{\mathrm{conf}}$	69.0	77.8	73.7
With $\mathcal{L}_{\mathrm{conf}}$	69.3	79.2	74.7

Appendix

Table 10: Effect of explicit memory confidence supervision in GTM.

TU update loops	SUC	$P_{\mathrm{N}}$	$P$
$\{2\}$	68.9	78.8	74.2
$\{2,4\}$	69.1	79.0	74.2
$\{4,6\}$	69.0	78.9	74.3
$\{2,4,6\}$	69.3	79.2	74.7
Every intermediate loop ( $k<K$ )	69.0	78.8	73.9

Appendix

Table 11: Effect of TU update schedule.

Candidate source	SUC	$P_{\mathrm{N}}$	$P$
Final memory $\mathbf{m}_{t}^{(K)}$	68.9	78.7	74.2
Selected final search $\bar{\mathbf{X}}_{t}^{(K)}$	69.0	78.8	74.6
Selected pre-loop search $\mathbf{X}_{t}^{(0)}$	69.3	79.2	74.7

Appendix

Table 12: Ablation of candidate source.

Response mass $\mu$	SUC	$P_{\mathrm{N}}$	$P$
0.50	68.8	78.6	74.0
0.65	69.1	78.7	74.4
0.75	69.3	79.2	74.7
0.85	68.7	76.7	73.9
0.95	68.9	78.6	74.2

Appendix

Table 13: Ablation of response-mass threshold.

Configuration	Params (M)	MACs (G)	FPS
Plain shared loop	6.373	9.609	108
+ TAL	6.377	9.618	101
+ GTM	6.379	9.620	98

Appendix

Table 14: Model complexity and inference efficiency of TAL and GTM.

	$K$	Params (M)	FPS	SUC	$P_{\mathrm{N}}$	$P$
❶	$4^{\dagger}$	6.4	151	67.8	78.0	72.5
❷	6	6.4	121	69.0	78.9	74.3
❸	8	6.4	98	69.3	79.2	74.7
❹	10	6.4	84	69.2	79.1	74.5
❺	12	6.4	74	68.9	78.9	74.3

Appendix

Table 15: Effect of inference depth.

Figure 3: Localization across recurrent depths using the same trained checkpoint. Columns correspond to loops $2$ , $4$ , $6$ , $8$ , and $12$ . Green dashed boxes denote ground truth, orange boxes denote predictions, and heatmaps show spatial responses; values report IoU.

Figure 4: Effect of TAL using separately trained models under identical settings. Upper and lower rows show results without and with TAL, respectively. Green dashed boxes denote ground truth, blue and orange boxes denote predictions, and heatmaps show spatial responses; values report IoU.

Figure 5: Effect of GTM using separately trained models under identical settings. Upper and lower rows show results without and with GTM, respectively. Green dashed boxes denote ground truth, blue and orange boxes denote predictions, and heatmaps show spatial responses; values report IoU.

Explore similar work

Aug 10, 2026cs.CV

FeedbackTrack: Visual-Cortex-Inspired Cross-Frame Feedback for Transformer Tracking

Visual object tracking requires effective temporal integration, yet most Transformer trackers still rely on predominantly feed-forward feature extraction. Existing temporal mechanisms typically update templates, prompts, queries, or prediction states, while intermediate representations are rarely reused to modulate corresponding processing stages. We propose \textbf{FeedbackTrack}, a visual-cortex-inspired framework that introduces sparse, group-level layer-aligned cross-frame feedback into pretrained Transformer trackers. Previous-frame intermediate states are detached, cached, and returned to corresponding Transformer groups in the current frame through two lightweight pathways: Query Feedback for token-level query modulation and Gate Feedback for context-dependent feature modulation. FeedbackTrack preserves the original tracking pipeline with only a fixed-size one-frame cache. Across SPMTrack and ARTrackV2, FeedbackTrack consistently improves five backbone configurations on LaSOT and GOT-10k, achieving 83.4 AO and 79.1 AUC with SPMTrack-G while adding less than 1% parameters. Controlled comparisons show that cross-frame feedback outperforms same-frame modulation by 1.8--3.2 AO points, demonstrating that the gains mainly come from recurrent historical information. Further analysis reveals a non-uniform depth-dependent organization of learned feedback strengths, highlighting the effectiveness of recurrent feedback for Transformer tracking.

Yueyang Cang, Xiaoteng Zhang, Zhiyuan Ning +2

Tsinghua University Beijing, China

Multi-Object Tracking Large Inter-Frame Displacements Photometric Supervision Feedback

Ask PDF

Oct 5, 2026cs.CV

Difference Feature Map Distillation: Transferring Inter-Sample Relational Knowledge Towards Efficient Transformer-Based Tracking

In autonomous driving perception, visual object tracking systems must satisfy stringent latency and power constraints while remaining robust in complex and dynamic environments. Although transformer-based trackers achieve state-of-the-art accuracy, their substantial computational and memory overheads hinder deployment on real-time, resource-constrained platforms. To move toward this goal, we propose Difference Feature Map Knowledge Distillation (DFM-KD), a novel relational distillation framework tailored for transformer-based visual object tracking. Unlike conventional feature distillation methods that minimize point-wise discrepancies (e.g., mean squared error) between teacher and student feature representations, DFM-KD transfers knowledge through inter-sample feature differences, explicitly aligning the relational structure of the feature space. By distilling how the teacher models appearance variation and consistency across samples, rather than enforcing similarity in absolute activations, DFM-KD enables the student to better capture the structural dynamics of visual changes within a batch. As a result, the distilled model exhibits enhanced feature robustness and improved tracking performance. Extensive experiments demonstrate that DFM-KD consistently outperforms conventional feature-level distillation methods in both tracking precision and success rates.

Zhicheng Ding, Xinyu Chu, Qing Tian

Department of Computer Science, Bowling Green State University, OH 43402, USA · Department of Computer Science, University of Alabama at Birmingham, AL 35294, USA

Multi-Object Tracking

Ask PDF

May 8, 2026cs.CV

An Efficient Token Compression Framework for Visual Object Tracking

Refining visual representations by eliminating their internal feature-level redundancy is crucial for simultaneously optimizing the performance and computational cost of models in visual tracking. To enhance their performance, many contemporary Transformer-based trackers leverage a larger number of historical template frames to capture richer spatio-temporal cues. However, this strategy leads to a massive number of input visual tokens. This creates two critical issues: it imposes a quadratic computational burden and can also degrade the tracker's overall performance. To bridge this gap, we propose a compress-then-interact tracking framework, ETCTrack, that learns to efficiently compress template tokens from historical template frames into a robust target representation, moving beyond handcrafted rules. Our method first employs the Adaptive Token Compressor to dynamically construct compact yet highly discriminative template tokens by filtering out redundant visual tokens. These refined template tokens are then processed by our Hierarchical Interaction Encoder to achieve a deep, adaptive interaction with the search features. Refined search features ensure subsequent precise target localization. Experiments on seven benchmarks demonstrate that our method outperforms current state-of-the-art trackers. ETCTrack-B224 reduces the number of template tokens by 60%, leading to a 21.4% reduction in MACs with only a 0.4% drop in accuracy. The source code are available at https://github.com/PJD-WJ/ETCTrack.

Weijing Wu, Qihua Liang, Bineng Zhong +3

Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin, 541004, China · Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin, 541004, China · University Engineering Research Center of Educational Intelligent Technology, Guangxi Normal University, Guilin, 541004, China +1

Token Compression Self-Supervised Vision Transformers Data Compression Methods Representation Learning

Ask PDF