cs.CVNov 11, 2026

Test-Time Selective Adaptation for Cross-Modal UAV Object Tracking with Token Shift

Authors: Yun Xiao, Hailong Ye, Zhicheng Zhao, Lei Liu, Amir Hussain

Organizations: Anhui University, Hefei, China. · Anhui University of Science and Technology, Huainan, China. · SDAIA-KFUPM Joint Research Centre for AI, Information and Computer Science Department, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia

Abstract

Cross-modal unmanned aerial vehicle (UAV) object tracking faces modality-induced appearance discrepancies and abrupt target-position shifts that can arise from modality switching, UAV motion, and inter-sensor offsets. Although existing methods model modality and spatial variations, they primarily rely on offline representation learning and offer limited localization-oriented adaptation during testing. To address these challenges, we propose Test-Time Selective Adaptation with Token Shift (TSAT), which derives spatial and temporal adaptation signals from unlabeled test sequences for online optimization. Specifically, we design a controllable token shift auxiliary task, which imposes explicit spatial correspondence between predefined token translations and resultant localization offsets, generating stable annotation-free spatial adaptation signals. Furthermore, we introduce a temporal backward prediction auxiliary task to exploit frame-wise temporal dependencies, effectively mitigating cumulative localization errors in continuous tracking. To suppress interference from unreliable tracking samples, we develop a reliability-aware update gating strategy to screen valid adaptation signals and stabilize online parameter updates. Experiments on CM-UOT demonstrate that TSAT achieves state-of-the-art performance against existing test-time adaptation methods, exhibiting superior robustness under frequent modality switching and severe spatial displacement. Experiments across tracking frameworks and datasets further validate the effectiveness and generalization capability of TSAT.

Figures & tables

Explore similar work

CardsList
  1. Cross-Modal UAV Object Tracking: State-Aware Representation Learning and A Unified Benchmark

    Jul 21, 2026Yun Xiao, Zhihong Hong, Jiandong Jin +3Multimodal Visual Object TrackingCross-Modal Representation Learning

  2. Semantic-Aware Temporal Adaptation for UAV Anti-UAV Tracking

    Jul 29, 2026Xiaozhen Qiao, Da Zhang, Yubin Guo +3Test-Time AdaptationVisual Object Tracking

  3. Dual-branch Distilled Transformer for Efficient Asymmetric UAV Tracking

    May 27, 2026Hongtao Yang, Bineng Zhong, Qihua Liang +4Visual Object TrackingEfficient ViTs