cs.CVNov 3, 2026

TAQ-MOTR: Tracking-Aware Detection Query Formation for End-to-End Multiple Object Tracking

Authors: Hanyang Liu, Hong Zhang, Ding Yuan, Yang Han, Yifan Yang

Organizations: School of Astronautics, Beihang University, 37 Xueyuan Road, Haidian District, Beijing 100191, China · Key Laboratory of Spacecraft Design Optimization and Dynamic Simulation Technology, Ministry of Education · State Key Laboratory of High-Efficiency Reusable Aerospace Transportation Technology · School of Software, Beihang University, 37 Xueyuan Road, Haidian District, Beijing 100191, China

Abstract

Tracking-by-propagation (TBP) methods perform end-to-end multi-object tracking by propagating track queries across frames, jointly handling detection and identity association without heuristic post-processing while naturally modeling temporal information. However, as detection and association are tightly coupled, detection queries in TBP are expected to discover currently untracked objects and provide complementary current-frame evidence for challenging tracks whose queries and identities are retained. Existing methods largely inherit detection-query formation from single-frame detection and construct such queries before the decoder with little awareness of trajectory states. Consequently, currently untracked objects may be missed, while challenging tracked objects with reduced visibility under occlusion may lack complementary evidence, causing association failures or duplicate responses. To address these issues, we propose TAQ-MOTR, a tracking-aware detection query formation framework that employs a pre-decoder Hybrid Query Generator (HQG) to fuse current-frame image features, track memory, and learnable memory. By conditioning detection queries on trajectory states, TAQ-MOTR adapts them to \rev{the discovery of currently untracked objects and tracked-object complementation We further introduce a training-only auxiliary detection branch whose one-to-one label assignment is decoupled from that of the main tracking branch, providing additional detection supervision to strengthen the shared HQG and decoder for the discovery of currently untracked objects. TAQ-MOTR improves performance on DanceTrack, SportsMOT, and BFT, achieving 72.0 HOTA, 63.1 AssA, and 76.6 IDF1 on the DanceTrack test set. Code is available at https://github.com/Magicmiya/TAQ-MOTR

Figures & tables

Explore similar work

Oct 28, 2026

Calibrating Track-Query Updates at Test Time for Multi-Object Tracking

Multi-Object Tracking (MOT) requires localizing multiple objects and maintaining their identities over time. Recent query-propagation trackers carry target-specific representations across frames and update them with current-frame image features, yet identity association remains difficult when visually similar targets overlap or cross. During such interactions, current-frame query updates may contain components directed toward nearby active tracks, which may affect identity association in later frames. To investigate this issue, we analyze track-query updates preceding identity switches (IDSWs), where the same target is assigned a different track ID. Our analysis shows that (i) propagated track-query representations change more abruptly before IDSWs, (ii) current-frame updates have larger positive projections along directions toward other active tracks, and (iii) bounding-box intersection over union (IoU) retrieves the counterpart track involved in a subsequent IDSW with high recall when that track remains active before the switch. Motivated by these findings, we propose Test-time Track-Query Calibration (TQC), a training-free and gradient-free method for query-propagation trackers. TQC calibrates the tracker output before propagation using a bounded correction estimated from update components directed toward nearby active tracks. It requires only track-query representations and bounding boxes available during inference, without retraining or architectural modification. Across nine DanceTrack and SportsMOT test settings, TQC improves HOTA and AssA in all settings and IDF1 in eight, with an average HOTA gain of +0.38 and limited inference overhead. Controlled variants on six validation settings further show that TQC's correction direction yields more consistent HOTA improvements than alternative directions with matched correction norms.
Apr 1, 2026

Out of Sight, Out of Track: Adversarial Attacks on Propagation-based Multi-Object Trackers via Query State Manipulation

Recent Tracking-by-Query-Propagation (TBP) methods have advanced Multi-Object Tracking (MOT) by enabling end-to-end (E2E) pipelines with long-range temporal modeling. However, this reliance on query propagation introduces unexplored architectural vulnerabilities to adversarial attacks. We present FADE, a novel attack framework designed to exploit these specific vulnerabilities. FADE employs two attack strategies targeting core TBP mechanisms: (i) Temporal Query Flooding: Generates spurious temporally consistent track queries to exhaust the tracker's limited query budget, forcing it to terminate valid tracks. (ii) Temporal Memory Corruption: Directly attacks the query updater's memory by severing temporal links via state de-correlation and erasing the learned feature identity of matched tracks. Furthermore, we introduce a differentiable pipeline to optimize these attacks for physical-world realizability by leveraging simulations of advanced perception sensor spoofing. Experiments on MOT17 and MOT20 benchmarks demonstrate that FADE is highly effective against state-of-the-art TBP trackers, causing significant identity switches and track terminations.
Sep 14, 2026

Beyond the Survey: A Systematic Empirical Study of Detection and Association in Visual MOT

This paper presents a comprehensive experimental evaluation and detailed analysis of state-of-the-art multi-object tracking algorithms, with an emphasis on quantifying the individual contributions of detection and association components to overall tracking performance. Unlike existing surveys that primarily offer theoretical categorizations or taxonomies of tracking methods, our work adopts a rigorous experimental perspective grounded in publicly available implementations, providing practical guidance for researchers and practitioners in method selection and system design. We introduce a unified pipeline diagram that consolidates the core components across the two main branches of visual multi-object tracking: tracking-by-detection and end-to-end deep learning paradigms, and systematically analyze the object detection, feature extraction, and data association modules. Through extensive empirical studies on standard benchmarks, including MOT16, MOT17, MOT20, SportsMOT, DanceTrack, and CrowdTrack datasets, we reveal critical insights: (1) detection quality dominates association strategy performance, with detector improvements yielding more than 10% gains compared to less than 5% from refined association strategies; (2) modern deep learning detectors paired with specialized re-identification models significantly outperform joint detection and embedding approaches; and (3) transformer-based end-to-end methods exhibit greater robustness to detection quality variations but at a substantial computational cost. Our findings from extensive experiments provide key insights into component-level effects in MOT, particularly the dominant influence of detection quality relative to association, while offering practical insights for designing and optimizing MOT systems under varying performance and robustness requirements. Code and experimental setups are available at github.com/linh-gist/VisualMOT.