SCATR: Mitigating New Instance Suppression in LiDAR-based Tracking-by-Attention via Second Chance Assignment and Track Query Dropout

Brian Cheong; Letian Wang; Sandro Papais; Steven L. Waslander

cs.CVMar 2, 2026

SCATR: Mitigating New Instance Suppression in LiDAR-based Tracking-by-Attention via Second Chance Assignment and Track Query Dropout

Authors: Brian Cheong, Letian Wang, Sandro Papais, Steven L. Waslander

Organizations: University of Toronto

Abstract

LiDAR-based tracking-by-attention (TBA) frameworks inherently suffer from high false negative errors, leading to a significant performance gap compared to traditional LiDAR-based tracking-by-detection (TBD) methods. This paper introduces SCATR, a novel LiDAR-based TBA model designed to address this fundamental challenge systematically. SCATR leverages recent progress in vision-based tracking and incorporates targeted training strategies specifically adapted for LiDAR. Our work's core innovations are two architecture-agnostic training strategies for TBA methods: Second Chance Assignment and Track Query Dropout. Second Chance Assignment is a novel ground truth assignment that concatenates unassigned track queries to the proposal queries before bipartite matching, giving these track queries a second chance to be assigned to a ground truth object and effectively mitigating the conflict between detection and tracking tasks inherent in tracking-by-attention. Track Query Dropout is a training method that diversifies supervised object query configurations to efficiently train the decoder to handle different track query sets, enhancing robustness to missing or newborn tracks. Experiments on the nuScenes tracking benchmark demonstrate that SCATR achieves state-of-the-art performance among LiDAR-based TBA methods, outperforming previous works by 7.6% AMOTA and successfully bridging the long-standing performance gap between LiDAR-based TBA and TBD methods. Ablation studies further validate the effectiveness and generalization of Second Chance Assignment and Track Query Dropout. Code can be found at the following link: https://github.com/TRAILab/SCATR

Figures & tables

Figure 1: a) In regular tracking-by-attention, when a track query is assigned to the vehicle, related proposal queries are suppressed, leading to sparse supervision of proposal queries, only from newborn objects (

PQ\;Z

). b) We propose Track Query Dropout, in which assigned track queries may not be propagated to subsequent frames. This provides proposal queries, such as

PQ\;Y

, positive supervision with the now unassigned car, conditioned on the absence of a related track query. c) Second Chance Assignment allows both unassigned track queries, like

TQ\;B

, to be assigned to unmatched ground truth objects.

Figure 2: Architecture diagram of SCATR. Given BEV features of the scene and a set of anchor proposal queries, a detection decoder performs an initial detection of objects in the scene. The top- $N_{PQ}$ proposal queries are concatenated with the track queries from the previous frame and passed to the track decoder. The top- $N_{TQ}$ output queries are propagated to the next frame as track queries. Second Chance Assignment allows both track queries and proposal queries to initialize tracks.

Figure 3: Creating an auxiliary query group by dropping the track query assigned to object $B$ trains the decoder to only suppress the potential proposal query for object $B$ conditioned on whether the assigned track query is present.

Figure 4: In Second Chance Assignment, unassigned track queries are also considered when matching unassigned ground truth objects, and may take on new track IDs during training.

Method		AMOTA $\uparrow$	AMOTP $\downarrow$	FP $\downarrow$	FN $\downarrow$	IDS $\downarrow$	mAP $\uparrow$
Detection	Tracking	Tracking-by-Detection
CenterPoint [ 31 ]	CenterPoint [ 31 ]	0.638	0.555	18612	22928	760	0.580
Centerpoint [ 31 ]	SimpleTrack [ 25 ]	0.668	0.550	17514	23451	575	0.580
TransFusion-L [ 1 ]	CenterPoint [ 31 ]	0.686	0.529	17851	23437	893	0.655
VoxelNext [ 5 ]	VoxelNext [ 5 ]	0.695	0.568	-	-	785	0.645
FocalFormer-L [ 4 ]	SimpleTrack [ 25 ]	0.715	0.549	16760	21142	888	0.687

Table 1: Tracking performance on the nuScenes [ 2 ] test split. Bold represents the best performing among Tracking by Attention methods. Underline represents the best performing overall. “-” indicates that a value is not provided by a previous work.

Method		AMOTA $\uparrow$	AMOTP $\downarrow$	FP $\downarrow$	FN $\downarrow$	IDS $\downarrow$	mAP $\uparrow$
Detection	Tracking	Tracking-by-Detection
CenterPoint [ 31 ]	CenterPoint [ 31 ]	0.637	0.606	13616	22573	640	0.564
CenterPoint [ 31 ]	SimpleTrack [ 25 ]	0.687	0.573	12983	19941	519	0.564
TransFusion-L [ 1 ]	CenterPoint [ 31 ]	0.686	-	17851	23437	893	0.655
VoxelNext [ 5 ]	VoxelNext [ 5 ]	0.702	0.64	12642	19179	729	0.605
FocalFormer-L [ 4 ]	SimpleTrack [ 25 ]	0.728	0.576	11039	19542	528	0.664

Table 2: Tracking performance on the nuScenes [ 2 ] validation split. Bold represents the best performing among Tracking by Attention methods. Underline represents the best performing overall. “-” indicates that a value is not provided by a previous work.

S.C. Assignment	TQ Dropout	mAP $\uparrow$	AMOTA $\uparrow$	AMOTP $\downarrow$	FP $\downarrow$	FN $\downarrow$	IDS $\downarrow$	NB Conf $\uparrow$	Trk Conf $\uparrow$
✗	✗	0.587	0.575	0.6963	11403	26490	1448	0.214	0.425
✓	✗	0.641	0.677	0.609	11348	17331	165	0.291	0.549
✗	1 group	0.585	0.570	0.7172	12135	26403	1395	0.2143	0.420
✓	1 group	0.645	0.688	0.609	10850	17556	163	0.305	0.560
✓	2 groups	0.648	0.703	0.607	11534	17579	241	0.306	0.565

Table 3: Ablation on the impacts of second chance assignment and track query dropout on SCATR. “NB Conf” and “TQ Conf” refer to the prediction confidence of newborn objects and tracked objects, respectively.

S.C. Assignment	TQ Dropout	mAP $\uparrow$	AMOTA $\uparrow$	AMOTP $\downarrow$	FP $\downarrow$	FN $\downarrow$	IDS $\downarrow$	NB Conf $\uparrow$	TQ Conf $\uparrow$
✗	✗	0.281	0.205	1.262	13848	63567	228	0.144	0.295
✓	✗	0.346	0.270	1.156	12026	52511	73	0.188	0.365
✗	✓	0.291	0.213	1.265	9719	62262	234	0.140	0.295
✓	1 group	0.276	0.288	1.165	11218	51742	92	0.186	0.363

Table 4: Ablation on the impacts of second chance assignment and track query dropout on SCATR-C using a vision backbone.

Experiment	mAP $\uparrow$	AMOTA $\uparrow$	AMOTP $\downarrow$	FP $\downarrow$	FN $\downarrow$	IDS $\downarrow$	TQ Recall
SCATR+det. loss+SimpleTrack	0.652	0.702 *	0.673*	10724 *	19281*	671*	0.177
SCATR TBA	0.645	0.688	0.609	10850	17556	163	0.996

Table 5: Ablation comparing our SCATR tracking-by-attention against a SCATR-like detector with SimpleTrack [ 25 ] tracking-by-detection. * indicates these values were obtained from applying SimpleTrack to the output detections.

Explore similar work

TFTrack: A Template-Free Framework for Efficient 3D Point Cloud Tracking
Sep 7, 2026Zhaofeng Hu, Sifan Zhou, Jiahao Nie +33D Point Tracking Visual Object Tracking
Stony Brook University · Carnegie Mellon University · Hangzhou Dianzi University +2
CARE: Camera-Residual Reserves for First Sightings in Adaptive LiDAR Sensing
Aug 25, 2026Jiachen Gong, Yun Li, Ehsan Javanmardi +2Autonomous Driving Lidar-Based 3D Object Detection
The University of Tokyo, Tokyo, Japan · National Institute of Informatics, Tokyo, Japan
SAVTrack: Selective Vote Aggregation for Reliability-Aware Point Cloud Tracking
Sep 15, 2026Sifan Zhou, Linyue Tan, Qiwei Wang +2Point Tracking 3D Point Tracking
School of Automation, Southeast University, Nanjing, China · University of Pennsylvania, Philadelphia, PA, USA · Harbin Institute of Technology (Shenzhen), Shenzhen, China +1

Sep 7, 2026J/K move · Enter open · S save

TFTrack: A Template-Free Framework for Efficient 3D Point Cloud Tracking

Zhaofeng Hu, Sifan Zhou, Jiahao Nie, Ziyu Zhao+2

Stony Brook University · Carnegie Mellon University · Hangzhou Dianzi University+2

3D Point Tracking Visual Object Tracking Autonomous Driving

Open paper AskPDF arXiv

LiDAR-based 3D Single Object Tracking (3D SOT) is critical for robotic perception and navigation and aims to localize dynamic objects across frames in sparse point clouds. Existing methods, rooted in the Siamese tracking paradigm from 2D vision, rely on costly dual-input designs and excessive motion modeling guided by template priors, hindering their efficiency. Our in-depth analysis reveals: (i) the template paradigm is redundant, as the previous bounding box center encodes sufficient historical context; (ii) complex motion modeling is unnecessary, as geometric alignment provides adequate motion priors. Based on the above findings, we propose the first Template-Free Tracking framework (TFTrack). The novel framework eliminates the need for template-search pairings and operates directly on the current frame guided solely by the prior bounding box center and size. We instantiate this paradigm into three variants: TFTrack-Voxel, TFTrack-Pillar, and TFTrack-Point, to explore different 3D representations under a unified framework, ensuring flexibility across sparse and dense scenes. Extensive experiments on KITTI and nuScenes benchmarks show that TFTrack is competitive with leading template-based trackers, while reducing FLOPs by approximately 50% and running at approximately 120 FPS. By simplifying overcomplicated motion-centric designs, TFTrack establishes a new minimalist paradigm for efficient 3D point cloud tracking, paving the way for real-time and resource-efficient deployment in embedded robotic systems, such as autonomous vehicles. The code is available at https://github.com/tftrack-anonymous/TFTrack/tree/main.

Abstract

Explore similar work

TFTrack: A Template-Free Framework for Efficient 3D Point Cloud Tracking

CARE: Camera-Residual Reserves for First Sightings in Adaptive LiDAR Sensing

SAVTrack: Selective Vote Aggregation for Reliability-Aware Point Cloud Tracking