FocusTrack: One-Stage Focus-and-Suppress Framework for 3D Point Cloud Object Tracking
Authors: Sifan Zhou, Jiahao Nie, Ziyu Zhao, Yichao Cao, Xiaobo Lu
Organizations: School of Automation Southeast University, Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education Nanjing, China · School of Information Technology and Artificial Intelligence, Zhejiang University of Finance and Economics Hangzhou, China · Big Data Institute Central South University Changsha, China
In 3D point cloud object tracking, the motion-centric methods have emerged as a promising avenue due to its superior performance in modeling inter-frame motion. However, existing two-stage motion-based approaches suffer from fundamental limitations: (1) error accumulation due to decoupled optimization caused by explicit foreground segmentation prior to motion estimation, and (2) computational bottlenecks from sequential processing. To address these challenges, we propose FocusTrack, a novel one-stage paradigms tracking framework that unifies motion-semantics co-modeling through two core innovations: Inter-frame Motion Modeling (IMM) and Focus-and-Suppress Attention. The IMM module employs a temp-oral-difference siamese encoder to capture global motion patterns between adjacent frames. The Focus-and-Suppress attention that enhance the foreground semantics via motion-salient feature gating and suppress the background noise based on the temporal-aware motion context from IMM without explicit segmentation. Based on above two designs, FocusTrack enables end-to-end training with compact one-stage pipeline. Extensive experiments on prominent 3D tracking benchmarks, such as KITTI, nuScenes, and Waymo, demonstrate that the FocusTrack achieves new SOTA performance while running at a high speed with 105 FPS.
Figures & tables
Figure 1. Comparison between two-stage motion-based tracking methods (a) and our two-stage motion-based tracking method (b). The two-stage motion-based methods M2Track ( Zheng et al., 2022 ; Zheng et al., 2023 ) series models motion relation for tracking in a two-stage manner. In contrast, our FocusTrack explores unified motion-foreground modeling for one-stage tracking.
Figure 2. Comparison with state-of-the-art methods. We visualize mean precision across all categories on KITTI dataset ( Geiger et al., 2012 ) with respect to running speed (FPS). FocusTrack shows the new SOTA in terms of both accuracy and speed.
Figure 3. One-stage focus-and-suppress framework ( FocusTrack ): This framework consists of a backbone that capture inter-frame differences for motion modeling. Then, it features a Focus-and-Suppress attention mechanism designed to suppress background noise while enhancing foreground semantics based on motion weights.
Mean
Car
Pedestrian
Van
Cyclist
Paradigm
Tracker
Source
(14,068)
(6,424)
(6,088)
(1,248)
(308)
FPS
Device
Match
SC3D ( Giancola et al., 2019 )
CVPR’19
31.2 / 48.5
41.3 / 57.9
18.2 / 37.8
40.4 / 47.0
41.5 / 70.4
2
GTX 1080Ti
P2B ( Qi et al., 2020 )
CVPR’20
42.4 / 60.0
56.2 / 72.8
28.7 / 49.6
40.8 / 48.4
32.1 / 44.7
40
GTX 1080Ti
PTT ( Shan et al., 2021 )
IROS’21
55.1 / 74.2
67.8 / 81.8
44.9 / 72.0
43.6 / 52.5
37.2 / 47.3
40
GTX 1080Ti
LTTR ( Cui et al., 2021 )
BMVC’21
48.7 / 65.8
65.0 / 77.1
33.2 / 56.8
35.8 / 45.6
66.2 / 89.9
23
GTX 1080Ti
MLVSNet ( Wang et al., 2021 )
ICCV’21
45.7 / 66.6
56.0 / 74.0
34.1 / 61.1
52.0 / 61.4
34.4 / 44.5
70
GTX 1080Ti
Table 1. Comparisons with state-of-the-art methods on KITTI dataset ( Geiger et al., 2012 ) . The upper and lower parts include two-stream and one-stream trackers, respectively. Success / Precision are used for evaluation. Bold and underline denote the best result and the second-best one, respectively. † means the methods was pre-trained on large-scale images and point cloud dataset ( Fan et al., 2019 ; Huang et al., 2019 ; Pang et al., 2022 ) .
Tracker
Mean
Mean
Car
Pedestrian
Truck
Trailer
Bus
(117,278)
Category
(64,159)
(33,227)
(13,587)
(3,352)
(2,953)
SC3D ( Giancola et al., 2019 )
20.70 / 20.20
25.78 / 22.90
22.31 / 21.93
11.29 / 12.65
30.67 / 27.73
35.28 / 28.12
29.35 / 24.08
P2B ( Qi et al., 2020 )
36.48 / 45.08
38.41 / 40.90
38.81 / 43.18
28.39 / 52.24
42.95 / 41.59
48.96 / 40.05
32.95 / 27.41
PTT ( Shan et al., 2021 )
36.33 / 41.72
40.38 / 41.81
41.22 / 45.26
19.33 / 32.03
50.23 / 48.56
51.70 / 46.50
39.40 / 36.70
BAT ( Zheng et al., 2021 )
38.10 / 45.71
40.59 / 42.42
40.73 / 43.29
28.83 / 53.32
45.34 / 42.58
52.59 / 44.89
35.44 / 28.01
PTTR ( Zhou et al., 2022 )
44.50 / 52.07
43.22 / 44.91
51.89 / 58.61
29.90 / 45.09
45.30 / 44.74
45.87 / 38.36
43.14 / 37.74
Table 2. Comparisons with state-of-the-art methods on nuScenes dataset ( Caesar et al., 2020 ) . Success / Precision are used for evaluation. Bold and underline denote the best result and the second-best one, respectively.
Vehicle
Pedestrian
Easy
Medium
Hard
Mean
Easy
Medium
Hard
Mean
Tracker
Mean
(67,832)
(61,252)
(56,647)
(185,731)
(85,280)
(82,253)
(74,219)
(241,752)
P2B ( Qi et al., 2020 )
33.0 / 43.8
57.1 / 65.4
52.0 / 60.7
47.9 / 58.5
52.6 / 61.7
18.1 / 30.8
17.8 / 30.0
17.7 / 29.3
17.9 / 30.1
BAT ( Zheng et al., 2021 )
34.1 / 44.4
61.0 / 68.3
53.3 / 60.9
48.9 / 57.8
54.7 / 62.7
19.3 / 32.6
17.8 / 29.8
17.2 / 28.3
18.2 / 30.3
V2B ( Hui et al., 2021 )
38.4 / 50.1
64.5 / 71.5
55.1 / 63.2
52.0 / 62.0
57.6 / 65.9
27.9 / 43.9
22.5 / 36.2
20.1 / 33.1
23.7 / 37.9
STNet ( Hui et al., 2022 )
40.4 / 52.1
65.9 / 72.7
57.5 / 66.0
54.6 / 64.7
59.7 / 68.0
29.2 / 45.3
24.7 / 38.2
22.2 / 35.8
25.5 / 39.9
Table 3. Generalization comparisons on Waymo Open Dataset (WOD) ( Sun et al., 2020 ) . All the model is pre-trained on KITTI and evaluated on WOD val set. Success / Precision are used for evaluation. Bold and underline denote the best result and the second-best one.
Setting
CNN
Linear
DWC
IMM
Car
Pedestrian
(a)
✓
✗
✗
✗
69.4 / 79.6
63.1 / 87.7
(b)
✓
✓
✗
✗
70.2 / 80.4
64.5 / 89.3
(c)
✓
✓
✓
✗
71.6 / 82.1
66.2 / 91.0
(d)
✓
✓
✓
✓
74.1 / 85.9
69.3 / 94.1
Table 4. Ablation results of different module on the KITTI val set for car and pedestrian.
Figure 4. The BEV feature visualization of FocusTrack.
Setting
Shared
Car
Pedestrian
FPS
(a)
✗
73.2 / 83.8
68.7 / 92.9
91
(b)
✓
74.1 / 85.9
69.3 / 94.1
105
Table 5. Ablation of w/o shared setting for CNN and Linear layers in FocusTrack block on the KITTI dataset.
Ratio
Resolution
Car
Pedestrian
FPS
2X
64 X 64
71.5 / 82.1
66.1 / 90.1
118
4X
32 X 32
72.4 / 81.9
67.2/ 91.5
112
8X
16 X 16
74.1 / 85.9
69.3 / 94.1
105
16X
8 X 8
73.6 / 83.6
68.4 / 92.5
98
Table 6. Ablation study on downsample ratio.
Figure 11
Figure 5. Visualization of tracking results compared with state-of-the-art motion-based M2Track ( Zheng et al., 2022 ) method.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Seen Classes → Unseen Classes
Van
Cyclist
Van, Cyclist (Tab. 1)
68.4 / 83.5
75.9 / 94.7
Car,Pedestrian → Van,Cyclist
65.8 / 79.4
73.1 / 91.8
Appendix
Table 7. Generalization experiments from seen object classes to unseen ones on KITTI ( Geiger et al., 2012 ) dataset.
nuScenes → KITTI nuScenes
Car
Pedestrian
KITTI (Tab. 1)
74.1 / 85.9
69.3 / 94.1
nuScenes → KITTI
58.6 / 72.3
63.1 / 88.9
Appendix
Table 8. Generalization experiments between KITTI ( Geiger et al., 2012 ) and nuScenes ( Caesar et al., 2020 ) datasets with distinct point cloud sparsity.
Visual object tracking follows objects across video frames, maintaining their locations and identities despite motion, occlusion, and changing appearance.