A social robot has to keep track of the people around it over time to plan its motion, a task handled by 3D multi-object tracking. For the LiDAR modality, trackers usually sit on top of a detector and link its detections across frames with a separate motion model or re-identification network. We instead show how a 3D LiDAR detector itself can be turned into a tracker by adding an instance-level contrastive objective on its intermediate voxel features. Each embedding is read from the voxels inside a detected box, and boxes are matched across frames by cosine similarity and the Hungarian algorithm, without a motion model. The detector backbone is fine-tuned together with this objective, which learns identity-aware features while leaving detection accuracy essen- tially intact. Because the method uses only the voxel features that Li- DAR detectors already produce, it is backbone-agnostic. On the JRDB 3D tracking leaderboard our LiDAR-only tracker ranks first on HOTA (38.58) and IDF1 (37.99), 13.04 and 9.30 points above the next pub- lished method, and second on MOTA and OSPA. Code is available at https://hermak56.github.io/CLIFT/.
Figures & tables
Figure 1: The CLIFT pipeline. A single forward pass through the 3D detector produces both predicted boxes (top, detection branch) and the voxel-level feature map. For each box, the tracking branch (bottom, this work) gathers the voxels inside it, mean-pools their features, projects and L2 -normalizes them into an instance embedding eit , and the online tracker associates embeddings across frames by cosine similarity and the Hungarian algorithm. The backbone is either frozen or jointly fine-tuned; the detection head is never modified.
Metric
Config A
Config E
HOTA ↑
30.70
38.58
DetA ↑
47.10
46.04
AssA ↑
20.07
32.41
AssPr ↑
32.96
70.06
MOTA ↑
40.24
42.55
IDF1 ↑
27.99
37.99
Table 1: JRDB 3D tracking test set. Configuration A is the frozen-backbone baseline; Configuration E uses joint backbone fine-tuning. Best per row in bold. Arrows give the direction of improvement. The transition from A to E adds 7.9 HOTA points and reduces identity switches by about 60%, at a small detection cost. The OSPA (2) row is the one exception, analyzed in Section 4.4 .
Method
HOTA ↑
MOTA ↑
IDF1 ↑
OSPA (2) ↓
DRFDFF †
24.78
43.37
27.47
0.903
ELPTNet [ 14 ]
25.54
32.65
28.69
0.783
S2MAT [ 13 ]
18.93
24.21
21.34
0.975
SS3D-MOT [ 23 ]
15.80
22.96
14.48
0.982
PC-DAN [ 22 ]
15.57
22.57
13.89
0.976
AB3DMOT [ 43 ]
11.81
19.34
10.67
0.971
Table 2: JRDB 3D tracking leaderboard. Best in bold, second best underlined. Baseline values are taken from the public leaderboard. † Entry on the public JRDB 3D tracking leaderboard [ 28 ] without an associated publication.
Config
HOTA ↑
DetA ↑
AssA ↑
MOTA ↑
IDF1 ↑
IDSW ↓
A
26.94
37.07
20.82
56.88
27.89
84,959
B
27.03
37.40
20.83
59.35
27.54
63,678
C
32.91
37.26
30.22
64.03
37.07
23,235
D
33.11
37.31
30.41
64.18
37.21
22,890
E
34.43
37.25
32.99
64.27
39.23
21,147
Table 3: Validation-set ablation over training configurations on the DCCLA backbone. Best per column in bold. A: same-frame augmentation, frozen. B: temporal pairs. C: K=4 multi-frame. D: A → B → C curriculum. E: joint fine-tune. The B → C step is the largest jump; E adds a further gain on top of C; DetA is stable throughout.
intra ↑
top-1 ↑
Backbone
t+1
t+10
t+1
t+10
frozen
0.936 / 0.872
0.836 / 0.749
0.687 / 0.851
0.419 / 0.594
joint
0.954 / 0.935
0.875 / 0.842
0.773 / 0.927
0.496 / 0.759
Table 4: Feature probe on the held-out validation sequences. Cells give detector features / embedding. Joint fine-tuning improves both, and the margin grows with the temporal gap.