3D Single Object Tracking (3D-SOT) aims to localize a target object across a sequence of LiDAR point clouds, given its 3D bounding box in the first frame. Recent methods have adopted a memory-based approach to utilize previously observed features of the target object, but remain limited to only a few recent frames. This work reveals that their temporal capacity is fundamentally constrained to short-term context due to severe temporal feature inconsistency and excessive memory overhead. To this end, we propose a robust long-term 3D-SOT framework, ChronoTrack, which preserves the temporal feature consistency while efficiently aggregating the diverse target features via long-term memory. Based on a compact set of learnable memory tokens, ChronoTrack leverages long-term information through two complementary objectives: a temporal consistency loss and a memory cycle consistency loss. The former enforces feature alignment across frames, alleviating temporal drift and improving the reliability of proposed long-term memory. In parallel, the latter encourages each token to encode diverse and discriminative target representations observed throughout the sequence via memory-point-memory cyclic walks. As a result, ChronoTrack achieves new state-of-the-art performance on multiple 3D-SOT benchmarks, demonstrating its effectiveness in long-term target modeling with compact memory while running at real-time speed of 42 FPS on a single RTX 4090 GPU. The code is available at https://github.com/ujaejoon/ChronoTrack
Figures & tables
Figure 1: (a) Illustration of the feature space in existing works versus ours. Different shapes and colors represent different parts of the object and different time indices, respectively. In existing methods, target features tend to drift as the target’s appearance changes, resulting in temporal inconsistency that diminishes the utility of earlier features in memory. In contrast, our method enforces temporal feature consistency across frames, mitigating drift and enabling more reliable memory readout. (b) Quantitative analysis of temporal feature consistency and its correlation with performance. We evaluate the temporal consistency by measuring the average cosine similarity between target features from each frame and those from a future frame. Compared to MBPTrack, our method maintains higher feature similarity as the temporal distance increases. This robustness allows our method to take advantage of longer context, whereas MBPTrack fails to benefit from increased temporal capacity and instead exhibits performance (Success and Precision) degradation.
Figure 2: Overall pipeline of ChronoTrack . The point features Ft of the current point cloud Pt are first extracted by the backbone network E . These features are then fed to the Memory-based Feature Refiner (MFR) along with the long-term target memory Xt−1FG and short-term background memory Xt−1BG , injecting long-term target cues and short-term contextual information into the current features, resulting in target-aware features Zt . The decoder D then predicts the 3D bounding box Bt and targetness mask Mt from these enriched features, and the memory is updated based on the predicted mask and current point features. To ensure the reliability of accumulated information in memory and foster memory diversity, two complementary losses are applied: temporal consistency loss LTC and memory cycle consistency loss LMCC .
Figure 3: Losses for temporal consistency and token diversity. (a) Target points from distant frames are transformed into canonical coordinates to identify approximate pairs of spatially aligned points that likely belong to the same part of the target object. A temporal consistency loss is then applied to each pair to enforce high feature similarity. (b) On the basis of these temporally aligned features, each memory token undergoes a two-step walk (memory → points → memory). The walk is optimized to simultaneously maximize the probability that each token returns to itself after a two-step walk and the likelihood of transiting through foreground points, thereby encouraging tokens to encode diverse and target-specific semantics.
Method
Car
Pedestrian
Van
Cyclist
Mean
Suc.
Prec.
Suc.
Prec.
Suc.
Prec.
Suc.
Prec.
Suc.
Prec.
SC3D [ 4 ]
41.3
57.9
18.2
37.8
40.4
47.0
41.5
70.4
31.2
48.5
P2B [ 13 ]
56.2
72.8
28.7
49.6
40.8
48.4
32.1
44.7
42.4
60.0
STNet [ 7 ]
72.1
84.0
49.9
77.2
58.0
70.6
73.5
93.7
61.3
80.1
M 2 -Track [ 25 ]
65.5
80.8
61.5
88.2
53.8
70.7
73.2
93.5
62.9
83.4
CXTrack [ 20 ]
69.1
81.6
67.0
91.5
60.0
71.8
74.2
94.3
67.5
85.3
Table 1: Comparison with the SOTA methods on KITTI. “Mean” denotes the average results weighted by frame numbers. Bold and underline represent the best and second-best results, respectively.
Method
Car
Pedestrian
Truck
Trailer
Bus
Mean
Suc.
Prec.
Suc.
Prec.
Suc.
Prec.
Suc.
Prec.
Suc.
Prec.
Suc.
Prec.
SC3D [ 4 ]
22.3
21.9
11.3
12.7
30.7
27.7
35.3
28.1
29.4
24.1
20.7
20.2
P2B [ 13 ]
38.8
42.2
28.4
52.2
43.0
41.5
49.0
40.1
33.0
27.4
36.5
45.1
M 2 -Track [ 25 ]
55.9
65.1
32.1
60.9
57.4
59.5
57.6
58.3
51.4
51.4
49.2
62.7
MBPTrack [ 21 ]
62.5
70.4
45.3
74.0
62.2
63.3
65.1
61.3
55.4
51.8
57.5
69.9
SCVTrack [ 23 ]
58.9
67.7
34.5
61.5
60.6
61.4
59.5
60.1
54.3
53.6
52.1
64.7
Table 2: Comparison with the SOTA methods on NuScenes.
Figure 4: Qualitative results on the KITTI dataset.
Method
Vehicle
Pedestrian
Mean
Suc.
Prec.
Suc.
Prec.
Suc.
Prec.
P2B [ 13 ]
52.6
35.4
17.9
29.6
33.0
33.5
M 2 -Track [ 25 ]
61.1
61.6
32.0
67.3
44.6
63.5
CXTrack [ 20 ]
57.1
66.1
30.7
49.4
42.2
56.7
MBPTrack [ 21 ]
61.9
71.9
33.7
52.7
46.0
61.0
SCVTrack [ 23 ]
61.3
69.8
32.2
50.0
44.8
58.6
Table 3: Comparison with the SOTA methods on WOD. The models are pre-trained on KITTI and directly tested on the WOD validation set without fine-tuning.
XFG
LTC
LMCC
Car
Pedestrian
Van
Cyclist
Mean
⋅
⋅
⋅
74.0 / 86.2
67.9 / 93.5
63.6 / 76.1
77.0 / 94.6
70.5 / 88.6
✓
⋅
⋅
73.3 / 85.7
66.5 / 90.4
59.4 / 75.1
75.7 / 94.6
69.2 / 87.0
✓
✓
⋅
76.1 / 87.0
67.7 / 92.8
63.6 / 77.0
75.4 / 94.5
71.3 / 88.8
✓
✓
✓
76.0 / 88.6
68.6 / 94.0
64.2 / 77.7
77.7 / 94.8
71.8 / 90.1
Table 4: Component ablation of ChronoTrack. XFG indicates that token-level memory is used instead of point-level memory. LTC and LMCC denote the temporal consistency loss and memory cycle consistency loss, respectively. Success/Precision metrics are used for evaluation.
Method
KITTI
S (29)
M (28)
L (28)
XL (27)
MBPTrack [ 21 ]
78.0 / 91.5
73.2 / 85.3
75.4 / 87.6
72.7 / 83.7
ChronoTrack
81.5 / 93.1
77.7 / 90.5
78.5 / 90.9
74.8 / 87.6
Improvement
+3.5 / +1.6
+4.5 / +5.2
+3.1 / +3.3
+2.1 / +3.9
Method
NuScenes
S (927)
M (966)
L (883)
XL (884)
Table 5: Performance on various sequence lengths on KITTI and NuScenes. We divide all tracklets into four groups based on sequence lengths, using the 25th, 50th, and 75th percentiles as thresholds, and denote them as S, M, L, and XL. Parentheses indicate the number of tracklets per group.
Figure 5: The diversity of the memory token. Color represents the index of the most similar token for each point.
Figure 6: Ablation on temporal capacity of background memory. Mean performance on the KITTI is reported.
Figure 7: Comparison of GPU memory overhead as the temporal capacity of foreground memory increases.
Setting
Car
Pedestrian
Van
Cyclist
Mean
K=16
74.8 / 86.8
68.6 / 93.0
62.4 / 76.7
78.1 / 94.8
71.1 / 88.7
K=32
76.0 / 88.6
68.6 / 94.0
64.2 / 77.7
77.7 / 94.8
71.8 / 90.1
K=64
74.8 / 86.7
67.7 / 92.8
63.5 / 76.4
76.7 / 94.4
70.8 / 88.6
Table 6: Performance comparison with different numbers of memory tokens K on the KITTI dataset.
Setting
Car
Pedestrian
Van
Cyclist
Mean
τcycle=
0.07
74.9 / 86.9
68.7 / 93.2
59.7 / 74.4
76.2 / 94.7
70.9 / 88.7
τcycle=
0.1
76.0 / 88.6
68.6 / 94.0
64.2 / 77.7
77.7 / 94.8
71.8 / 90.1
τcycle=
1.0
73.6 / 86.2
68.2 / 92.4
63.9 / 78.2
76.5 / 94.8
70.4 / 88.4
Table 7: Performance comparison with different temperature τcycle of memory cycle consistency loss on the KITTI dataset.
Setting
Car
Pedestrian
Van
Cyclist
Mean
τdist=10cm
74.9 / 86.7
69.4 / 93.3
64.1 / 78.0
76.6 / 94.6
71.6 / 89.0
τdist=30cm
76.0 / 88.6
68.6 / 94.0
64.2 / 77.7
77.7 / 94.8
71.8 / 90.1
τdist=70cm
74.5 / 86.8
67.9 / 92.9
63.8 / 77.6
76.9 / 94.8
70.8 / 88.8
Table 8: Performance comparison with different distance thresholds τdist on the KITTI dataset.
Mehod
# Params
Inference Time
FPS
Suc./Prec.
MBPTrack [ 21 ]
1.9M
15ms
67
73.4 / 84.8
M3SOT [ 11 ]
4.3M
48ms
21
75.9 / 87.4
ChronoTrack
2.9M
24ms
42
76.0 / 88.6
Table 9: Comparison of model size and inference time with state-of-the-art methods.
Visual object tracking follows objects across video frames, maintaining their locations and identities despite motion, occlusion, and changing appearance.
LiDAR-based 3D Single Object Tracking (3D SOT) is critical for robotic perception and navigation and aims to localize dynamic objects across frames in sparse point clouds. Existing methods, rooted in the Siamese tracking paradigm from 2D vision, rely on costly dual-input designs and excessive motion modeling guided by template priors, hindering their efficiency. Our in-depth analysis reveals: (i) the template paradigm is redundant, as the previous bounding box center encodes sufficient historical context; (ii) complex motion modeling is unnecessary, as geometric alignment provides adequate motion priors. Based on the above findings, we propose the first Template-Free Tracking framework (TFTrack). The novel framework eliminates the need for template-search pairings and operates directly on the current frame guided solely by the prior bounding box center and size. We instantiate this paradigm into three variants: TFTrack-Voxel, TFTrack-Pillar, and TFTrack-Point, to explore different 3D representations under a unified framework, ensuring flexibility across sparse and dense scenes. Extensive experiments on KITTI and nuScenes benchmarks show that TFTrack is competitive with leading template-based trackers, while reducing FLOPs by approximately 50% and running at approximately 120 FPS. By simplifying overcomplicated motion-centric designs, TFTrack establishes a new minimalist paradigm for efficient 3D point cloud tracking, paving the way for real-time and resource-efficient deployment in embedded robotic systems, such as autonomous vehicles. The code is available at https://github.com/tftrack-anonymous/TFTrack/tree/main.
Zhaofeng Hu, Sifan Zhou, Jiahao Nie +3
Stony Brook University · Carnegie Mellon University · Hangzhou Dianzi University +2
We present LIST3R, an instance-aware framework for long-sequence 3D reconstruction inspired by the way humans organize spatial memory around stable and recognizable objects. LIST3R organizes long-sequence reconstruction around instance anchors, using them to reconnect fragmented subsequences and consolidate local observations into a coherent global 3D scene. Given a long video, our approach partitions it into overlapping subsequences and builds a structured local instance library for each partial reconstruction, maintaining persistent trackable anchors with semantic and geometric evidence. These anchors are matched across subsequences to recover revisited regions and provide object-aware constraints for fragment alignment, producing a consistent global reconstruction. During this process, the evolving geometric evidence updates the local instance libraries and progressively organizes them into a unified global 3D instance library. Experiments on long-sequence benchmarks show that our method produces more accurate trajectories and higher-quality 3D reconstructions, highlighting the effectiveness of persistent instance anchors for organizing long-horizon 3D reconstruction. Our code is available on the project page: https://yixn965.github.io/LIST3R/.
Jing Gao, Wei Wang, Feiran Wang +1
Beijing Jiaotong University · University of Illinois Chicago
Existing point tracking models face a fundamental tradeoff: they can either track a sparse set of query points over long horizons, or track all points across only short clips. We introduce TrackEverything, a 3D point tracker that breaks this trade-off by representing videos as persistent 3D scene tracks in world coordinates. Grounded in the insight that videos are 2D projections of an underlying 3D world, TrackEverything decouples model complexity from video duration, allowing it to scale with unique physical scene geometry instead. Our approach introduces three key innovations. First, we employ a voxelization-based de-duplication mechanism at sliding-window boundaries to merge co-located tracks, preventing repeated observations of the same surface from redundantly accumulating. Second, we decompose tracking into an endpoint refiner that predicts each point's destination and static-versus-dynamic classification, followed by a lightweight trajectory refiner that decodes dense trajectories exclusively for dynamic points. Third, we propose 3D WAFT, replacing memory-prohibitive 4D correlation volumes with efficient feature sampling in the scene cloud. To the best of our knowledge, TrackEverything is the first 3D tracker capable of tracking all visible points across videos exceeding 1000 frames within 40 GB of GPU memory. On TAPVid-3D, TrackEverything outperforms all open-source all-frame dense 3D trackers by more than 20% APD on short clips, while remaining competitive with state-of-the-art sparse trackers on long sequences, despite tracking far more points.