Fusion-Poly: A Polyhedral Framework Based on Spatial-Temporal Fusion for 3D Multi-Object Tracking

Xian Wu; Yitao Wu; Xiaoyu Li; Zijia Li; Lijun Zhao; Lining Sun

cs.CVMar 9, 2026

Fusion-Poly: A Polyhedral Framework Based on Spatial-Temporal Fusion for 3D Multi-Object Tracking

Authors: Xian Wu, Yitao Wu, Xiaoyu Li, Zijia Li, Lijun Zhao, Lining Sun

Organizations: State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150006, China.

Abstract

LiDAR-camera 3D multi-object tracking (MOT) combines rich visual semantics with accurate depth cues to improve trajectory consistency and tracking reliability. In practice, however, LiDAR and cameras operate at different sampling rates. To maintain temporal alignment, existing data pipelines usually synchronize heterogeneous sensor streams and annotate them at a reduced shared frequency, forcing most prior methods to perform spatial fusion only at synchronized timestamps through projection-based or learnable cross-sensor association. As a result, abundant asynchronous observations remain underexploited, despite their potential to support more frequent association and more robust trajectory estimation over short temporal intervals. To address this limitation, we propose Fusion-Poly, a spatial-temporal fusion framework for 3D MOT that integrates asynchronous LiDAR and camera data. Fusion-Poly associates trajectories with multi-modal observations at synchronized timestamps and with single-modal observations at asynchronous timestamps, enabling higher-frequency updates of motion and existence states. The framework contains three key components: a frequency-aware cascade matching module that adapts to synchronized and asynchronous frames according to available detection modalities; a frequency-aware trajectory estimation module that maintains trajectories through high-frequency motion prediction, differential updates, and confidence-calibrated lifecycle management; and a full-state observation alignment module that improves cross-modal consistency at synchronized timestamps by optimizing image-projection errors. On the nuScenes test set, Fusion-Poly achieves 76.5% AMOTA, establishing a new state of the art among tracking-by-detection 3D MOT methods. Extensive ablation studies further validate the effectiveness of each component. Code will be released.

Figures & tables

Fig. 1: Fusion-Poly differs from existing LiDAR-Camera 3D MOT methods by explicitly exploiting asynchronous sensor data at different frequencies, increasing matching frequency, and improving tracking robustness.

Fig. 2: The pipeline of Fusion-Poly. We propose a unified LiDAR-Camera 3D MOT framework that jointly performs cross-modal fusion and cross-frequency integration by simultaneously incorporating synchronous and asynchronous data.

Method	Detector	Input	AMOTA $\uparrow$	MOTA $\uparrow$	IDS $\downarrow$
MotionTrack [ 31 ]	MotionTrack	C + L	55.0	49.0	8716
EagerMOT [ 6 ]	CP & CR	C + L	67.7	56.8	1156
Poly-MOT [ 1 ]	LK-F	C + L	75.4	62.1	292
CAMO-MOT [ 20 ]	BEVF & FC	C + L	75.3	63.5	324
Fast-Poly [ 2 ]	LK-F	C + L	75.8	62.8	326
MC-Track [ 32 ]	LK-F	C + L	76.3	63.4	242

TABLE I: Performance comparison on the nuScenes test set. CP: CenterPoint [ 25 ] ; LK-F: LargeKernel-F [ 26 ] ; MV2D: Mv2DFusion [ 27 ] ; CR : Cascade R-CNN [ 28 ] ; BEVF: BEVFusion [ 29 ] ; FC: FocalsConv [ 30 ] .

Method	Detector	Input	AMOTA $\uparrow$	MOTA $\uparrow$	IDS $\downarrow$
MotionTrack [ 31 ]	MotionTrack	C + L	64.1	59.7	-
EagerMOT [ 6 ]	CP & CR	C + L	71.2	-	899
DINO-MOT [ 21 ]	CP & DINO	C + L	73.1	-	216
Fast-Poly [ 2 ]	CP	L	73.7	63.2	414
MC-Track [ 32 ]	CP	L	74.0	64.0	275
EMMS-MOT [ 7 ]	CP & CR	C + L	74.3	-	247

TABLE II: Performance comparison on the nuScenes val set. CP: CenterPoint [ 25 ] ; LK-F: LargeKernel-F [ 26 ] ; MV2D: Mv2DFusion [ 27 ] ; CR: Cascade R-CNN [ 28 ] ; BEVF: BEVFusion [ 29 ] ; FC: FocalsConv [ 30 ] ; DINO: DINOv2 [ 33 ] .

Index	Async. Data	FACM	FATE	GAAM	AMOTA $\uparrow$	AMOTP $\downarrow$	MOTA $\uparrow$	IDS $\downarrow$	FP $\downarrow$	FN $\downarrow$
Exp0	–	–	–	–	75.5	0.557	65.9	884	12841	16804
Exp1	–	✓	–	–	76.6	0.543	66.7	470	13332	15935
Exp2	–	✓	✓	–	76.8	0.538	66.7	443	13189	15917
Exp3	–	✓	✓	✓	76.9	0.541	67.0	446	13131	15894
Exp4	✓	–	–	–	75.4	0.565	65.6	886	12330	17425
Exp5	✓	✓	–	–	76.6	0.557	67.3	463	13168	15678

TABLE III: Ablation experiment of Fusion-Poly on nuScenes val set. Async. Data : using 4 Hz camera-only 2D detections; FACM: matching for sync./async. frames; FATE: confidence-calibrated lifecycle management; GAAM: geometry-aware alignment.

Metric	AMOTA $\uparrow$	MOTA $\uparrow$	IDS $\downarrow$
Eucl	77.0	67.2	420
GIoU	77.0	67.2	414
IoU	77.1	67.4	416

TABLE IV: Comparison of different metric settings in GAAM on nuScenes val set. Eucl means Euclidean distance.

Method	Deviation	AMOTA $\uparrow$	MOTA $\uparrow$	Degradation $\downarrow$
EagerMOT $\dagger$	-	70.3	60.5	-
	0.1	49.3	40.0	-29.9%
	0.2	40.4	34.3	-42.5%
	0.3	36.5	31.6	-48.1%
Fusion-Poly	-	77.1	67.3	-
	0.1	66.5	57.7	-13.8%

TABLE V: Comparison of noise robustness on the nuScenes val set. Zero-mean Gaussian noise with varying standard deviations is injected into camera extrinsics to simulate real-world sensor noise.

\dagger

: We reproduced the results of EagerMOT [ 6 ] using the official code. All methods use the same detectors: CenterPoint [ 25 ] and Cascade R-CNN [ 28 ] .

Method	MA	P3DA	P2DA	AMOTA $\uparrow$	MOTA $\uparrow$	IDS $\downarrow$
Exp0	✓	-	-	75.5	65.6	809
Exp1	✓	✓	-	76.5	66.5	420
Exp2	✓	✓	✓	77.1	67.3	416

TABLE VI: Comparison of association strategy on nuScenes val set. MA, P3DA, P2DA are defined in Section III-C .

Fig. 3: Visualization results of Fusion-Poly with different inputs. In this figure, 4 Hz and 2 Hz denote the input frequency of Cascade R-CNN results, and both settings adopt 2 Hz CenterPoint results.

Method	AMOTA $\uparrow$	MOTA $\uparrow$	IDS $\downarrow$
Average	72.9	61.7	554
EMA	73.8	62.4	526
Max	75.9	65.7	484
Noisy-OR (Ours)	77.1	67.3	416

TABLE VII: Comparison of different score update strategies between

s_{\text{fused}}

and

s_{t,t-1}

on nuScenes val set. EMA: exponentially weighted moving average.

Fig. 4: Performance comparison with identical key parameters across all categories.

3D MOT

3D multi-object tracking (3D MOT) estimates object locations and identities over time from cameras, LiDAR, radar, and other sensors.

See all 40 papers

Apr 15, 2026

Radar-Informed 3D Multi-Object Tracking under Adverse Conditions

The challenge of 3D multi-object tracking is achieving robustness in real-world applications, for example under adverse conditions and maintaining consistency as distance increases. To overcome these challenges, sensor fusion approaches that combine LiDAR, cameras, and radar have emerged. However, existing multimodal methods usually treat radar as another learned feature inside the network. When the overall model degrades in difficult environments, the robustness advantages that radar could provide are also reduced. In this paper we propose RadarMOT, a radar-informed 3D multi-object tracking framework that explicitly uses radar point clouds as additional observations to refine state estimation and recover objects missed by the detector at long ranges. Evaluations on the MAN-TruckScenes dataset show that RadarMOT consistently improves the Average Multi-Object Tracking Accuracy (AMOTA) by 12.7% at long range and up to 10.3% in adverse weather. The code will be available at https://github.com/bingxue-xu/radarmot

Bingxue Xu, Emil Hedemalm, Ajinkya Khoche +1

KTH Royal Institute of Technology, Stockholm 10044, Sweden.

3D MOT Multimodal Robustness Multimodal Sensor Fusion Autonomous Driving

Ask PDF

Sep 22, 2026

CDKF-Track: Cluster-aware Data-Driven Kalman Filtering for Cooperative 3D Multi-Object Tracking

Multi-Object Tracking (MOT) is essential for EdgeAI perception systems, where accurate object localization and reliable identification enable safe decision-making. Singleagent MOT suffers from occlusions, sensor noise, and partial scene understanding in complex real-world scenarios. While multi-agent systems improve robustness by exploiting shared information, they introduce redundant measurements that lead to false data associations, and still struggle to capture nonlinear object dynamics. To address these challenges, we propose CDKFTrack, a Cluster-aware Data-Driven Kalman Filtering framework for Cooperative 3D MOT. The proposed method first fuses multivehicle 3D LiDAR detections through a Graph Laplacian-based formulation. Then, a cluster-aware redundancy reduction scheme groups spatially related detections and selects representative observations to reduce duplicate inputs to the tracker. The resulting detections are processed by a data-driven Kalman filter that learns object motion dynamics from data, reducing dependence on predefined linear motion assumptions. Furthermore, a wavelet-based temporal refinement module leverages the multiresolution decomposition property of wavelets to attenuate shortterm positional fluctuations and improve trajectory continuity. To the best of our knowledge, CDKF-Track is the first framework to jointly address detection-level fusion redundancy and learnable motion modeling in cooperative 3D MOT. Experimental results on the real-world V2V4Real dataset indicate that CDKF-Track achieves up to 27.99% improvements in tracking accuracy over state-of-the-art multi-agent MOT methods.

Maria Damanaki, Nikos Piperigkos, Alexandros Gkillas +1

Industrial Systems Institute, Athena Research Center, Patras Science Park, Greece · 3Dpt. of Informatics & Telecom., University of Ioannina, Arta, Greece · Avisense.AI, Patras, Greece +1

Kalman Filtering Cooperative Perception 3D MOT

Ask PDF

Oct 27, 2026

FastBEVFusion: Simple Multi-Modal Fusion for 3D Object Detection at 50 FPS

Reliable real-time 3D object detection based on LiDAR and camera data is critical for safety in autonomous driving, where systems must accurately perceive their surroundings with minimal latency. Many state-of-the-art systems achieve high-quality detection results but fail to meet real-time constraints, limiting their practical use. To address this challenge, we propose FastBEVFusion, a simple, fast, and robust fusion system for 3D object detection. Based on modality-specific features, FastBEVFusion projects these features into semi-aligned modality-specific BEV spaces, without relying on a computationally intensive and potentially error-prone depth estimation. Given these BEV spaces, we propose a light-weight, soft, and dynamic fusion module based on cross-attention, which adapts dynamically to the scene context and enables feature fusion across the entire BEV spaces. Experiments on the complex nuScenes dataset show that FastBEVFusion outperforms state-of-the-art real-time systems, and achieves over 50 FPS without deployment libraries. Additionally, our system shows strong robustness to calibration errors, making FastBEVFusion a powerful 3D object detection system for real-world applications.

Tom Sanitz, Emre Gülsoylu, Simone Frintrop +1

University of Hamburg, Hamburg, Germany

3D Object Detection Camera-LiDAR Fusion Feature Fusion

Ask PDF

Abstract

3D MOT

Explore similar work

Radar-Informed 3D Multi-Object Tracking under Adverse Conditions

CDKF-Track: Cluster-aware Data-Driven Kalman Filtering for Cooperative 3D Multi-Object Tracking

FastBEVFusion: Simple Multi-Modal Fusion for 3D Object Detection at 50 FPS