Multiobject tracking (MOT) is a fundamental capability in autonomous navigation, radar and sonar systems, as well as 6G integrated sensing-and-communication (ISAC) networks. In this paper, we propose a neural-enhanced track-before-detect (TkBD) framework that operates directly on complex-valued raw signals and learns a latent representation of unmodeled propagation effects during runtime. We introduce an environment-dependent latent intensity variable that characterizes the signal power of each object in the shared propagation environment. The probability density function of this intensity variable is represented by a normalizing flow and trained online in an unsupervised manner within the sequential Bayesian inference procedure. The learning objective is derived from a variational Bayes formulation by maximizing the evidence lower bound (ELBO), thereby accounting for the uncertainties inherent in the Bayesian TkBD problem. To enable scalable inference, we make use of a recently introduced Bayesian TkBD framework that combines belief propagation, which avoids high-dimensional operations, with sequential Monte Carlo methods for efficient low-dimensional updates.
Figures & tables
Fig. 1 : Multiobject tracking in a wireless communication network with a base station (BS) and user equipment (UE). Raw received data are used directly as measurements, enabling unmodeled propagation effects to be learned and exploited for improved tracking performance.
Fig. 2 : Block diagram of the normalizing-flow-based module used to learn the prior intensity PDF p(γ;c) online.
Object Index
Initial State (Pos. + Vel.)
Birth & Death Time
1
[10,25,0.5,0]
[2, 72]
2
[25,10,0,0.5]
[5, 75]
3
Random
[10, 70]
4
Random
[18, 78]
5
Random
[25, 75]
TABLE I : Parameters describing simulated object tracks.
Fig. 3 : Total GOSPA error and localization-error component obtained by the four considered tracking methods over 100 simulation runs.
Fig. 4 : Radar intensity maps. Left: ground-truth intensity ϱ(i)2 computed from the radar equations used for data generation. Right: mean of the learned prior intensity PDF p(γ;c) produced by the proposed neural-enhanced TkBD method.
Fig. 5 : Pointwise difference between the ground-truth radar-equation intensity and the learned prior intensity shown in Fig. 4 . The small residuals indicate that the proposed neural-enhanced TkBD method accurately learns the radar-intensity model online.
Fig. 6 : Representative tracking snapshots at time steps k=25 , 30 , 35 , and 40 , shown from top left to bottom right. Red crosses and black circles denote the true and estimated object positions at the current time step, respectively. Red curves show the true historical trajectories. The background shows the Bartlett spectrum obtained by matched filtering.
Kalman filter (KF)-based multi-object tracking (MOT) remains a strong baseline for autonomous driving due to its strong performance, computational efficiency and interpretability. In most practical systems, the process noise and measurement noise covariances are defined globally and shared across object classes, presuming identical uncertainty characteristics across heterogeneous traffic participants. This work revisits this assumption and proposes CANMOT, a class-aware and object-aligned noise modeling framework for KF-based 3D MOT. Class-specific diagonal process and measurement covariance matrices are introduced and optionally expressed in the object coordinate frame to preserve longitudinal-lateral anisotropy. Systematic experiments on the nuScenes benchmark show that class-aware and object-aligned noise modeling improves tracking performance and substantially reduces identity switches compared to state-of-the-art (SotA). In addition, the consistency of the estimated uncertainty is analyzed using the Average Normalized Estimation Error Squared (ANEES) and χ2-based violation tests. The results reveal severe overconfidence in standard KF-based MOT baselines. While the proposed formulation improves calibration without modifying the underlying filtering framework, it still exhibits substantial inconsistency, highlighting the need for further research in this area. Code is available at https://github.com/rst-tu-dortmund/learned-3d-nms.
Timo Osterburg, Stefan Schütte, Torsten Bertram
Institute of Control Theory and Systems Engineering, TU Dortmund University, Germany.
Multi-object tracking (MOT) plays a fundamental role in visual perception, where accurate trajectory prediction is essential for reliable target association under complex motion patterns. Recent trackers have improved motion modeling with densely activated artificial neural networks, yet they largely overlook whether such dense responses are necessary for trajectory prediction. In this paper, we formulate activation sparsity preference (ASP) by tackling two key questions: 1. How can we identify a model architecture that appropriately and formally explains ASP, and 2. How can we translate this explanation into competitive tracking performance. Theoretical analysis shows that sparse gating is no worse than state-independent dropout under the same activation rate. Based on this insight, SpikingMOT is proposed as a spike-driven tracker that adaptively models sparse trajectory dynamics with spiking neural networks (SNNs). Specifically, SpikingMOT decomposes each trajectory state into pseudo-trajectory bases and uses the current prediction error to calibrate the posterior for next-frame prediction. With this brain-inspired loop, SpikingMOT achieves state-of-the-art performance in extensive experiments, 74.9 HOTA on SportsMOT and 56.5 HOTA on DanceTrack, while reducing the parameters and energy by 72% and 86.7%, respectively. These results bring SNNs into MOT, opening a promising direction for efficient tracking.
Yiding Sun, Xiangyang Yang, Dongxu Zhang +7
1Xi’an Jiaotong University · 2Peking University · 3Guilin University of Technology
We propose a graph-based tracking formulation for multi-object tracking (MOT) where target detections contain kinematic information and re-identification features (attributes). Our method applies a successive shortest paths (SSP) algorithm to a tracking graph defined over a batch of frames. The edge costs in this tracking graph are computed via a message-passing network, a graph neural network (GNN) variant. The parameters of the GNN, and hence, the tracker, are learned end-to-end on a training set of example ground-truth tracks and detections. Specifically, learning takes the form of bilevel optimization guided by our novel loss function. We evaluate our algorithm on simulated scenarios to understand its sensitivity to scenario aspects and model hyperparameters. Across varied scenario complexities, our method compares favorably to a strong baseline.
Griffin Golias, Masa Nakura-Fan, Vitaly Ablavsky
Applied Physics Laboratory University of Washington Seattle, WA 98105 USA · Paul G. Allen School of Computer Science University of Washington Seattle, WA 98105 USA