Associating measurements with tracks is a crucial step in Multi-Object Tracking (MOT) to guarantee the safety of autonomous vehicles. To manage the exponentially growing number of track hypotheses, truncation becomes necessary. In the δ-Generalized Labeled Multi-Bernoulli (δ-GLMB) filter application, this truncation typically involves the ranked assignment problem, solved by Murty's algorithm or the Gibbs sampling approach, both with limitations in terms of complexity or accuracy, respectively. With the motivation to improve these limitations, this paper addresses the ranked assignment problem arising from data association tasks with an approach that employs Graph Neural Networks (GNNs). The proposed Ranked Assignment Prediction Graph Neural Network (RAPNet) uses bipartite graphs to model the problem, harnessing the computational capabilities of deep learning. The conclusive evaluation compares the RAPNet with Murty's algorithm and the Gibbs sampler, showing accuracy improvements compared to the Gibbs sampler.
Figures & tables
Fig. 1 : Architecture of the RAPNet. Left part: Encoding of the node and edge features. Before the multiplication step of xs and xt , the feature vectors are indexed with the respective edge index vectors to fit the dimensions. Right part: Decoding using LSTM layers and creation of predicted assignments. The channel sizes Cenc=32 , Clstm=128 and Cdec=32 are shown above each neural network layer. The final output ypred is of shape ∣E∣×kmax .
Fig. 2 : Plots of the two sweeps of the parameters kmax and νs evaluated with the scores accuracy, wp and cost. For readability reasons in the accuracy plots, only the accuracies of the first 4 solutions for the RAPNet with the post-processing and the Gibbs sampler are shown. The legends on the right are also valid for the left plots. In the legends, RAPNet-PP and RAPNet-a indicate the versions with and without post-processing, respectively.
Fig. 3 : Distribution of matrix sizes in the simulation data.
Framework
Accuracies
wp
Cost
i=1
i=2
i=3
i=4
RAPNet-a
0.99
0.91
0.73
0.54
2.74
5.68
RAPNet-PP
0.99
0.95
0.82
0.66
2.80
3.23
Gibbs
1
0.18
0.06
0.04
2.11
14.10
TABLE I : Comparison of the RAPNet versions to the Gibbs sampler on simulation data.
Rows
Graph
RAP-
RAP-
PP 32
PP
Gibbs
Murty
νs
Creation
Net 32
Net
1
0.29
0.12
3.04
0.32
0.53
0.05
0.03
3
0.30
0.15
3.03
0.39
0.59
0.12
0.14
5
0.30
0.19
3.06
0.45
0.64
0.19
0.28
7
0.31
0.25
3.11
0.47
0.68
0.26
0.43
9
0.31
0.32
3.15
0.48
0.70
0.32
0.61
TABLE II : Time consumption of the different modules, in ms.
We propose a graph-based tracking formulation for multi-object tracking (MOT) where target detections contain kinematic information and re-identification features (attributes). Our method applies a successive shortest paths (SSP) algorithm to a tracking graph defined over a batch of frames. The edge costs in this tracking graph are computed via a message-passing network, a graph neural network (GNN) variant. The parameters of the GNN, and hence, the tracker, are learned end-to-end on a training set of example ground-truth tracks and detections. Specifically, learning takes the form of bilevel optimization guided by our novel loss function. We evaluate our algorithm on simulated scenarios to understand its sensitivity to scenario aspects and model hyperparameters. Across varied scenario complexities, our method compares favorably to a strong baseline.
Griffin Golias, Masa Nakura-Fan, Vitaly Ablavsky
Applied Physics Laboratory University of Washington Seattle, WA 98105 USA · Paul G. Allen School of Computer Science University of Washington Seattle, WA 98105 USA
This paper presents DeepMTL2R, an open-source deep learning framework for Multi-task Learning to Rank (MTL2R), where multiple relevance criteria must be optimized simultaneously. DeepMTL2R integrates heterogeneous relevance signals into a unified, context-aware model by leveraging the self-attention mechanism of transformer architectures, enabling effective learning across diverse and potentially conflicting objectives. The framework includes 21 state-of-the-art multi-task learning algorithms and supports multi-objective optimization to identify Pareto-optimal ranking models. By capturing complex dependencies and long-range interactions among items and labels, DeepMTL2R provides a scalable and expressive solution for modern ranking systems and facilitates controlled comparisons across MTL strategies. We demonstrate its effectiveness on a publicly available dataset, report competitive performance, and visualize the resulting trade-offs among objectives. DeepMTL2R is available at https://github.com/amazon-science/DeepMTL2R.
Multi-object tracking (MOT) from UAV imagery presents unique challenges: altitude varies across sequences, objects are small and densely packed, and frequent occlusion causes identity switches. Existing graph-based trackers assume fixed spatial context and treat all objects uniformly, ignoring the heterogeneous lifecycle states of detections, active tracklets, and lost targets. We propose HDST-GNN, a Heterogeneous Dynamic Spatiotemporal Graph Neural Network with three novel contributions. First, Altitude-Adaptive Edge Construction estimates a camera-altitude proxy from mean object area and adjusts the graph connectivity radius accordingly. Second, Heterogeneous Node Representation models detections (Type-D), confirmed tracklets (Type-T), and lost tracklets (Type-L) as distinct node types with dedicated projections and typed edge relations. Third, Occlusion-Gated Temporal Aggregation gates each node's attention contribution by its occlusion confidence, preventing occluded nodes from corrupting neighbour embeddings. HDST-GNN is trained end-to-end with a differentiable Sinkhorn head using joint cross-entropy and triplet loss. On VisDrone2019-MOT with oracle detections, HDST-GNN achieves 94.51% MOTA and 97.24% IDF1, outperforming SORT by +5.0 MOTA points and reducing identity switches by 81%. With real YOLOv8n detections, HDST-GNN reduces identity switches by 49% vs. SORT. Ablation studies confirm the independent contribution of each component.