LEO: Graph Attention Network based Hybrid Multi Sensor Extended Object Fusion and Tracking for Autonomous Driving Applications

Mayank Mayank; Bharanidhar Duraisamy; Florian Geiss

cs.LGApr 2, 2026

LEO: Graph Attention Network based Hybrid Multi Sensor Extended Object Fusion and Tracking for Autonomous Driving Applications

Authors: Mayank Mayank, Bharanidhar Duraisamy, Florian Geiss

Organizations: Research and Development, Mercedes-Benz AG, Germany

Abstract

Accurate shape and trajectory estimation of dynamic objects is essential for reliable automated driving. Classical Bayesian extended-object models offer theoretical robustness and efficiency but depend on completeness of a-priori and update-likelihood functions, while deep learning methods bring adaptability at the cost of dense annotations and high compute. We bridge these strengths with LEO (Learned Extension of Objects), a spatio-temporal Graph Attention Network that fuses multi-modal production-grade sensor tracks to learn adaptive fusion weights, ensure temporal consistency, and represent multi-scale shapes. Using a task-specific parallelogram ground-truth formulation, LEO models complex geometries (e.g. articulated trucks and trailers) and generalizes across sensor types, configurations, object classes, and regions, remaining robust for challenging and long-range targets. Evaluations on the Mercedes-Benz DRIVE PILOT SAE L3 dataset demonstrate real-time computational efficiency suitable for production systems; additional validation on public datasets such as View of Delft (VoD) further confirms cross-dataset generalization.

Figures & tables

Panel of Figure 1 — Fig. 1 : Mercedes-Benz EQS sensors used by DRIVE PILOT [ 28 ] (a) and auto-labelling pipeline (b).

Panel of Figure 2 — Fig. 2 : Comparison of sensor field-of-view–based shape abstractions under occlusions, with overlays of the FOVs for RADAR (60°) and LiDAR (120°), along with oval-shaped extension point covariances and lane-wise points for evaluation (a) and multi-sensor target shape segments (b).

Fig. 3 : Statistics of target vehicle characteristics in train and test sequences

Fig. 4 : LEO architecture : Tracks from multi-modality sensors are first embedded with state vectors and timestamps, concatenated across six frames (120 ms), and represented as a spatio-temporal graph with intra- and inter-modal edges for GAT-based attention. The LEO architecture then projects inputs (

128\times 48\times 11

) into a latent space (

128\times 48\times 128

), processes them through four stacked GATConv [ 37 ] layers with dual attention, normalization, ELU activation, dropout, and residual connections, and aggregates multi-head outputs via pooling into

128\times 128

embeddings. A final feed-forward projection maps these to

128\times 8

parallelogram parameters

\hat{\mathbf{y}}

, enabling efficient joint spatio-temporal reasoning for shape fusion.

Panel of Figure 5 — Fig. 5 : Parallelogram object representation with velocity vector represented as an arrow from the reference point, which is at the left-rear vertex (a) and the proposed Shape Fusion architecture (b).

	Driving	Cut-Ins	Hours	Fusion Objects
Train Sequence	326	410	12.3 hrs	1.46 mil.
Test Sequence	79	60	2.31 hrs	0.44 mil.

TABLE I: Dataset composition for training and testing sequences.

Parameter	$l_{1}$				$l_{2}$
	with $\alpha_{ij}^{\text{inter}}$		without $\alpha_{ij}^{\text{inter}}$		with $\alpha_{ij}^{\text{inter}}$		without $\alpha_{ij}^{\text{inter}}$
	MAE	Error (%)	MAE	Error (%)	MAE	Error (%)	MAE	Error (%)
GIoU (-)	0.78	–	0.27	–	0.76	–	0.31	–
DIoU (-)	0.82	–	0.23	–	0.76	–	0.34	–
$RF_{x}$ (m)	0.21	0.60	3.98	11.45	0.40	1.35	3.18	10.49
$RF_{y}$ (m)	0.11	2.94	0.64	16.42	0.14	4.95	0.67	23.67

TABLE II : Global KPIs for Shape Estimation of Fused Objects for LEO

Class	Cfg	Recall (IoU=0.25)	mASE	mAOE	mAVE
Car	L	0.75	0.40	0.60	1.27
Car	L+R	0.81	0.16	0.58	0.45
Pedestrian	L	0.73	0.20	0.85	1.01
Pedestrian	L+R	0.75	0.33	0.58	0.51
Cyclist	L	0.32	0.66	0.93	4.17
Cyclist	L+R	0.64	0.26	0.39	1.14

TABLE III: Class-wise performance comparison for LiDAR-only (CenterPoint [ 43 ] ) and LiDAR+RADAR (LEO) for test split on VoD dataset.

GIoU	CP x	CP y	FP x	FP y
0.55 / 0.22	1.39 / 0.36	0.28 / 0.16	2.37 / 13.39	0.35 / 0.49
0.76 / 0.48	0.48 / 0.43	0.22 / 1.30	1.02 / 7.54	0.28 / 0.58
0.75 / 0.47	0.52 / 0.64	0.11 / 0.12	1.11 / 7.21	0.16 / 0.34
0.89 / 0.83	0.33 / 0.35	0.10 / 0.18	0.45 / 1.12	0.13 / 0.43
0.76 / 0.73	0.69 / 0.39	0.30 / 0.31	1.27 / 2.64	0.37 / 0.46
0.74 / 0.64	0.93 / 0.82	0.17 / 0.17	1.60 / 3.53	0.23 / 0.35

TABLE IV: Ablation Study : Lane-wise performance analysis for LEO without LRR, LRL, and SMPC. For each setting, results are reported across three consecutive rows corresponding to the ego lane (EL), left lane (LL), and right lane (RL).

GIoU	CP x	CP y	FP x	FP y
0.91 / 0.84	0.10 / 0.27	0.07 / 0.16	0.21 / 0.87	0.10 / 0.37
0.79 / 0.77	0.19 / 0.34	0.20 / 0.25	0.64 / 2.30	0.23 / 0.40
0.77 / 0.71	0.23 / 0.55	0.10 / 0.13	0.82 / 3.17	0.17 / 0.31

TABLE V: Lane-wise analysis for LEO. Each row corresponds to a lane (EL, LL, RL), and values report mean IoU (–) and MAE, grouped by length classification (

l_{1}/l_{2}

Panel of Figure 11 — Fig. 6 : Evaluation results across diverse real world scenarios.

Multi-Object Tracking

Multi-object tracking (MOT) links detections across video frames to maintain object identities through motion, occlusion, and changing viewpoints.

See all 193 papers

Multi-Modal Sensor Fusion using Hybrid Attention for Autonomous Driving
Apr 6, 2026Mayank Mayank, Bharanidhar Duraisamy, Florian Geiß +1Camera-Radar Fusion 3D Object Detection
Research and Development, Mercedes-Benz AG, Germany · Department of Computer Science, University of Freiburg, Germany
Fusion4CA: Boosting 3D Object Detection via Comprehensive Image Exploitation
Mar 5, 2026Kang Luo, Xin Chen, Yangyi Xiao +13D Object Detection Camera-LiDAR Fusion
IRMV Lab, the Department of Automation, Shanghai Jiao Tong University.
Think as Needed: Geometry-Driven Adaptive Perception for Autonomous Driving
May 11, 2026Donghyun Kim, Jaehyoung ParkAdaptive Inference Lidar-Based 3D Object Detection
Stony Brook University

Apr 6, 2026J/K move · Enter open · S save

Multi-Modal Sensor Fusion using Hybrid Attention for Autonomous Driving

Mayank Mayank, Bharanidhar Duraisamy, Florian Geiß, Abhinav Valada

Research and Development, Mercedes-Benz AG, Germany · Department of Computer Science, University of Freiburg, Germany

Camera-Radar Fusion 3D Object Detection

Open paper AskPDF arXiv

Accurate 3D object detection for autonomous driving requires complementary sensors. Cameras provide dense semantics but unreliable depth, while millimeter-wave radar offers precise range and velocity measurements with sparse geometry. We propose MMF-BEV, a radar-camera BEV fusion framework that leverages deformable attention for cross-modal feature alignment on the View-of-Delft (VoD) 4D radar dataset [1]. MMF-BEV builds a BEVDepth [2] camera branch and a RadarBEVNet [3] radar branch, each enhanced with Deformable Self-Attention, and fuses them via a Deformable Cross-Attention module. We evaluate three configurations: camera-only, radar-only, and hybrid fusion. A sensor contribution analysis quantifies per-distance modality weighting, providing interpretable evidence of sensor complementarity. A two-stage training strategy - pre-training the camera branch with depth supervision, then jointly training radar and fusion modules stabilizes learning. Experiments on VoD show that MMF-BEV consistently outperforms unimodal baselines and achieves competitive results against prior fusion methods across all object classes in both the full annotated area and near-range Region of Interest.

Abstract

Multi-Object Tracking

Explore similar work

Multi-Modal Sensor Fusion using Hybrid Attention for Autonomous Driving

Fusion4CA: Boosting 3D Object Detection via Comprehensive Image Exploitation

Think as Needed: Geometry-Driven Adaptive Perception for Autonomous Driving