Accurate shape and trajectory estimation of dynamic objects is essential for reliable automated driving. Classical Bayesian extended-object models offer theoretical robustness and efficiency but depend on completeness of a-priori and update-likelihood functions, while deep learning methods bring adaptability at the cost of dense annotations and high compute. We bridge these strengths with LEO (Learned Extension of Objects), a spatio-temporal Graph Attention Network that fuses multi-modal production-grade sensor tracks to learn adaptive fusion weights, ensure temporal consistency, and represent multi-scale shapes. Using a task-specific parallelogram ground-truth formulation, LEO models complex geometries (e.g. articulated trucks and trailers) and generalizes across sensor types, configurations, object classes, and regions, remaining robust for challenging and long-range targets. Evaluations on the Mercedes-Benz DRIVE PILOT SAE L3 dataset demonstrate real-time computational efficiency suitable for production systems; additional validation on public datasets such as View of Delft (VoD) further confirms cross-dataset generalization.
Figures & tables
Fig. 1 : Mercedes-Benz EQS sensors used by DRIVE PILOT [ 28 ] (a) and auto-labelling pipeline (b).
Fig. 2 : Comparison of sensor field-of-view–based shape abstractions under occlusions, with overlays of the FOVs for RADAR (60°) and LiDAR (120°), along with oval-shaped extension point covariances and lane-wise points for evaluation (a) and multi-sensor target shape segments (b).
Fig. 3 : Statistics of target vehicle characteristics in train and test sequences
Fig. 4 : LEO architecture : Tracks from multi-modality sensors are first embedded with state vectors and timestamps, concatenated across six frames (120 ms), and represented as a spatio-temporal graph with intra- and inter-modal edges for GAT-based attention. The LEO architecture then projects inputs ( 128×48×11 ) into a latent space ( 128×48×128 ), processes them through four stacked GATConv [ 37 ] layers with dual attention, normalization, ELU activation, dropout, and residual connections, and aggregates multi-head outputs via pooling into 128×128 embeddings. A final feed-forward projection maps these to 128×8 parallelogram parameters y^ , enabling efficient joint spatio-temporal reasoning for shape fusion.
Fig. 5 : Parallelogram object representation with velocity vector represented as an arrow from the reference point, which is at the left-rear vertex (a) and the proposed Shape Fusion architecture (b).
Driving
Cut-Ins
Hours
Fusion Objects
Train Sequence
326
410
12.3 hrs
1.46 mil.
Test Sequence
79
60
2.31 hrs
0.44 mil.
TABLE I: Dataset composition for training and testing sequences.
Parameter
l1
l2
with αijinter
without αijinter
with αijinter
without αijinter
MAE
Error (%)
MAE
Error (%)
MAE
Error (%)
MAE
Error (%)
GIoU (-)
0.78
–
0.27
–
0.76
–
0.31
–
DIoU (-)
0.82
–
0.23
–
0.76
–
0.34
–
RFx (m)
0.21
0.60
3.98
11.45
0.40
1.35
3.18
10.49
RFy (m)
0.11
2.94
0.64
16.42
0.14
4.95
0.67
23.67
TABLE II : Global KPIs for Shape Estimation of Fused Objects for LEO
Class
Cfg
Recall (IoU=0.25)
mASE
mAOE
mAVE
Car
L
0.75
0.40
0.60
1.27
L+R
0.81
0.16
0.58
0.45
Pedestrian
L
0.73
0.20
0.85
1.01
L+R
0.75
0.33
0.58
0.51
Cyclist
L
0.32
0.66
0.93
4.17
L+R
0.64
0.26
0.39
1.14
TABLE III: Class-wise performance comparison for LiDAR-only (CenterPoint [ 43 ] ) and LiDAR+RADAR (LEO) for test split on VoD dataset.
GIoU
CP x
CP y
FP x
FP y
0.55 / 0.22
1.39 / 0.36
0.28 / 0.16
2.37 / 13.39
0.35 / 0.49
0.76 / 0.48
0.48 / 0.43
0.22 / 1.30
1.02 / 7.54
0.28 / 0.58
0.75 / 0.47
0.52 / 0.64
0.11 / 0.12
1.11 / 7.21
0.16 / 0.34
0.89 / 0.83
0.33 / 0.35
0.10 / 0.18
0.45 / 1.12
0.13 / 0.43
0.76 / 0.73
0.69 / 0.39
0.30 / 0.31
1.27 / 2.64
0.37 / 0.46
0.74 / 0.64
0.93 / 0.82
0.17 / 0.17
1.60 / 3.53
0.23 / 0.35
TABLE IV: Ablation Study : Lane-wise performance analysis for LEO without LRR, LRL, and SMPC. For each setting, results are reported across three consecutive rows corresponding to the ego lane (EL), left lane (LL), and right lane (RL).
GIoU
CP x
CP y
FP x
FP y
0.91 / 0.84
0.10 / 0.27
0.07 / 0.16
0.21 / 0.87
0.10 / 0.37
0.79 / 0.77
0.19 / 0.34
0.20 / 0.25
0.64 / 2.30
0.23 / 0.40
0.77 / 0.71
0.23 / 0.55
0.10 / 0.13
0.82 / 3.17
0.17 / 0.31
TABLE V: Lane-wise analysis for LEO. Each row corresponds to a lane (EL, LL, RL), and values report mean IoU (–) and MAE, grouped by length classification ( l1/l2 ).
Fig. 6 : Evaluation results across diverse real world scenarios.
Accurate 3D object detection for autonomous driving requires complementary sensors. Cameras provide dense semantics but unreliable depth, while millimeter-wave radar offers precise range and velocity measurements with sparse geometry. We propose MMF-BEV, a radar-camera BEV fusion framework that leverages deformable attention for cross-modal feature alignment on the View-of-Delft (VoD) 4D radar dataset [1]. MMF-BEV builds a BEVDepth [2] camera branch and a RadarBEVNet [3] radar branch, each enhanced with Deformable Self-Attention, and fuses them via a Deformable Cross-Attention module. We evaluate three configurations: camera-only, radar-only, and hybrid fusion. A sensor contribution analysis quantifies per-distance modality weighting, providing interpretable evidence of sensor complementarity. A two-stage training strategy - pre-training the camera branch with depth supervision, then jointly training radar and fusion modules stabilizes learning. Experiments on VoD show that MMF-BEV consistently outperforms unimodal baselines and achieves competitive results against prior fusion methods across all object classes in both the full annotated area and near-range Region of Interest.
Nowadays, an increasing number of works fuse LiDAR and RGB data in the bird's-eye view (BEV) space for 3D object detection in autonomous driving systems. However, existing methods suffer from over-reliance on the LiDAR branch, with insufficient exploration of RGB information. To tackle this issue, we propose Fusion4CA, which is built upon the classic BEVFusion framework and dedicated to fully exploiting visual input with plug-and-play components. Specifically, a contrastive alignment module is designed to calibrate image features with 3D geometry, and a camera auxiliary branch is introduced to mine RGB information sufficiently during training. For further performance enhancement, we leverage an off-the-shelf cognitive adapter to make the most of pretrained image weights, and integrate a standard coordinate attention module into the fusion stage as a supplementary boost. Experiments on the nuScenes dataset demonstrate that our method achieves 69.7% mAP with only 6 training epochs and a mere 3.48% increase in inference parameters, yielding a 1.2% improvement over the baseline which is fully trained for 20 epochs. Extensive experiments in a simulated lunar environment further validate the effectiveness and generalization of our method. Our code will be released through Fusion4CA.
Kang Luo, Xin Chen, Yangyi Xiao +1
IRMV Lab, the Department of Automation, Shanghai Jiao Tong University.
Autonomous driving scenes range from empty highways to dense intersections with dozens of interacting road users, yet current 3D detection models apply a fixed computation budget to every frame, wasting resources on simple scenes while lacking capacity for complex ones. Existing approaches compound this problem: Transformer-based interaction models scale quadratically with the number of detected objects, and frame-by-frame processing causes the system to immediately forget objects the moment they become occluded. We propose Enhanced HOPE, an adaptive perception architecture that measures the geometric complexity of each incoming LiDAR frame using an unsupervised statistical estimator and routes it through a shallow or deep processing path accordingly, requiring no manual scene labels. To keep interaction modeling efficient, we replace quadratic pairwise attention with a linear-time subspace-based network that groups nearby objects into clusters and processes them jointly. The computational savings from these two mechanisms free up resources for a persistent temporal memory module that retains previously detected objects and traffic rules across frames, enabling the system to recall occluded objects seconds after they disappear from view. On the nuScenes and CARLA benchmarks, Enhanced HOPE reduces latency by 38% on simple scenes with no accuracy loss, improves mean Average Precision by 2.7 points on rare long-tail scenarios, and tracks objects through occlusions lasting over 5 seconds, where all tested baselines fail.