Organizations: National University of Singapore · Shenzhen Tsinghua International Graduate School, Tsinghua Univeristy · Xiaomi EV · Independent Researcher · Chinese University of Hong Kong · The University of Manchester · Renmin University of China
High-definition (HD) maps are essential for autonomous driving, yet multi-modal fusion often suffers from inconsistency between camera and LiDAR modalities, leading to performance degradation under low-light conditions, occlusions, or sparse point clouds. To address this, we propose SEFMAP, a Subspace-Expert Fusion framework for robust multimodal HD map prediction. The key idea is to explicitly disentangle BEV features into four semantic subspaces: LiDAR-private, Image-private, Shared, and Interaction. Each subspace is assigned a dedicated expert, thereby preserving modality-specific cues while capturing cross-modal consensus. To adaptively combine expert outputs, we introduce an uncertainty-aware gating mechanism at the BEV-cell level, where unreliable experts are down-weighted based on predictive variance, complemented by a usage balance regularizer to prevent expert collapse. To enhance robustness in degraded conditions and promote role specialization, we further propose distribution-aware masking: during training, modality-drop scenarios are simulated using EMA-statistical surrogate features, and a specialization loss enforces distinct behaviors of private, shared, and interaction experts across complete and masked inputs. Experiments on nuScenes and Argoverse2 benchmarks demonstrate that SEFMAP achieves state-of-the-art performance, surpassing prior methods by +4.2% and +4.8% in mAP, respectively. SEF-MAPprovides a robust and effective solution for multi-modal HD map prediction under diverse and degraded conditions.
Figures & tables
Fig. 1: Overall architecture of SEF-MAP framework. The system takes multi-view images and LiDAR point clouds as inputs, which are encoded into BEV features through respective encoders. These features are then decomposed into four semantic subspaces (LiDAR-private, Image-private, Shared, and Interaction) via linear transformations. During training, distribution-aware masking creates surrogate features using EMA statistics to simulate modality degradation scenarios, with specialization losses enforcing expert roles. During inference, only the intact forward pass is performed without masking. A mixture of experts processes each subspace, and an uncertainty-aware gating mechanism dynamically weights expert outputs based on their predicted variance to generate the final HD map prediction.
Method
Modality
Backbone
Epochs
APped
APdivider
APboundary
mAP
HDMapNet [ 67 ]
C
E-B0
30
14.4
21.7
33.0
23.0
HDMapNet [ 67 ]
L
PointPillars
30
10.4
24.1
37.9
24.1
HDMapNet [ 67 ]
C&L
E-B0 & PointPillars
30
16.3
29.6
46.7
31.0
VectorMapNet [ 68 ]
C
R50
110
36.1
47.3
39.3
40.9
VectorMapNet [ 68 ]
L
PointPillars
110
25.7
37.6
38.6
34.0
VectorMapNet [ 68 ]
C&L
R50 & PointPillars
110
37.6
50.5
47.5
45.2
TABLE I: Comparisons with state-of-the-art methods on nuScenes val set. Our method outperforms all existing approaches in both single-class APs and overall mAP.
Method
Modality
Backbone
Epochs
APped
APdivider
APboundary
mAP
HDMapNet [ 67 ]
C
E-B0
30
13.1
5.7
37.6
18.8
VectorMapNet [ 68 ]
C
R50
110
38.3
36.1
39.2
37.9
MapTR † [ 69 ]
C
R50
6
58.7
59.3
60.3
59.4
MapTR † [ 69 ]
C&L
R50 & SECOND
6
65.1
61.6
75.1
67.3
SEF-Map (Ours)
C&L
R50 & SECOND
6
70.7
66.7
78.8
72.1(+4.8)
TABLE II: Results on Argoverse2 dataset. † denotes our re-implementation following the setting in the paper.
Fig. 2: Qualitative results on nuScenes. We present two sample scenes from nuScenes: (a) Ground Truth. (b) Baseline (MapTR). (c) SEF-MAP (SD and DAM). (d) SEF-MAP (full). The red boxes highlight challenging regions where our method demonstrates significant improvements over the baseline. In these highlighted areas, the baseline MapTR produces noticeable errors and incomplete map predictions. The full SEF-MAP model with uncertainty-aware gating (d) achieves the most precise vectorized map reconstruction, particularly in the challenging regions marked by red boxes, demonstrating the effectiveness of our subspace decomposition and adaptive fusion strategy.
SD
DAM
UAG
Mod.
APped
APdivider
APboundary
mAP
×
×
×
C&L
55.9
62.3
69.3
62.5
✓
×
×
C&L
56.8
63.1
70.2
63.4
×
✓
×
C&L
57.7
64.6
71.0
64.4
×
×
✓
C&L
56.1
62.8
69.7
62.9
✓
✓
×
C&L
59.1
66.7
72.0
65.9
✓
×
✓
C&L
59.2
64.9
70.7
64.9
TABLE III: Ablation on subspace decomposition (SD), distribution-aware masking (DAM), and uncertainty-aware gating (UAG). The full model combining all three components achieves the state-of-the-art performance, outperforming individual modules and pairwise combinations on nuScenes val (24 epochs).
Expert Setting
APped
APdivider
APboundary
mAP
MapTR [ 69 ] (baseline)
55.9
62.3
69.3
62.5
Only L+I (Private)
57.7
64.2
69.4
63.8
Only S+Int (Cross-modal)
54.6
60.5
66.7
60.6
SEF-Map (Full)
61.6
66.7
71.8
66.7
TABLE IV: Expert group ablation configurations and results . The full model achieves the best performance, significantly outperforming configurations with only private experts (L+I) or only cross-modal experts (S+Int).