Organizations: National University of Singapore · Shenzhen Tsinghua International Graduate School, Tsinghua Univeristy · Xiaomi EV · Independent Researcher · Chinese University of Hong Kong · The University of Manchester · Renmin University of China
High-definition (HD) maps are essential for autonomous driving, yet multi-modal fusion often suffers from inconsistency between camera and LiDAR modalities, leading to performance degradation under low-light conditions, occlusions, or sparse point clouds. To address this, we propose SEFMAP, a Subspace-Expert Fusion framework for robust multimodal HD map prediction. The key idea is to explicitly disentangle BEV features into four semantic subspaces: LiDAR-private, Image-private, Shared, and Interaction. Each subspace is assigned a dedicated expert, thereby preserving modality-specific cues while capturing cross-modal consensus. To adaptively combine expert outputs, we introduce an uncertainty-aware gating mechanism at the BEV-cell level, where unreliable experts are down-weighted based on predictive variance, complemented by a usage balance regularizer to prevent expert collapse. To enhance robustness in degraded conditions and promote role specialization, we further propose distribution-aware masking: during training, modality-drop scenarios are simulated using EMA-statistical surrogate features, and a specialization loss enforces distinct behaviors of private, shared, and interaction experts across complete and masked inputs. Experiments on nuScenes and Argoverse2 benchmarks demonstrate that SEFMAP achieves state-of-the-art performance, surpassing prior methods by +4.2% and +4.8% in mAP, respectively. SEF-MAPprovides a robust and effective solution for multi-modal HD map prediction under diverse and degraded conditions.
Figures & tables
Fig. 1: Overall architecture of SEF-MAP framework. The system takes multi-view images and LiDAR point clouds as inputs, which are encoded into BEV features through respective encoders. These features are then decomposed into four semantic subspaces (LiDAR-private, Image-private, Shared, and Interaction) via linear transformations. During training, distribution-aware masking creates surrogate features using EMA statistics to simulate modality degradation scenarios, with specialization losses enforcing expert roles. During inference, only the intact forward pass is performed without masking. A mixture of experts processes each subspace, and an uncertainty-aware gating mechanism dynamically weights expert outputs based on their predicted variance to generate the final HD map prediction.
Method
Modality
Backbone
Epochs
APped
APdivider
APboundary
mAP
HDMapNet [ 67 ]
C
E-B0
30
14.4
21.7
33.0
23.0
HDMapNet [ 67 ]
L
PointPillars
30
10.4
24.1
37.9
24.1
HDMapNet [ 67 ]
C&L
E-B0 & PointPillars
30
16.3
29.6
46.7
31.0
VectorMapNet [ 68 ]
C
R50
110
36.1
47.3
39.3
40.9
VectorMapNet [ 68 ]
L
PointPillars
110
25.7
37.6
38.6
34.0
VectorMapNet [ 68 ]
C&L
R50 & PointPillars
110
37.6
50.5
47.5
45.2
TABLE I: Comparisons with state-of-the-art methods on nuScenes val set. Our method outperforms all existing approaches in both single-class APs and overall mAP.
Method
Modality
Backbone
Epochs
APped
APdivider
APboundary
mAP
HDMapNet [ 67 ]
C
E-B0
30
13.1
5.7
37.6
18.8
VectorMapNet [ 68 ]
C
R50
110
38.3
36.1
39.2
37.9
MapTR † [ 69 ]
C
R50
6
58.7
59.3
60.3
59.4
MapTR † [ 69 ]
C&L
R50 & SECOND
6
65.1
61.6
75.1
67.3
SEF-Map (Ours)
C&L
R50 & SECOND
6
70.7
66.7
78.8
72.1(+4.8)
TABLE II: Results on Argoverse2 dataset. † denotes our re-implementation following the setting in the paper.
Fig. 2: Qualitative results on nuScenes. We present two sample scenes from nuScenes: (a) Ground Truth. (b) Baseline (MapTR). (c) SEF-MAP (SD and DAM). (d) SEF-MAP (full). The red boxes highlight challenging regions where our method demonstrates significant improvements over the baseline. In these highlighted areas, the baseline MapTR produces noticeable errors and incomplete map predictions. The full SEF-MAP model with uncertainty-aware gating (d) achieves the most precise vectorized map reconstruction, particularly in the challenging regions marked by red boxes, demonstrating the effectiveness of our subspace decomposition and adaptive fusion strategy.
SD
DAM
UAG
Mod.
APped
APdivider
APboundary
mAP
×
×
×
C&L
55.9
62.3
69.3
62.5
✓
×
×
C&L
56.8
63.1
70.2
63.4
×
✓
×
C&L
57.7
64.6
71.0
64.4
×
×
✓
C&L
56.1
62.8
69.7
62.9
✓
✓
×
C&L
59.1
66.7
72.0
65.9
✓
×
✓
C&L
59.2
64.9
70.7
64.9
TABLE III: Ablation on subspace decomposition (SD), distribution-aware masking (DAM), and uncertainty-aware gating (UAG). The full model combining all three components achieves the state-of-the-art performance, outperforming individual modules and pairwise combinations on nuScenes val (24 epochs).
Expert Setting
APped
APdivider
APboundary
mAP
MapTR [ 69 ] (baseline)
55.9
62.3
69.3
62.5
Only L+I (Private)
57.7
64.2
69.4
63.8
Only S+Int (Cross-modal)
54.6
60.5
66.7
60.6
SEF-Map (Full)
61.6
66.7
71.8
66.7
TABLE IV: Expert group ablation configurations and results . The full model achieves the best performance, significantly outperforming configurations with only private experts (L+I) or only cross-modal experts (S+Int).
High-definition (HD) maps are essential for autonomous driving systems. In constructing such maps, onboard multi-view camera images, standard-definition maps and satellite images provide crucial information. However, due to the modality and perspective differences among these data sources, existing methods often struggle to effectively align and fuse them, making online HD map construction still challenging. To address these issues, we propose Driver2Map, an online HD map construction model inspired by human drivers. Unlike existing HD map construction models that utilize only two modalities, our Driver2Map can simultaneously exploit three modalities. Specifically, we propose a "two-stage alignment" strategy to reduce spatial misalignment across different modalities. Additionally, we introduce "Pose-Guided BEV Fusion", a BEV (bird's-eye-view) generation module that leverages camera pose information to adaptively weight multi-view features, thereby effectively suppressing cross-view feature overlap during BEV generation. Also, we design a "Pretrained Prior for Map Refinement" module to refine the initial prediction by learning map structure priors, thus improving the HD map prediction under dynamic occlusions. Extensive experiments demonstrate that Driver2Map outperforms existing methods on both IoU and AP metrics.
Online High-Definition (HD) map construction is a key component of autonomous driving. Recent methods rely on multi-view camera images for cost-effective HD map segmentation, but cameras lack depth information for accurate scene geometry. In contrast, LiDAR provides precise 3D measurements but lacks dense semantic cues. In this work, we propose LIE, LiDAR-only semantic map construction method that employ Knowledge Distillation (KD) to handle the lack of dense semantic and texture cues. Specifically, the teacher branch fuses student LiDAR features and the corresponding 2D intensity map tile to provide dense supervision for segmenting map elements using online distillation scheme. Experimental results show that our method outperforms all single-modality approaches, achieving 8.2% higher mIoU than the state-of-the-art camera-based model on nuScenes. LIE is robust over long ranges and under challenging weather and lighting, and efficiently adapts to Argoverse2 with only 10% fine-tuning, surpassing camera-based models trained on the full dataset. Source code will be available here.
Kanak Mazumder, Fabian B. Flohr
Munich University of Applied Sciences, Intelligent Vehicles Lab (IVL), Munich, Germany
Autonomous driving systems benefit from high-definition (HD) maps that provide critical information about road infrastructure. The online construction of HD maps offers a scalable approach to generate local vectorized maps from onboard sensor observations. Existing methods commonly adopt bird's-eye-view (BEV) features as the intermediate scene representation, encoding the surrounding space with fixed-resolution dense grids. However, map elements are spatially sparse yet require fine-grained geometric localization, making uniformly allocated BEV representations redundant and less effective for vectorized map prediction. In this work, we propose GaussianMap, an online HD map construction framework that learns an adaptive Gaussian representation of the surrounding scene. This representation consists of a set of Gaussian primitives on the BEV plane, each encoding a flexible local region with geometric properties and a feature vector, allowing the model to allocate representational capacity to map-relevant regions. To generate such a representation from sensor observations, we introduce a feed-forward Gaussian encoder that progressively refines these primitives through Gaussian interaction modeling and multi-sensor feature aggregation. The refined Gaussian representation is then splatted into a BEV feature map and decoded into vectorized map predictions. Extensive experiments on nuScenes and Argoverse 2 datasets demonstrate that GaussianMap achieves state-of-the-art performance in both camera-only and camera-LiDAR fusion settings. Our code will be made publicly available.
Hongyu Lyu, Julie Stephany Berrio Perez, Mao Shan +1
The University of Sydney, Australian Centre for Robotics, Sydney, Australia. · The University of Queensland, Australia.