Organizations: Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong, China · College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China · Institute of Information Engineering, CAS, Beijing, China · Xiaomi EV, Beijing, China
Robust 3D object detection in adverse weather is highly challenging due to the varying reliability of different sensors. While existing LiDAR-4D radar fusion methods improve robustness, they predominantly rely on fixed or weakly adaptive pipelines, failing to dy-namically adjust modality preferences as environmental conditions change. To bridge this gap, we reformulate multi-modal perception as a weather-conditioned branch routing problem. Instead of computing a single fused output, our framework explicitly maintains three parallel 3D feature streams: a pure LiDAR branch, a pure 4D radar branch, and a condition-gated fusion branch. Guided by a condition token extracted from visual and semantic prompts, a lightweight router dynamically predicts sample-specific weights to softly aggregate these representations. Furthermore, to prevent branch collapse, we introduce a weather-supervised learning strategy with auxiliary classification and diversity regularization to enforce distinct, condition-dependent routing behaviors. Extensive experiments on the K-Radar benchmark demonstrate that our method achieves state-of-the-art performance. Furthermore, it provides explicit and highly interpretable insights into modality preferences, transparently revealing how adaptive routing robustly shifts reliance between LiDAR and 4D radar across diverse adverse-weather scenarios. The source code with be released.
Figures & tables
Figure 1. Comparison between existing LiDAR-4D radar fusion methods and our proposed approach. Existing methods rely on a single, fixed fused representation, which struggles to adapt to changing environments. In contrast, our weather-conditioned branch routing explicitly maintains multiple representation branches and dynamically assigns adaptive weights based on the current weather condition.
Figure 2. Framework overview of the proposed weather-conditioned branch routing method. The model takes LiDAR, 4D radar, and camera images as inputs. First, a condition token is generated by fusing visual features and semantic text prompts, which is further refined by sensor context. Next, LiDAR and radar features are processed through three parallel streams: a LiDAR-only branch, a radar-only branch, and a condition-gated fusion branch. A branch router then utilizes the condition token to predict adaptive weights ( wL,wF,wR ). These weights softly aggregate the branch-wise BEV features for the final 3D detection head. During training, auxiliary weather classification and diversity losses are applied to ensure meaningful routing behavior.
Methods
Modality
IoU
Metric
Weather Condition
Total
Normal
Overcast
Fog
Rain
Sleet
Lightsnow
Heavysnow
RTNH ( Paek et al., 2022 )
4DR
0.3
APBEV
41.1
41.0
44.6
45.4
32.9
50.6
81.5
56.3
AP3D
37.4
37.6
42.0
41.2
29.2
49.1
63.9
43.1
0.5
APBEV
36.0
35.8
41.9
44.8
30.2
34.5
63.9
55.1
AP3D
14.1
19.7
20.5
15.9
13.0
13.5
21.0
6.36
RTNH* ( Paek et al., 2022 )
L
0.3
APBEV
76.5
76.5
88.2
86.3
77.3
55.3
81.1
59.5
Table 1. Quantitative results of LiDAR and 4D radar-based 3D object detection methods on K-Radar dataset. We present the modality of each method (L: LiDAR, 4DR: 4D radar) and detailed performance for each weather condition. Best in bold with blue background ( best ), second in light blue background ( second). * denotes our reproduced results.
Method
Branch Routing
Laux
Ldiv
AP BEV0.5
AP 3D0.5
AP BEV0.3
AP 3D0.3
w/o branch routing (force Fusion only)
✓
✓
54.8
22.0
78.6
68.8
w/o weather supervision ( Laux , Ldiv )
✓
70.7
43.0
77.4
73.9
w/o Ldiv
✓
✓
64.8
30.2
78.3
72.9
Full model
✓
✓
✓
71.5
43.5
81.2
76.8
Table 2. Ablation study of the proposed method on K-Radar. Branch routing denotes the weather-conditioned multi-branch aggregation, Laux is the auxiliary weather classification loss, and Ldiv is the weather diversity regularization.
Table 3. Comparison of different routing designs on K-Radar. The MLP-based router (Ours) achieves the best overall performance and shows stronger robustness across adverse weather conditions.
Hyperparameter
Value
AP 3D0.5 (%)
AP BEV0.5 (%)
λaux
0.00
27.5
56.3
0.05
43.5
70.8
0.10
43.5
71.5
0.20
33.9
63.4
λdiv
0.00
30.2
64.8
0.01
37.9
68.7
Table 4. Ablation study on key hyperparameters of the proposed method. We evaluate the impact of the auxiliary weather supervision weight ( λaux ), diversity regularization weight ( λdiv ), and entropy regularization weight ( λent ). Best results are highlighted in bold.
Figure 3. Qualitative results and interpretable routing behavior under diverse weather conditions. From top to bottom: Normal, Rain, Overcast, Sleet and Heavy Snow scenarios. For each scene, we visualize the camera view with 3D bounding box predictions (left), the adaptive branch weights wL,wF,wR predicted by the weather-conditioned router (middle), and the corresponding radar and LiDAR BEV representations (right).
Robust 3D object detection in adverse weather conditions is challenging due to sensor limitations. Although combining complementary modalities such as LiDAR and 4D RADAR has shown promise, the sparsity of these sensors becomes apparent in adverse weather with reduced reflections, leading to objects with few or no point cloud returns. To address this limitation, camera sensors provide visual cues even when LiDAR and RADAR signals are weakened. However, cameras themselves are also vulnerable to adverse weather, where some regions become unreliable due to snow or rain occluding the camera lens. While some camera-fusion methods designed for adverse weather learn to weigh image regions via confidence maps, these maps receive no direct supervision and are learned solely through the detection loss. We introduce Reliability-Aware Fusion (RAF), which explicitly supervises per-pixel reliability estimation and provides a direct learning signal for identifying and suppressing unreliable visual cues. Our framework leverages pretrained LiDAR-RADAR networks, keeping their backbones frozen while only training the added camera branch, BEV fusion encoder, and detection head. Extensive experiments on the K-Radar and VoD datasets demonstrate that integrating RAF consistently improves detection accuracy over LiDAR-RADAR baselines, achieving up to +6.5 APBEV and +7.4 AP3D gains. Code is available at https://github.com/parkie0517/RAF.
Robust 3D object detection under adverse weather remains a critical hurdle for autonomous driving. Despite progress with LiDAR-4D radar fusion, most methods are constrained by a closed-world assumption, implicitly requiring training and test weather to align in both type and severity. This premise fails in practice: the open-ended nature of weather, and even variations within a single type like rain, cause dramatically different LiDAR degradation patterns, leading to significant performance drops in unseen conditions. To address this, we present Dual-Critic Guided Diffusion Alignment (DCDA), a weather-agnostic framework that learns to recover degraded LiDAR features toward a clean manifold. Rather than modeling specific weather types, DCDA employs a 4D radar-conditioned diffusion process to progressively refine features, guided by two complementary critics. (i) A detection-guided critic, anchored by a pre-trained clean-weather model, ensures that the refined features retain object-level discriminability and localization accuracy. (ii) A weather adversarial critic enforces holistic distributional consistency with clean-weather representations. By aligning features through semantic and distributional constraints rather than explicit weather modeling, DCDA generalizes effectively to unseen weather types and severities without requiring paired data or weather labels. We further introduce a structured open-weather benchmark with held-out type-severity combinations and extensive experiments verify DCDA's advantages.
Shuyao Li, Chuanxing Geng, Heyang Sun +2
Nanjing University of Aeronautics and Astronautics, Nanjing, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China
Robust 3D object detection under adverse weather conditions is crucial for autonomous driving. However, most existing methods simply combine all weather samples for training while overlooking data distribution discrepancies across different weather scenarios, leading to performance conflicts. To address this issue, we introduce AW-MoE, the framework that innovatively integrates Mixture of Experts (MoE) into weather-robust multi-modal 3D object detection approaches. AW-MoE incorporates Image-guided Weather-aware Routing (IWR), which leverages the superior discriminability of image features across weather conditions and their invariance to scene variations for precise weather classification. Based on this accurate classification, IWR selects the top-K most relevant Weather-Specific Experts (WSE) that handle data discrepancies, ensuring optimal detection under all weather conditions. Additionally, we propose a Unified Dual-Modal Augmentation (UDMA) for synchronous LiDAR and 4D Radar dual-modal data augmentation while preserving the realism of scenes. Extensive experiments on the real-world dataset demonstrate that AW-MoE achieves ~ 15% improvement in adverse-weather performance over state-of-the-art methods, while incurring negligible inference overhead. Moreover, integrating AW-MoE into established baseline detectors yields performance improvements surpassing current state-of-the-art methods. These results show the effectiveness and strong scalability of our AW-MoE. We will release the code publicly at https://github.com/windlinsherlock/AW-MoE.
Hongwei Lin, Xun Huang, Chenglu Wen +1
Fujian Key Laboratory of Urban Intelligent Sensing and Computing and the School of Informatics, Xiamen University, Xiamen, FJ 361005, China · Zhongguancun Academy, Beijing, China