AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection

Hongwei Lin; Xun Huang; Chenglu Wen; Cheng Wang

cs.CVMar 17, 2026

AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection

Authors: Hongwei Lin, Xun Huang, Chenglu Wen, Cheng Wang

Organizations: Fujian Key Laboratory of Urban Intelligent Sensing and Computing and the School of Informatics, Xiamen University, Xiamen, FJ 361005, China · Zhongguancun Academy, Beijing, China

Abstract

Robust 3D object detection under adverse weather conditions is crucial for autonomous driving. However, most existing methods simply combine all weather samples for training while overlooking data distribution discrepancies across different weather scenarios, leading to performance conflicts. To address this issue, we introduce AW-MoE, the framework that innovatively integrates Mixture of Experts (MoE) into weather-robust multi-modal 3D object detection approaches. AW-MoE incorporates Image-guided Weather-aware Routing (IWR), which leverages the superior discriminability of image features across weather conditions and their invariance to scene variations for precise weather classification. Based on this accurate classification, IWR selects the top-K most relevant Weather-Specific Experts (WSE) that handle data discrepancies, ensuring optimal detection under all weather conditions. Additionally, we propose a Unified Dual-Modal Augmentation (UDMA) for synchronous LiDAR and 4D Radar dual-modal data augmentation while preserving the realism of scenes. Extensive experiments on the real-world dataset demonstrate that AW-MoE achieves ~ 15% improvement in adverse-weather performance over state-of-the-art methods, while incurring negligible inference overhead. Moreover, integrating AW-MoE into established baseline detectors yields performance improvements surpassing current state-of-the-art methods. These results show the effectiveness and strong scalability of our AW-MoE. We will release the code publicly at https://github.com/windlinsherlock/AW-MoE.

Figures & tables

Fig. 1: Comparison of weather-type discriminability between camera images and LiDAR point clouds. (a, b) Camera images exhibit distinct visual characteristics and robustness to scene variations, facilitating accurate weather classification. (c, d) In contrast, LiDAR point clouds suffer from ambiguous inter-class geometric distortions and scene-induced intra-class distribution shifts, which obscure the boundaries between different weather conditions.

Fig. 2: (a) Performance changes of L4DR [ 12 ] after fine-tuning on a single weather condition under different weather scenarios. (b) Statistics of data volume across different weather conditions in the K-Radar dataset [ 11 ] .

Fig. 3: Method comparison between Point-cloud Feature-based Routing (PFR) and the proposed Image-guided Weather-aware Routing (IWR).

Fig. 4: AW-MoE Framework. (a) Unified Dual-Modal Augmentation (UDMA): Synchronously augments LiDAR and 4D Radar point clouds. Its GT Sampling only selects ground truths matching the scene’s weather. (b) Image-guided Weather-aware Routing (IWR): Uses an Image-based Weather Classifier to predict the scene weather and routes the feature to the top-K most relevant Weather-Specific Experts. (c) Weather-Specific Experts (WSE): Each expert is specialized for a weather condition, extracting robust weather-specific features and regressing bounding boxes with tailored sensitivity.

Fig. 5: The architecture of the AW-MoE-LRC framework. The pipeline comprises three stages: (i) LiDAR-Guided Image Feature Lifting, where sparse LiDAR depth assists in predicting 3D frustum features from images; (ii) 3D Geometry Transformation and BEV Pooling, which projects and aggregates these features into the ego-vehicle BEV space; and (iii) Multi-Modal Feature Fusion, which concatenates the aligned camera, LiDAR, and 4D Radar BEV features along the channel dimension for final convolution-based integration.

Fig. 6: Architecture of the proposed Image-based Weather Classifier.

Method	Total	Normal	Overcast	Fog	Rain	Sleet	Light Snow	Heavy Snow
PFR	71.3	98.9	12.0	51.9	76.7	57.6	2.1	58.3
IWR	99.0	99.8	100.0	97.3	99.0	97.1	99.3	98.7

TABLE I: Comparison of weather classification accuracy between Point-cloud Feature-based Routing (PFR) and the proposed Image-guided Weather-aware Routing (IWR).

Method	Modality	IoU	Metric	Total	Normal	Overcast	Fog	Rain	Sleet	Light Snow	Heavy Snow
RTNH [ 11 ] (NeurIPS 2022)	4DR	0.3	$AP_{BEV}$	41.1	41.0	44.6	45.4	32.9	50.6	81.5	56.3
		0.3	$AP_{3D}$	37.4	37.6	42.0	41.2	29.2	49.1	63.9	43.1
		0.5	$AP_{BEV}$	36.0	35.8	41.9	44.8	30.2	34.5	63.9	55.1
		0.5	$AP_{3D}$	14.1	19.7	20.5	15.9	13.0	13.5	21.0	6.36
RTNH [ 11 ] (NeurIPS 2022)	L	0.3	$AP_{BEV}$	76.5	76.5	88.2	86.3	77.3	55.3	81.1	59.5
		0.3	$AP_{3D}$	72.7	73.1	76.5	84.8	64.5	53.4	80.3	52.9

TABLE II: Quantitative results of different 3D object detection methods on K-Radar dataset. We present the modality of each method (L: LiDAR, 4DR: 4D Radar) and detailed performance for each weather condition. Best in bold , second in underline , and ∗ indicates results reproduced using open code.

Method	Modality	IoU	Total	Normal	Overcast	Fog	Rain	Sleet	Light Snow	Heavy Snow	FPS [HZ]
AW-MoE	L+4DR	0.3	83.9	84.2	90.0	95.3	84.4	72.9	90.2	64.0	12.41
AW-MoE	L+4DR	0.5	61.5	59.0	67.2	85.7	63.5	43.3	70.1	53.1	12.41
AW-MoE-LRC	L+4DR+C	0.3	84.3	84.7	91.0	95.3	84.0	72.9	89.6	63.7	10.02
AW-MoE-LRC	L+4DR+C	0.5	61.8	60.2	70.4	85.8	63.4	43.7	69.9	52.8	10.02

TABLE III: Performance (

AP_{3D}

) of AW-MoE and its camera-integrated variant, AW-MoE-LRC.

Method	IoU	Total	Normal	Overcast	Fog	Rain	Sleet	Light Snow	Heavy Snow
RTNH (4DR) [ 11 ]	0.3	37.4	37.6	42.0	41.2	29.2	49.1	63.9	43.1
RTNH (4DR) [ 11 ]	0.5	14.1	19.7	20.5	15.9	13.0	13.5	21.0	6.4
RTNH (4DR) - AW-MoE	0.3	65.7	64.4	72.2	88.7	58.3	64.0	71.2	65.0
RTNH (4DR) - AW-MoE	0.5	35.7	28.6	44.3	72.9	34.9	32.4	42.7	45.3
	0.3	+28.3	+26.8	+30.2	+47.5	+29.1	+14.9	+7.3	+21.9
Improvement	0.5	+21.6	+8.9	+23.8	+57.0	+21.9	+18.9	+21.7	+38.9

TABLE IV: Performance (

AP_{3D}

) comparison of AW-MoE when extended to different 3D object detection baselines.

Method	Sensors	FPS [HZ]	FLOPS [GB]	Param [M]
L4DR [ 12 ]	L+4DR	13.94	142.65	59.73
L4DR - AW-MoE	L+4DR	12.41	143.41	143.46
RTNH (4DR) [ 11 ]	4DR	14.77	502.57	17.35
RTNH (4DR) - AW-MoE	4DR	14.50	503.25	61.41
RTNH (L) [ 11 ]	L	14.62	502.53	17.35
RTNH (L) - AW-MoE	L	14.20	503.22	61.41

TABLE V: FPS, and FLOPS comparison of detectors before and after applying AW-MoE (corresponding to Table IV ).

Method	IoU	Metric	Total	Normal	Overcast	Fog	Rain	Sleet	Light Snow	Heavy Snow	Non-normal
WAGTS	0.3	$AP_{BEV}$	87.4	87.2	94.7	96.3	87.9	78.8	94.7	71.7	86.9
	0.3	$AP_{3D}$	82.5	83.6	87.3	93.1	83.6	69.7	89.2	65.5	81.6
	0.5	$AP_{BEV}$	83.2	82.1	90.3	96.1	84.4	70.4	93.8	67.6	83.4
	0.5	$AP_{3D}$	59.3	58.1	60.6	77.7	63.6	37.4	63.6	50.2	60.1
WSGTS	0.3	$AP_{BEV}$	88.2	87.7	94.4	96.5	88.3	79.5	95.4	72.8	87.9
	0.3	$AP_{3D}$	83.6	84.1	89.9	93.4	84.3	70.2	89.0	66.0	82.4

TABLE VI: Performance comparison between Weather-Specific GT Sampling (WSGTS) and Weather-Agnostic GT Sampling (WAGTS). Non-normal denotes the aggregate of all non-normal weather conditions.

Fig. 7: Comparison of our AW-MoE, L4DR [ 12 ] , and InterFusion [ 50 ] visualization results under Normal, Overcast, Rainy and Sleet weather conditions.

Fig. 8: Comparison of our AW-MoE, L4DR [ 12 ] , and InterFusion [ 50 ] visualization results under Fog, Light Snow and Heavy Snow weather conditions.

Routing	IoU	Metric	Total	Normal	Overcast	Fog	Rain	Sleet	Light Snow	Heavy Snow
PFR	0.3	$AP_{BEV}$	58.7	87.3	94.3	23.1	77.7	34.7	70.2	70.9
	0.3	$AP_{3D}$	52.8	84.0	89.3	11.4	73.0	29.1	64.8	63.6
	0.5	$AP_{BEV}$	55.6	82.1	91.4	22.9	74.4	31.4	69.6	66.9
	0.5	$AP_{3D}$	35.4	57.8	66.5	6.2	52.4	16.6	46.6	49.9
IWR	0.3	$AP_{BEV}$	88.2	87.7	94.5	96.7	88.8	81.0	95.4	70.2
	0.3	$AP_{3D}$	83.9	84.2	90.0	95.3	84.4	72.9	90.2	64.0

TABLE VII: Performance comparison of AW-MoE using different routing strategies: Point-cloud Feature-based Routing (PFR) and Image-guided Weather-aware Routing (IWR).

Training Strategy	IoU	Total	Normal	Overcast	Fog	Rain	Sleet	Light Snow	Heavy Snow	Non-normal
Direct Training	0.3	74.9	80.2	84.1	86.5	75.9	53.2	75.8	53.8	72.8
Direct Training	0.5	54.2	54.5	63.3	64.3	53.5	38.8	58.8	41.1	52.3
AW-MoE Training Strategy	0.3	83.9	84.2	90.0	95.3	84.4	72.9	90.2	64.0	83.9
AW-MoE Training Strategy	0.5	61.5	59.0	67.2	85.7	63.5	43.3	70.1	53.1	61.5

TABLE VIII: Performance (

AP_{3D}

) comparison between AW-MoE training strategy and direct end-to-end training. Non-normal denotes the aggregate of all non-normal weather conditions.

Parameter K	IoU	Total	Normal	Overcast	Fog	Rain	Sleet	Light Snow	Heavy Snow	Non-normal
K = 1	0.3	83.9	84.2	90.0	95.3	84.4	72.9	90.2	64.0	83.9
K = 1	0.5	61.5	59.0	67.2	85.7	63.5	43.3	70.1	53.1	61.5
K = 2	0.3	84.2	84.9	90.0	95.0	84.8	72.8	88.4	64.8	82.8
K = 2	0.5	61.1	59.3	67.2	81.5	63.7	44.2	67.8	52.7	62.5
K = 3	0.3	83.2	84.1	89.0	94.1	84.7	69.9	88.2	64.4	82.2
K = 3	0.5	61.0	59.3	69.0	79.7	63.5	42.7	70.2	53.1	62.3

TABLE IX: Effects of different top-K values in Image-guided Weather-aware Routing (IWR). Non-normal denotes the aggregate of all non-normal weather conditions.

Parameter K	Metric	Total (0.3 / 0.5)	Non-normal (0.3 / 0.5)
K = 1	$AP_{BEV}$	82.8 / 79.1	86.0 / 82.0
K = 1	$AP_{3D}$	77.0 / 53.3	80.1 / 56.0
K = 2	$AP_{BEV}$	84.3 / 79.9	87.8 / 82.7
K = 2	$AP_{3D}$	77.3 / 55.8	80.5 / 58.3

TABLE X: Robustness analysis of IWR under ambiguous weather conditions with varying Top-K values. (0.3 / 0.5) indicates the IoU value.

Explore similar work

Open-Weather Robust 3D Detection via Dual-Critic Diffusion Alignment
Jul 2, 2026Shuyao Li, Chuanxing Geng, Heyang Sun +2Lidar-Based 3D Object Detection Domain Generalization
Nanjing University of Aeronautics and Astronautics, Nanjing, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China
Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection
Apr 7, 2026Hongsheng Li, Lingfeng Zhang, Zexian Yang +4Object Detection Multimodal Sensor Fusion
Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong, China · College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China · Institute of Information Engineering, CAS, Beijing, China +1
UniDA3D: A Unified Domain-Adaptive Framework for Multi-View 3D Object Detection
Mar 30, 2026Hongjing Wu, Cheng Chi, Jinlin Wu +3Multi-View 3D Object Detection Autonomous Driving Perception
the Shenzhen Campus of Sun Yat-sen University, Shenzhen, China · The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · the Beijing Academy of Artificial Intelligence, Beijing, China +2

Jul 2, 2026J/K move · Enter open · S save

Open-Weather Robust 3D Detection via Dual-Critic Diffusion Alignment

Shuyao Li, Chuanxing Geng, Heyang Sun, Qiang Zhou+1

Nanjing University of Aeronautics and Astronautics, Nanjing, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China

Lidar-Based 3D Object Detection Domain Generalization 3D Object Detection

Open paper AskPDF arXiv

Robust 3D object detection under adverse weather remains a critical hurdle for autonomous driving. Despite progress with LiDAR-4D radar fusion, most methods are constrained by a closed-world assumption, implicitly requiring training and test weather to align in both type and severity. This premise fails in practice: the open-ended nature of weather, and even variations within a single type like rain, cause dramatically different LiDAR degradation patterns, leading to significant performance drops in unseen conditions. To address this, we present Dual-Critic Guided Diffusion Alignment (DCDA), a weather-agnostic framework that learns to recover degraded LiDAR features toward a clean manifold. Rather than modeling specific weather types, DCDA employs a 4D radar-conditioned diffusion process to progressively refine features, guided by two complementary critics. (i) A detection-guided critic, anchored by a pre-trained clean-weather model, ensures that the refined features retain object-level discriminability and localization accuracy. (ii) A weather adversarial critic enforces holistic distributional consistency with clean-weather representations. By aligning features through semantic and distributional constraints rather than explicit weather modeling, DCDA generalizes effectively to unseen weather types and severities without requiring paired data or weather labels. We further introduce a structured open-weather benchmark with held-out type-severity combinations and extensive experiments verify DCDA's advantages.

Abstract

Explore similar work

Open-Weather Robust 3D Detection via Dual-Critic Diffusion Alignment

Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection

UniDA3D: A Unified Domain-Adaptive Framework for Multi-View 3D Object Detection