Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection

Hongsheng Li; Lingfeng Zhang; Zexian Yang; Liang Li; Rong Yin; Xiaoshuai Hao; Wenbo Ding

cs.CVApr 7, 2026

Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection

Authors: Hongsheng Li, Lingfeng Zhang, Zexian Yang, Liang Li, Rong Yin, Xiaoshuai Hao, Wenbo Ding

Organizations: Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong, China · College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China · Institute of Information Engineering, CAS, Beijing, China · Xiaomi EV, Beijing, China

Abstract

Robust 3D object detection in adverse weather is highly challenging due to the varying reliability of different sensors. While existing LiDAR-4D radar fusion methods improve robustness, they predominantly rely on fixed or weakly adaptive pipelines, failing to dy-namically adjust modality preferences as environmental conditions change. To bridge this gap, we reformulate multi-modal perception as a weather-conditioned branch routing problem. Instead of computing a single fused output, our framework explicitly maintains three parallel 3D feature streams: a pure LiDAR branch, a pure 4D radar branch, and a condition-gated fusion branch. Guided by a condition token extracted from visual and semantic prompts, a lightweight router dynamically predicts sample-specific weights to softly aggregate these representations. Furthermore, to prevent branch collapse, we introduce a weather-supervised learning strategy with auxiliary classification and diversity regularization to enforce distinct, condition-dependent routing behaviors. Extensive experiments on the K-Radar benchmark demonstrate that our method achieves state-of-the-art performance. Furthermore, it provides explicit and highly interpretable insights into modality preferences, transparently revealing how adaptive routing robustly shifts reliance between LiDAR and 4D radar across diverse adverse-weather scenarios. The source code with be released.

Figures & tables

Figure 1. Comparison between existing LiDAR-4D radar fusion methods and our proposed approach. Existing methods rely on a single, fixed fused representation, which struggles to adapt to changing environments. In contrast, our weather-conditioned branch routing explicitly maintains multiple representation branches and dynamically assigns adaptive weights based on the current weather condition.

Figure 2. Framework overview of the proposed weather-conditioned branch routing method. The model takes LiDAR, 4D radar, and camera images as inputs. First, a condition token is generated by fusing visual features and semantic text prompts, which is further refined by sensor context. Next, LiDAR and radar features are processed through three parallel streams: a LiDAR-only branch, a radar-only branch, and a condition-gated fusion branch. A branch router then utilizes the condition token to predict adaptive weights ( $w_{L},w_{F},w_{R}$ ). These weights softly aggregate the branch-wise BEV features for the final 3D detection head. During training, auxiliary weather classification and diversity losses are applied to ensure meaningful routing behavior.

Methods	Modality	IoU	Metric	Weather Condition
Methods	Modality	IoU	Metric	Total	Normal	Overcast	Fog	Rain	Sleet	Lightsnow	Heavysnow
RTNH ( Paek et al., 2022 )	4DR	0.3	$\textit{AP}_{\mathrm{BEV}}$	41.1	41.0	44.6	45.4	32.9	50.6	81.5	56.3
		0.3	$\textit{AP}_{\mathrm{3D}}$	37.4	37.6	42.0	41.2	29.2	49.1	63.9	43.1
		0.5	$\textit{AP}_{\mathrm{BEV}}$	36.0	35.8	41.9	44.8	30.2	34.5	63.9	55.1
		0.5	$\textit{AP}_{\mathrm{3D}}$	14.1	19.7	20.5	15.9	13.0	13.5	21.0	6.36
RTNH* ( Paek et al., 2022 )	L	0.3	$\textit{AP}_{\mathrm{BEV}}$	76.5	76.5	88.2	86.3	77.3	55.3	81.1	59.5

Table 1. Quantitative results of LiDAR and 4D radar-based 3D object detection methods on K-Radar dataset. We present the modality of each method (L: LiDAR, 4DR: 4D radar) and detailed performance for each weather condition. Best in bold with blue background ( best ), second in light blue background ( second). * denotes our reproduced results.

Method	Branch Routing	$L_{\mathrm{aux}}$	$L_{\mathrm{div}}$	AP ${}_{\mathrm{BEV}}^{0.5}$	AP ${}_{\mathrm{3D}}^{0.5}$	AP ${}_{\mathrm{BEV}}^{0.3}$	AP ${}_{\mathrm{3D}}^{0.3}$
w/o branch routing (force Fusion only)		✓	✓	54.8	22.0	78.6	68.8
w/o weather supervision ( $L_{\mathrm{aux}}$ , $L_{\mathrm{div}}$ )	✓			70.7	43.0	77.4	73.9
w/o $L_{\mathrm{div}}$	✓	✓		64.8	30.2	78.3	72.9
Full model	✓	✓	✓	71.5	43.5	81.2	76.8

Table 2. Ablation study of the proposed method on K-Radar. Branch routing denotes the weather-conditioned multi-branch aggregation,

L_{\mathrm{aux}}

is the auxiliary weather classification loss, and

L_{\mathrm{div}}

is the weather diversity regularization.

Method	All		Overcast		Fog		Sleet		Light Snow		Heavy Snow
Method	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]
Transformer router	48.17	27.31	30.83	7.40	63.74	48.17	40.77	27.66	65.72	43.69	53.30	35.33
Cross-attention router	56.8	33.9	41.2	18.6	71.5	53.8	44.8	28.7	71.4	49.6	54.9	28.1
MLP router (ours)	71.5	43.5	88.3	63.3	88.1	63.4	49.0	30.5	83.0	66.8	57.2	33.7

Table 3. Comparison of different routing designs on K-Radar. The MLP-based router (Ours) achieves the best overall performance and shows stronger robustness across adverse weather conditions.

Hyperparameter	Value	AP ${}_{\mathrm{3D}}^{0.5}$ (%)	AP ${}_{\mathrm{BEV}}^{0.5}$ (%)
$\lambda_{\text{aux}}$	0.00	27.5	56.3
	0.05	43.5	70.8
	0.10	43.5	71.5
	0.20	33.9	63.4
$\lambda_{\text{div}}$	0.00	30.2	64.8
	0.01	37.9	68.7

Table 4. Ablation study on key hyperparameters of the proposed method. We evaluate the impact of the auxiliary weather supervision weight (

\lambda_{\text{aux}}

), diversity regularization weight (

\lambda_{\text{div}}

), and entropy regularization weight (

\lambda_{\text{ent}}

). Best results are highlighted in bold.

Figure 3. Qualitative results and interpretable routing behavior under diverse weather conditions. From top to bottom: Normal, Rain, Overcast, Sleet and Heavy Snow scenarios. For each scene, we visualize the camera view with 3D bounding box predictions (left), the adaptive branch weights $w_{L},w_{F},w_{R}$ predicted by the weather-conditioned router (middle), and the corresponding radar and LiDAR BEV representations (right).

Explore similar work

RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weather
Jul 6, 2026Heejun Park, Jaeseok Jeong, Kuk-Jin YoonCamera-Radar Fusion Lidar-Based 3D Object Detection
Visual Intelligence Lab., KAIST
Open-Weather Robust 3D Detection via Dual-Critic Diffusion Alignment
Jul 2, 2026Shuyao Li, Chuanxing Geng, Heyang Sun +2Lidar-Based 3D Object Detection Domain Generalization
Nanjing University of Aeronautics and Astronautics, Nanjing, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China
AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection
Mar 17, 2026Hongwei Lin, Xun Huang, Chenglu Wen +13D Object Detection Autonomous Driving
Fujian Key Laboratory of Urban Intelligent Sensing and Computing and the School of Informatics, Xiamen University, Xiamen, FJ 361005, China · Zhongguancun Academy, Beijing, China

Jul 6, 2026J/K move · Enter open · S save

RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weather

Heejun Park, Jaeseok Jeong, Kuk-Jin Yoon

Visual Intelligence Lab., KAIST

Camera-Radar Fusion Lidar-Based 3D Object Detection Camera-LiDAR Fusion 3D Object Detection Multimodal Sensor Fusion

Open paper AskPDF arXiv

Robust 3D object detection in adverse weather conditions is challenging due to sensor limitations. Although combining complementary modalities such as LiDAR and 4D RADAR has shown promise, the sparsity of these sensors becomes apparent in adverse weather with reduced reflections, leading to objects with few or no point cloud returns. To address this limitation, camera sensors provide visual cues even when LiDAR and RADAR signals are weakened. However, cameras themselves are also vulnerable to adverse weather, where some regions become unreliable due to snow or rain occluding the camera lens. While some camera-fusion methods designed for adverse weather learn to weigh image regions via confidence maps, these maps receive no direct supervision and are learned solely through the detection loss. We introduce Reliability-Aware Fusion (RAF), which explicitly supervises per-pixel reliability estimation and provides a direct learning signal for identifying and suppressing unreliable visual cues. Our framework leverages pretrained LiDAR-RADAR networks, keeping their backbones frozen while only training the added camera branch, BEV fusion encoder, and detection head. Extensive experiments on the K-Radar and VoD datasets demonstrate that integrating RAF consistently improves detection accuracy over LiDAR-RADAR baselines, achieving up to +6.5

AP_{BEV}

and +7.4

AP_{3D}

gains. Code is available at https://github.com/parkie0517/RAF.

Abstract

Explore similar work

RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weather

Open-Weather Robust 3D Detection via Dual-Critic Diffusion Alignment

AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection