UniDA3D: A Unified Domain-Adaptive Framework for Multi-View 3D Object Detection

Hongjing Wu; Cheng Chi; Jinlin Wu; Yanzhao Su; Zhen Lei; Wenqi Ren

cs.CVMar 30, 2026

UniDA3D: A Unified Domain-Adaptive Framework for Multi-View 3D Object Detection

Authors: Hongjing Wu, Cheng Chi, Jinlin Wu, Yanzhao Su, Zhen Lei, Wenqi Ren

Organizations: the Shenzhen Campus of Sun Yat-sen University, Shenzhen, China · The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · the Beijing Academy of Artificial Intelligence, Beijing, China · the National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China · the Xi’an Research Institute of High-tech, Xi’an, China

Abstract

Camera-only 3D object detection is critical for autonomous driving, offering a cost-effective alternative to LiDAR based methods. In particular, multi-view 3D object detection has emerged as a promising direction due to its balanced trade-off between performance and cost. However, existing methods often suffer significant performance degradation under complex environmental conditions such as nighttime, fog, and rain, primarily due to their reliance on training data collected mostly in ideal conditions. To address this challenge, we propose UniDA3D, a unified domain-adaptive multi-view 3D object detector designed for robust perception under diverse adverse conditions. UniDA3D formulates nighttime, rainy, and foggy scenes as a unified multi target domain adaptation problem and leverages a novel query guided domain discrepancy mitigation (QDDM) module to align object features between source and target domains at both batch and global levels via query-centric adversarial and contrastive learning. Furthermore, we introduce a domain-adaptive teacher student training pipeline with an exponential-moving-average teacher and dynamically updated high-quality pseudo labels to enhance consistency learning and suppress background noise in unlabeled target domains. In contrast to prior approaches that require separate training for each condition, UniDA3D performs a single unified training process across multiple domains, enabling robust all-weather 3D perception. On a synthesized multi-view 3D benchmark constructed by generating nighttime, rainy, and foggy counterparts from nuScenes (nuScenes-Night, nuScenes-Rain, and nuScenes-Haze), UniDA3D consistently outperforms state of-the-art camera-only multi-view 3D detectors under extreme conditions, achieving substantial gains in mAP and NDS while maintaining real-time inference efficiency.

Figures & tables

Fig. 1: Overview of the proposed UniDA3D. UniDA3D first synthesizes nighttime, rainy, and foggy multi-view images to assist training. A domain-adaptive teacher–student self-training pipeline is employed to transfer knowledge from the source domain to the target domains. UniDA3D is further equipped with a query-guided domain discrepancy mitigation (QDDM) module to align object-level features across domains at both batch and global scales.

Fig. 2: Details of the query-guided domain discrepancy mitigation (QDDM) module. QDDM leverages 3D object queries to align source and target features at the object level. It combines adversarial training and contrastive learning with dynamically updated global representation, promoting robust and consistent cross-domain 3D object detection.

Methods	Backbone	Image Size	mAP $\uparrow$	NDS $\uparrow$	mATE $\downarrow$	mASE $\downarrow$	mAOE $\downarrow$	mAVE $\downarrow$	mAAE $\downarrow$
BEVDet (CBGS) [ 40 ]	ResNet50	256 $\times$ 704	6.98	14.82	91.65	50.68	84.18	136.67	60.21
PETR [ 42 ]	ResNet50	512 $\times$ 1408	11.97	15.51	92.69	74.40	147.76	136.79	37.72
StreamPETR [ 1 ]	ResNet50	256 $\times$ 704	15.60	32.53	82.84	30.91	64.91	49.59	24.48
SOLOFusion [ 10 ]	ResNet50	256 $\times$ 704	17.84	32.91	77.64	30.78	78.28	47.02	26.41
BEVDet4D [ 9 ]	ResNet50	256 $\times$ 704	22.33	37.27	76.82	28.64	61.64	49.51	22.35
BEVFormer v2 [ 45 ]	ResNet50	640 $\times$ 1600	25.56	35.37	80.78	29.39	53.37	89.09	21.50

TABLE I: Comparison with state-of-the-art methods on the nuScenes-Night val set. Models are grouped by backbone (ResNet and V2-99) for a fair comparative analysis. Bold indicates the best result.

Table 4

Fig. 3: t-SNE visualization results (Blue point: Source Domain, Red point: Target Domain). The distribution consistency between red and blue points reflects the discrepancy in features extracted by the model. A smaller gap between the distributions indicates stronger domain adaptation capability.

EMA Ratio	mAP $\uparrow$	NDS $\uparrow$	mATE $\downarrow$
0.9	40.83	51.10	67.08
0.99	40.90	51.15	67.14
0.999	33.44	45.12	65.31

TABLE IV: Effect of EMA ratio. All experiments are conducted on nuScenes-Night val set.

Filtering threshold	mAP $\uparrow$	NDS $\uparrow$	mATE $\downarrow$
0.3	40.09	50.54	69.83
0.6	40.41	51.37	66.60
0.9	40.90	51.15	67.14

TABLE V: Effect of filtering threshold. The filtering threshold refers to the confidence threshold used to retain high-quality pseudo labels generated by the teacher model. All experiments are conducted on nuScenes-Night val set.

Self-training	QDDM	mAP $\uparrow$	NDS $\uparrow$	mATE $\downarrow$
		26.78	41.58	70.93
✓		40.90	51.15	67.14
	✓	41.88	51.72	65.91
✓	✓	45.04	53.76	65.87

TABLE VI: Effectiveness of Self-training and QDDM modules. All experiments are conducted on nuScenes-Night val set.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Backbone	FPS
StreamPETR	ResNet-50	28.0
StreamPETR	V2-99	16.7
UniDA3D (ours)	ResNet-50	28.5
UniDA3D (ours)	V2-99	17.9

Appendix

TABLE I: Inference speed (FPS) comparison on RTX4090.

Training Method	mAP	NDS	Condition
Hybrid	45.04	53.76	Night
	45.32	54.71	Rain
	46.85	55.90	Haze
Night	44.93	54.44	Night
Rain	45.26	54.83	Rain
Haze	46.77	55.61	Haze

Appendix

TABLE II: Comparison of training strategies under different weather conditions.

TABLE III: Ablation on domain adversarial (

\lambda_{\text{dom}}

) and contrastive (

\lambda_{\text{con}}

) loss weights.

Explore similar work

AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection
Mar 17, 2026Hongwei Lin, Xun Huang, Chenglu Wen +13D Object Detection Autonomous Driving
Fujian Key Laboratory of Urban Intelligent Sensing and Computing and the School of Informatics, Xiamen University, Xiamen, FJ 361005, China · Zhongguancun Academy, Beijing, China
Open-Weather Robust 3D Detection via Dual-Critic Diffusion Alignment
Jul 2, 2026Shuyao Li, Chuanxing Geng, Heyang Sun +2Lidar-Based 3D Object Detection Domain Generalization
Nanjing University of Aeronautics and Astronautics, Nanjing, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China
MUSDA: Multi-source Multi-modality Unsupervised Domain Adaptive 3D Object Detection for Autonomous Driving
May 11, 2026Xiaohu Lu, Hamed Khatounabadi, Hayder RadhaMulti-Source Domain Adaptation Unsupervised Domain Adaptation
Electrical and Computer Engineering Michigan State University East Lansing, USA

Mar 17, 2026J/K move · Enter open · S save

AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection

Hongwei Lin, Xun Huang, Chenglu Wen, Cheng Wang

Fujian Key Laboratory of Urban Intelligent Sensing and Computing and the School of Informatics, Xiamen University, Xiamen, FJ 361005, China · Zhongguancun Academy, Beijing, China

3D Object Detection Autonomous Driving Mixture of Experts Multimodal Sensor Fusion

Open paper AskPDF arXiv

Robust 3D object detection under adverse weather conditions is crucial for autonomous driving. However, most existing methods simply combine all weather samples for training while overlooking data distribution discrepancies across different weather scenarios, leading to performance conflicts. To address this issue, we introduce AW-MoE, the framework that innovatively integrates Mixture of Experts (MoE) into weather-robust multi-modal 3D object detection approaches. AW-MoE incorporates Image-guided Weather-aware Routing (IWR), which leverages the superior discriminability of image features across weather conditions and their invariance to scene variations for precise weather classification. Based on this accurate classification, IWR selects the top-K most relevant Weather-Specific Experts (WSE) that handle data discrepancies, ensuring optimal detection under all weather conditions. Additionally, we propose a Unified Dual-Modal Augmentation (UDMA) for synchronous LiDAR and 4D Radar dual-modal data augmentation while preserving the realism of scenes. Extensive experiments on the real-world dataset demonstrate that AW-MoE achieves ~ 15% improvement in adverse-weather performance over state-of-the-art methods, while incurring negligible inference overhead. Moreover, integrating AW-MoE into established baseline detectors yields performance improvements surpassing current state-of-the-art methods. These results show the effectiveness and strong scalability of our AW-MoE. We will release the code publicly at https://github.com/windlinsherlock/AW-MoE.

Abstract

Explore similar work

AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection

Open-Weather Robust 3D Detection via Dual-Critic Diffusion Alignment

MUSDA: Multi-source Multi-modality Unsupervised Domain Adaptive 3D Object Detection for Autonomous Driving