CoIn3D: Revisiting Configuration-Invariant Multi-Camera 3D Object Detection

Zhaonian Kuang; Rui Ding; Haotian Wang; Xinhu Zheng; Meng Yang; Gang Hua

cs.CVMar 5, 2026

CoIn3D: Revisiting Configuration-Invariant Multi-Camera 3D Object Detection

Authors: Zhaonian Kuang, Rui Ding, Haotian Wang, Xinhu Zheng, Meng Yang, Gang Hua

Organizations: National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University · Intelligent Transportation Thrust of the Systems Hub, HKUST(GZ) · Amazon Alexa AI

Abstract

Multi-camera 3D object detection (MC3D) has attracted increasing attention with the growing deployment of multi-sensor physical agents, such as robots and autonomous vehicles. However, MC3D models still struggle to generalize to unseen platforms with new multi-camera configurations. Current solutions simply employ a meta-camera for unified representation but lack comprehensive consideration. In this paper, we revisit this issue and identify that the devil lies in spatial prior discrepancies across source and target configurations, including different intrinsics, extrinsics, and array layouts. To address this, we propose CoIn3D, a generalizable MC3D framework that enables strong transferability from source configurations to unseen target ones. CoIn3D explicitly incorporates all identified spatial priors into both feature embedding and image observation through spatial-aware feature modulation (SFM) and camera-aware data augmentation (CDA), respectively. SFM enriches feature space by integrating four spatial representations, such as focal length, ground depth, ground gradient, and Plücker coordinate. CDA improves observation diversity under various configurations via a training-free dynamic novel-view image synthesis scheme. Extensive experiments demonstrate that CoIn3D achieves strong cross-configuration performance on landmark datasets such as NuScenes, Waymo, and Lyft, under three dominant MC3D paradigms represented by BEVDepth, BEVFormer, and PETR.

Figures & tables

Figure 1: CoIn3D effectively enables model transferability from source configuration A to unseen target configurations B, C, …, covering variations in intrinsics, extrinsics, and array layouts. Our framework can be applied to three dominant MC3D paradigms, represented by BEVDepth [ 22 ] , BEVFormer [ 23 ] , and PETR [ 25 ] .

Figure 2: Illustration of our CoIn3D framework for generalizable MC3D across multi-camera configurations. During training, we apply the camera-aware data augmentation (CDA) to generate $N$ images with randomly sampled camera configurations, followed by spatial-aware feature modulation (SFM). SFM modulates activations using an inverse focal map to obtain focal-invariant features, then projects prior maps (ground depth, gradient map, and Plücker raymap) to create spatial embeddings, which are added to the focal-invariant features. These maps are concatenated with image input and features to provide raw priors. Finally, the spatial-aware features can be easily integrated into MC3D for downstream tasks. During inference, we use raw images and apply spatial-aware modulation to generalize to new camera configurations. Our framework is applicable to dominant MC3D paradigms, including bottom-up BEV, top-down BEV, and sparse-queries.

Figure 3: Illustration of the spatial discrepancies under different camera configurations: (a) focal-ambiguity for a same object; (b) ground depth and depth increasing rate under different camera heights; (c) the scene structure (1st row), depth distribution (2nd row), and Plücker raymap (3rd row) for surround-view cameras.

Figure 4: Illustration of the training-free ego-centric Gaussians construction pipeline. We transform reconstructed texture point clouds into Gaussian representations using predefined parameters.

Setting	Nuscenes $\rightarrow$ Waymo					Waymo $\rightarrow$ Nuscenes
Method	mAP $\uparrow$	mATE $\downarrow$	mASE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$	mAP $\uparrow$	mATE $\downarrow$	mASE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$
Oracle	0.552	0.528	0.148	0.085	0.649	0.475	0.577	0.177	0.147	0.587
Direct Transfer	0.040	1.303	0.265	0.790	0.178	0.032	1.305	0.768	0.532	0.133
CAM-Convs [ 8 ]	0.045	1.301	0.253	0.773	0.185	0.038	1.308	0.316	0.506	0.215
Single-DGOD [ 44 ]	0.034	1.305	0.262	0.855	0.164	0.014	1.000	1.000	1.000	0.007
DG-BEV [ 40 ]	0.297	0.822	0.216	0.372	0.415	0.303	0.689	0.218	0.171	0.472

Table 1: Comparison of our scheme with existing methods based on BEVDepth. “Nuscenes

\rightarrow

Waymo” refers to train on Nuscenes and test on Waymo with different configurations, and similarly for the other datasets. “Direct Transfer” refers to directly use BEVDepth to test on target dataset without any adaptation. “Oracle” refers to training and testing on target dataset, serving as an upper bound for performance.

Setting	Method	BEVFormer					PETR
Setting	Method	mAP $\uparrow$	mATE $\downarrow$	mASE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$	mAP $\uparrow$	mATE $\downarrow$	mASE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$
N $\rightarrow$ L	DT	0.149	1.031	0.755	1.241	0.115	0.013	1.240	0.761	1.484	0.046
	PD-BEV	0.208	/	/	/	0.355	0.032	/	/	/	0.091
	Ours	0.237	0.850	0.191	0.407	0.377	0.332	0.880	0.170	0.210	0.456
N $\rightarrow$ W	DT	0.054	1.146	0.747	1.606	0.069	0.048	1.191	0.748	1.543	0.066
N $\rightarrow$ W	Ours	0.249	0.872	0.239	0.365	0.379	0.200	1.141	0.242	0.505	0.309

Table 2: Generalization experiment of our scheme based on BEVFormer and PETR. Our framework can apply to both top-down BEV and sparse queries schemes, represented by BEVFormer and PETR respectively. Notably, most previous methods do not support these two paradigms. N, L, W, and DT represent Nuscenes, Lyft, Waymo, and Direct Transfer, respectively.

Module			mAP $\uparrow$	mATE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$
CDA	SFM	CA	mAP $\uparrow$	mATE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$
		✓	0.040	1.303	0.790	0.178
✓		✓	0.190	1.021	0.255	0.347
✓	✓	✓	0.381	0.706	0.168	0.504
	✓		0.215	0.887	0.312	0.358
✓			0.041	1.310	0.513	0.224

Table 3: Main ablation of our two modules CDA and SFM. CA refers the Camera-Aware SE module designed by BEVDepth.

Spatial Priors				mAP $\uparrow$	mATE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$
IF	GD	GG	PR	mAP $\uparrow$	mATE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$
				0.041	1.310	0.513	0.224
✓				0.332	0.811	0.176	0.462
✓	✓			0.365	0.696	0.182	0.498
✓	✓	✓		0.375	0.684	0.177	0.506
✓	✓	✓	✓	0.381	0.687	0.155	0.513

Table 4: Ablation on our SFM based on CDA. IF, GD, GG, and PR represent the inverse focal map, ground depth map, ground gradient map, and Plücker raymap, respectively.

F-Aug	NVS-Aug	mAP $\uparrow$	mATE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$
Augmentation		mAP $\uparrow$	mATE $\downarrow$	mAOE $\downarrow$	NDS* $\uparrow$
		0.215	0.887	0.312	0.358
✓		0.314	0.990	0.192	0.418
✓	✓	0.381	0.687	0.155	0.513

Table 5: Ablation on our CDA based on SFM. F-Aug and NVS-Aug represent focal augmentation and novel view synthesis augmentation, respectively.

Figure 5: The detailed ego-centric Gaussian construction pipeline.

Datasets	Nuscenes	Lyft(fleet 1)	Lyft(fleet 2)	Waymo
image resolution $(w,h)$	(1600,900)	(1224,1024)	(1920,1080)	(1920,1280)(F,FL,FR)
				(1920,886)(SL,SR)
focal length	1250(F,FL,FR,BL,BR)	880	1100	2050
	800(B)
horizontal FoV (°)	65(F,FL,FR,BL,BR)	70	80	50
	90(B)

Table 6: The details of camera configurations of Nuscenes, Lyft, and Waymo. Camera flags F, FL, FR, B, BL, BR, SL, SR represent front, front left, front right, back, back left, back right, side left, and side right, respectively. Nuscenes and Lyft possess F, FL, FR, B, BL, and BR. Waymo possesses F, FL, FR, SL, and SR. Consider that camera installed pitch (

t_{y}

) and roll (

t_{x}

) angle is generally set to 0, we only show the yaw angle (

t_{z}

Setting	Method	All		Car		Pedestrian		Two-wheel
Setting	Method	mAP	NDS*	mAP	NDS*	mAP	NDS*	mAP	NDS*
Mix $\rightarrow$ Nuscenes	Oracle	0.255	0.335	0.385	0.508	0.205	0.247	0.175	0.249
	Direct Mix	0.267	0.350	0.411	0.537	0.213	0.258	0.176	0.255
	ScaleBEV	0.293	0.358	0.447	0.540	0.246	0.275	0.185	0.258
	CoIn3D	0.358	0.441	0.542	0.693	0.321	0.360	0.211	0.270
Mix $\rightarrow$ Lyft	Oracle	0.290	0.390	0.408	0.530	0.114	0.199	0.349	0.440

Table 7: Results of Nuscenes+Waymo+Lyft joint training (Mix), reported on 3 classes(car, pedestrian, two-wheel object). ScaleBEV [ 27 ] are reported on BEVDet, others are reported on BEVDepth. Oracle means training on the single source dataset.

Setting	mAP	NDS*
CDSA N $\rightarrow$ L	0.224	0.372
UniDrive N $\rightarrow$ L	0.264	0.440
CoIn3D N $\rightarrow$ L	0.375	0.534
CDSA N $\rightarrow$ W	0.116	0.259
UniDrive N $\rightarrow$ W	0.144	0.298
CoIn3D N $\rightarrow$ W	0.384	0.513

Table 8: Comparison of CoIn3D and two input-adjustment methods: Cross-dataset sensor alignment (CDSA) [ 51 ] and Unidrive [ 20 ] .

Setting	mAP	NDS*
Oracle	0.436	0.554
DT	0.126	0.342
CoIn3D	0.401	0.524

Table 9: Results of Nuscenes-val re-rendered in Waymo Cfg.

Setting	mAP	NDS*
N $\rightarrow$ W ${}^{{}^{\prime}}$	0.516	0.602
W $\rightarrow$ W ${}^{{}^{\prime}}$	0.638	0.702

Table 10: Results of sequences with significant road slope in W-val (W

{}^{{}^{\prime}}

). Results are reported on BEVDepth with CoIn3D.

Dataset	Method	mAP $\uparrow$	NDS ∗ $\uparrow$
Nuscenes	BEVDepth [ 22 ]	0.475	0.587
	UDGA-BEV [ 3 ]	0.497	0.603
	Ours	0.529	0.630
Lyft	BEVDepth	0.602	0.684
	UDGA-BEV	0.630	0.702
	Ours	0.624	0.699

Table 11: Source dataset performance comparison.

Method	mAP $\uparrow$	NDS ∗ $\uparrow$
FSDepth [ 42 ]	0.223	0.374
SFM@(InvF)	0.260	0.404
SFM@(InvF,GD)	0.274	0.425
SFM@(InvF,PR)	0.272	0.429
Cat@(ALL)	0.247	0.391
SFM@(ALL)	0.320	0.463

Table 12: More ablation results regrading to priors.

Figure 6: Qualitatively visualization of novel view images of Nuscenes.

Figure 7: Qualitatively visualization of novel view images of Lyft.

Figure 8: Qualitatively visualization of novel view images of Waymo.

Figure 9: Qualitatively visualization of prediction results under 3d textured point cloud. Red boxes denote the ground truth boxes and green boxes denote the predicted boxes. The upper row shows results by directly using raw BEVDepth to inference on target configuration. The bottom row shows results by applying our CoIn3D.

Method	mAP $\uparrow$	NDS ∗ $\uparrow$
conv1	0.282	0.430
conv1+conv1	0.307	0.454
conv3+conv3	0.297	0.448
conv3	0.320	0.463

Table 13: More ablation results regrading to embedding projectors.

Multi-View 3D Object Detection

Multi-View 3D Object Detection combines images from multiple cameras to locate and classify objects in three-dimensional space.

See all 25 papers

Towards Intrinsic-Aware Monocular 3D Object Detection
Mar 28, 2026Zhihao Zhang, Abhinav Kumar, Xiaoming Liu3D Object Detection Monocular 3D Object Detection
Michigan State University · University of North Carolina at Chapel Hill
CAM3DNet: Comprehensively mining the multi-scale features for 3D Object Detection with Multi-View Cameras
Apr 18, 2026Mingxi Pang, Dingheng Wang, Zekun Li +4Multi-Scale Feature Fusion Multi-View 3D Object Detection
College of Intelligent Science and Technology, National University of Defense Technology, Changsha, 410073, China · Northwest Institute of Mechanical & Electrical Engineering, Xianyang, 712099, China. · School of Aerospace Engineering, Xi’an Jiaotong University, Xi’an, 710049, China
CoGoal3D: Collaborative 3D Object Detection with 3D-Aware Fusion and Refinement
Jul 21, 2026Zhihao Yang, Zhiyu Xiang, Peng Xu +5Point Cloud Surface Reconstruction 3D Object Detection
Zhejiang University, Hangzhou, China · Zhejiang Provincial Key Laboratory of Multi-Modal Communication Networks and Intelligent Information Processing, Hangzhou, China · China Jiliang University, Hangzhou, China

Mar 28, 2026J/K move · Enter open · S save

Towards Intrinsic-Aware Monocular 3D Object Detection

Zhihao Zhang, Abhinav Kumar, Xiaoming Liu

Michigan State University · University of North Carolina at Chapel Hill

3D Object Detection Monocular 3D Object Detection Domain Generalization Representation Learning

Open paper AskPDF arXiv

Monocular 3D object detection (Mono3D) aims to infer object locations and dimensions in 3D space from a single RGB image. Despite recent progress, existing methods remain highly sensitive to camera intrinsics and struggle to generalize across diverse settings, since intrinsics govern how 3D scenes are projected onto the image plane. We propose MonoIA, a unified intrinsic-aware framework that models and adapts to intrinsic variation through a language-grounded representation. The key insight is that intrinsic variation is not a numeric difference but a perceptual transformation that alters apparent scale, perspective, and spatial geometry. To capture this effect, MonoIA employs large language models and vision-language models to generate intrinsic embeddings that encode the visual and geometric implications of camera parameters. These embeddings are hierarchically integrated into the detection network via an Intrinsic Adaptation Module, allowing the model to modulate its feature representations according to camera-specific configurations and maintain consistent 3D detection across intrinsics. This shifts intrinsic modeling from numeric conditioning to semantic representation, enabling robust and unified perception across cameras. Extensive experiments show that MonoIA achieves new state-of-the-art results on standard benchmarks including KITTI, Waymo, and nuScenes (e.g., +1.18% on the KITTI leaderboard), and further improves performance under multi-dataset training (e.g., +4.46% on KITTI Val).

Abstract

Multi-View 3D Object Detection

Explore similar work

Towards Intrinsic-Aware Monocular 3D Object Detection

CAM3DNet: Comprehensively mining the multi-scale features for 3D Object Detection with Multi-View Cameras

CoGoal3D: Collaborative 3D Object Detection with 3D-Aware Fusion and Refinement