Towards Intrinsic-Aware Monocular 3D Object Detection

Zhihao Zhang; Abhinav Kumar; Xiaoming Liu

cs.CVMar 28, 2026

Towards Intrinsic-Aware Monocular 3D Object Detection

Authors: Zhihao Zhang, Abhinav Kumar, Xiaoming Liu

Organizations: Michigan State University · University of North Carolina at Chapel Hill

Abstract

Monocular 3D object detection (Mono3D) aims to infer object locations and dimensions in 3D space from a single RGB image. Despite recent progress, existing methods remain highly sensitive to camera intrinsics and struggle to generalize across diverse settings, since intrinsics govern how 3D scenes are projected onto the image plane. We propose MonoIA, a unified intrinsic-aware framework that models and adapts to intrinsic variation through a language-grounded representation. The key insight is that intrinsic variation is not a numeric difference but a perceptual transformation that alters apparent scale, perspective, and spatial geometry. To capture this effect, MonoIA employs large language models and vision-language models to generate intrinsic embeddings that encode the visual and geometric implications of camera parameters. These embeddings are hierarchically integrated into the detection network via an Intrinsic Adaptation Module, allowing the model to modulate its feature representations according to camera-specific configurations and maintain consistent 3D detection across intrinsics. This shifts intrinsic modeling from numeric conditioning to semantic representation, enabling robust and unified perception across cameras. Extensive experiments show that MonoIA achieves new state-of-the-art results on standard benchmarks including KITTI, Waymo, and nuScenes (e.g., +1.18% on the KITTI leaderboard), and further improves performance under multi-dataset training (e.g., +4.46% on KITTI Val).

Figures & tables

Figure 1: MonoIA enables intrinsic awareness in Mono3D on KITTI Val. Existing Mono3D detectors [ 84 , 56 , 89 ] lack intrinsic awareness and thus generalize poorly to images with unseen intrinsics. In contrast, our intrinsic-aware MonoIA achieves superior performance under seen intrinsics and demonstrates strong generalization to the unseen one.

Figure 2: Impact of intrinsic variation on image appearance. Left: The two images show the same object in the same $3$ D position but captured with different intrinsics. As the focal length increases, the object appears larger and the FoV is smaller. Right: Schematic illustration of how intrinsic variations affect object appearance.

Figure 3: Overview of MonoIA. (a) Training Stage: MonoIA is a unified intrinsic-aware detection framework built upon two designs. The Intrinsic Encoder leverages the knowledge of LLM and CLIP to convert numeric intrinsics into semantically meaningful embeddings that capture their perceptual and geometric effects, providing a strong prior for generalization across cameras. The Intrinsic Adaptation Module bridges this semantic knowledge with visual perception through a lightweight Connector and hierarchical fusion, enabling the detector to interpret visual features in an intrinsic-aware manner and maintain consistent $3$ D detection under diverse camera settings. (b) Testing Stage: For each test intrinsic, we retrieve its two nearest seen intrinsics together with their embeddings, and then apply a Hybrid Interpolation Strategy that adaptively switches between nearest-neighbor selection and linear interpolation. If the intrinsic gap is $\leq 32$ px, the nearest seen embedding is reused; otherwise, the two nearest embeddings are linearly interpolated to synthesize the test intrinsic embedding.

Figure 4: LLM-Guided Description Generation. Images rendered with diverse camera intrinsics are fed into an LLM, which generates concise descriptions linking each intrinsic’s perceptual and geometric effects with its numeric focal value, forming semantic intrinsic descriptions.

Figure 5: Cosine similarity of intrinsic embeddings under different encoding strategies (a) Numeric-only encoding produces uniformly high similarity, showing that CLIP text embeddings of raw focal values lack discriminative structure. (b) Our Intrinsic Encoder, which integrates LLM-generated perceptual descriptions with numeric grounding, yields a smooth and ordered similarity pattern, indicating a structured and geometry-aware intrinsic space.

Method	Seen Focals (px)				Unseen Focals (px)
Method	700	900	1100	1300	600	650	750	800	850	950	1000	1050	1150	1200	1250	1350	1400
MonoDETR [ 84 ]	19.15	18.90	16.76	14.22	14.09	16.67	18.55	17.89	16.21	16.54	15.12	15.06	13.66	12.30	11.88	10.08	7.51
MonoDGP [ 56 ]	22.51	21.04	19.96	16.74	17.42	19.28	19.78	19.07	18.51	17.33	16.03	15.63	13.18	12.43	12.47	10.27	7.56
MonoCoP [ 89 ]	23.88	23.30	22.59	18.50	18.18	21.70	22.49	21.44	20.20	18.61	17.69	16.43	14.57	13.46	13.11	12.73	11.11
MonoIA (Ours)	24.41	24.36	23.69	21.20	22.43	23.41	24.13	22.93	23.64	22.48	22.65	22.52	19.07	20.54	20.80	19.25	16.99

Table 1: Results on seen and unseen focal lengths. Seen focals include 700, 900, 1100, and 1300 px. Unseen focals include interpolated focals that lie within the training interval and extrapolated focals that extend beyond the training range. MonoIA achieves the highest AP

{}_{3\text{D}}

across all focal lengths and maintains strong robustness even under extrapolated intrinsics.

Figure 6: Visualization of learned intrinsic embeddings. PCA (Left) and t-SNE (Right). Both views exhibit a smooth, ordered distribution along focal length, indicating that the intrinsic embedding space learned by MonoIA is geometrically consistent and well structured, facilitating interpolation for unseen intrinsics.

Method	GT	$\pm 5$ px	$\pm 10$ px	$\pm 15$ px
MonoDETR [ 84 ]	18.55	16.89	14.95	11.21
MonoDGP [ 56 ]	19.78	17.76	15.38	12.66
MonoCoP [ 89 ]	22.49	20.53	19.22	15.42
MonoIA (Ours)	24.13	23.88	22.34	18.98

Table 2: Results under intrinsic mismatch with different perturbation levels. We evaluate performance under the ground truth and perturbed focal lengths (

\pm 5

\pm 10

\pm 15

px).

Method	Extra Data	Test, AP ${}_{3\text{D}}$ ( $\relbar\joinrel\mathrel{\RHD}$ )			Test, AP ${}_{\text{BEV}}$ ( $\relbar\joinrel\mathrel{\RHD}$ )			Val, AP ${}_{3\text{D}}$ ( $\relbar\joinrel\mathrel{\RHD}$ )			Val, AP ${}_{\text{BEV}}$ ( $\relbar\joinrel\mathrel{\RHD}$ )
Method	Extra Data	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
OccupancyM3D [ 54 ]	LiDAR	25.55	17.02	14.79	35.38	24.18	21.37	26.87	19.96	17.15	35.72	26.60	23.68
OPA-3D [ 67 ]	Depth	24.68	17.17	14.14	32.50	23.14	20.30	24.97	19.40	16.59	33.80	25.51	22.13
MonoTAKD [ 40 ]	LiDAR	27.91	19.43	16.51	38.75	27.76	24.14	34.36	22.61	19.88	42.86	29.41	26.47
MonoUNI [ 25 ]	None	24.75	16.73	13.49	$-$	$-$	$-$	24.51	17.18	14.01	$-$	$-$	$-$
MonoDETR [ 84 ]	None	25.00	16.47	13.58	33.60	22.11	18.60	28.84	20.61	16.38	37.86	26.95	22.80

Table 3: KITTI Leaderboard (Test) and Val results at IoU

{}_{3\text{D}}

\mathbf{\geq 0.7}

. MonoIA achieves SoTA performance across all metrics, demonstrating that our intrinsic aware design also improves standard

3

D benchmarks.

Method	Trained on	$AP_{3D}^{KIT}$	$AP_{3D}^{NU}$
MonoCoP [ 89 ]	KIT	23.98	$-$
MonoCoP [ 89 ]	NU	$-$	7.39
MonoCoP [ 89 ]	KIT $+$ NU	17.26	6.21
MonoCoP [ 89 ] + VD [ 2 ]	KIT $+$ NU	23.15	7.42
MonoIA	KIT	24.40	$-$
MonoIA	NU	$-$	8.12

Table 4: Multi dataset training results. Our intrinsic aware design helps bridge inter dataset discrepancies and improves overall detection performance across KITTI and nuScenes. [Key: KIT = KITTI, NU = nuScenes, Way = Waymo, VD = Virtual Depth]

Method	AP ${}_{3\text{D}}$		AP ${}_{\text{BEV}}$
Method	Easy	Mod.	Easy	Mod.
DEVIANT [ 29 ]	9.69	8.33	16.28	14.36
MonoDETR [ 84 ]	9.53	8.19	16.39	14.41
MonoDGP [ 56 ]	10.04	8.78	16.55	14.53
MonoCoP [ 89 ]	10.85	9.71	17.83	15.86
MonoIA (Ours)	12.33	10.74	19.56	17.33

Table 5: nuScenes Val Results. MonoIA achieves SoTA performance on

3

D detection and BEV detection under IoU

\geq 0.7

. [Key: First , Second ]

Changed	Row Index	From $\relbar\joinrel\mathrel{\RHD}$ To		AP ${}_{3\text{D}}$ , IoU $\geq 0.7$			AP ${}_{3\text{D}}$ , IoU $\geq 0.5$
Changed	Row Index	From $\relbar\joinrel\mathrel{\RHD}$ To		Easy	Mod.	Hard	Easy	Mod.	Hard
Baseline	1	Single Focal		32.40	23.64	20.31	71.30	54.70	48.66
Baseline	2	Synthetic Images		29.77	21.71	17.46	69.53	51.20	46.87
Intrinsic Encoder	3	Yes $\relbar\joinrel\mathrel{\RHD}$	No	29.80	22.16	17.76	69.61	52.63	46.57
Intrinsic Encoder	4	Frozen $\relbar\joinrel\mathrel{\RHD}$	Trainable	29.76	21.85	18.77	68.81	52.30	47.18
Connector	5	Yes $\relbar\joinrel\mathrel{\RHD}$	No	31.97	22.85	19.40	69.54	52.04	48.25

Table 6: Ablation studies on KITTI validate the effectiveness of each module in enabling intrinsic-aware detection.

Method	AP ${}_{3\text{D}}$	Efficiency
Method	AP ${}_{3\text{D}}$	#Param (M)	GFLOPs
MonoDETR [ 84 ]	20.61	35.93	59.72
MonoDGP [ 56 ]	22.34	38.90	68.99
MonoCoP [ 89 ]	23.98	42.50	71.77
MonoIA (Ours)	24.40	42.63	71.77

Table 7: Efficiency comparison on the KITTI Val set. MonoIA achieves the SoTA performance while maintaining comparable model size and computational cost to prior works.

Item	Value
optimizer	AdamW
learning rate	2e-4
weight decay	1e-4
number of feature scales	4
hidden dim	256
nheads	8

Table A1 : Main hyperparameters of MonoIA.

Figure A1 : Examples of synthetic images.

Figure A2 : Overiew of Generation of Intrinsic Texts.

Method	Training Cates	Resolution	Training Datasets			$AP_{3D}^{KIT}$	$AP_{3D}^{NU}$	$AP_{3D}^{Way}$
Method	Training Cates	Resolution	KITTI	nuScenes	Waymo	$AP_{3D}^{KIT}$	$AP_{3D}^{NU}$	$AP_{3D}^{Way}$
MonoDETR [ 84 ]	Car	1280 $\times$ 384	✓			20.61
MonoDGP [ 56 ]	Car	1280 $\times$ 384	✓			22.49
MonoCoP [ 89 ]	ALL	1280 $\times$ 384	✓			23.64
MonoIA	ALL	1280 $\times$ 384	✓			24.40
MonoIA	ALL	896 $\times$ 512		✓			7.09

Table A2 : Detailed results on Multi-Dataset Training. [Key: KIT: KITTI, NU: nuScenes, Way: Waymo]

Method	Extra Data	Venue	AP ${}_{3\text{D}}$ ( $\relbar\joinrel\mathrel{\RHD}$ )			AP ${}_{\text{BEV}}$ ( $\relbar\joinrel\mathrel{\RHD}$ )
Method	Extra Data	Venue	Easy	Mod.	Hard	Easy	Mod.	Hard
OccupancyM3D [ 54 ]	LiDAR	CVPR 24	26.87	19.96	17.15	35.72	26.60	23.68
OPA-3D [ 67 ]	Depth	ICRA 23	24.97	19.40	16.59	33.80	25.51	22.13
MonoFlex [ 85 ]	None	CVPR 21	23.64	17.51	14.83	$-$	$-$	$-$
GUP Net [ 46 ]		CVPR 21	22.76	16.46	13.72	31.07	22.94	19.75
DEVIANT [ 29 ]		ECCV 22	24.63	16.54	14.52	32.60	23.04	19.99

Table A3 : KITTI Val results at IoU

{}_{3\text{D}}

\geq 0.7

. MonoIA achieves SoTA performance across all metrics. [Key: First , Second]

IoU ${}_{3\text{D}}$	Difficulty	Method	APH ${}_{3\text{D}}$ %				AP ${}_{3\text{D}}$ %
IoU ${}_{3\text{D}}$	Difficulty	Method	All	0-30	30-50	50- $\infty$	All	0-30	30-50	50- $\infty$
0.7	Level 1	GUP Net [ 46 ] in [ 29 ]	2.27	6.11	0.80	0.03	2.28	6.15	0.81	0.03
		DEVIANT [ 29 ]	2.67	6.90	0.98	0.02	2.69	6.95	0.99	0.02
		MonoDETR [ 84 ] in [ 89 ]	2.10	5.94	0.73	0.12	2.11	5.99	0.73	0.12
		MonoDGP [ 56 ] in [ 89 ]	2.39	6.62	0.84	0.12	2.41	6.67	0.84	0.12
		MonoCoP [ 89 ]	2.70	7.38	1.06	0.16	2.72	7.44	1.07	0.16

Table A4 : Waymo Val Vehicle results. MonoIA outperforms all methods on most metrics across both difficulty (Level

1

and Level

2

) and IoU threshold (

0.5

and

0.7

). [Key: First , Second ]

Method	IoU ${}_{3\text{D}}$ $\geq 0.7$				IoU ${}_{3\text{D}}$ $\geq 0.5$
	AP ${}_{3\text{D}}$		AP ${}_{\text{BEV}}$		AP ${}_{3\text{D}}$		AP ${}_{\text{BEV}}$
	Easy	Mod.	Easy	Mod.	Easy	Mod.	Easy	Mod.
DEVIANT [ 29 ]	9.69	8.33	16.28	14.36	31.47	28.22	35.61	31.93
MonoDETR [ 84 ]	9.53	8.19	16.39	14.41	31.81	28.35	35.70	31.96
MonoDGP [ 56 ]	10.04	8.78	16.55	14.53	29.56	26.17	32.67	29.44
MonoCoP [ 89 ]	10.85	9.71	17.83	15.86	33.70	29.91	37.44	34.01

Table A5 : nuScenes Val Results. MonoIA achieves SoTA performance on

3

D detection and BEV detection. [Key: First , Second ]

Panel of Figure 21 — Figure A3 : Generalizability of Intrinsic Awareness (IA) on different baseline methods .

Panel of Figure 22 — Figure A4 : Generalizability of Intrinsic Awareness (IA) on different image backbones .

Number of Intrinsic Texts	AP ${}_{\!3\text{D}\!}$ 70 (%) ( $\relbar\joinrel\mathrel{\RHD}$ )
Number of Intrinsic Texts	Easy	Mod	Hard
0	29.80	22.16	17.76
1	32.53	23.69	20.09
12	33.96	24.17	20.55
24	33.61	24.40	20.80
36	33.64	23.41	20.83

Table A6 : Impact of Number of Intrinsic Texts.

Learned Tokens	AP ${}_{\!3\text{D}\!}$ 70 (%) ( $\relbar\joinrel\mathrel{\RHD}$ )
Learned Tokens	Easy	Mod	Hard
[700]	31.78	23.26	19.82
[700, 900]	32.32	23.54	20.14
[700, 900, 1100]	33.68	24.13	20.72
[700, 900, 1100, 1300]	33.61	24.40	20.80
[700, 900, 1100, 1300, 1500]	32.36	23.41	19.93

Table A7 : Impact of Number of Intrinsic Embeddings.

Figure A5 : Qualitative results on KITTI. [Key: MonoIA , MonoCoP ]

Figure A6 : Qualitative results on nuScenes. [Key: MonoIA , MonoCoP ]

Figure A7 : Qualitative results on Waymo. [Key: MonoIA , MonoCoP ]

3D Object Detection

3D object detection identifies objects and estimates their locations, sizes, and orientations from images, point clouds, radar, or fused sensors.

See all 181 papers

MoCA3D: Monocular 3D Bounding Box Prediction in the Image Plane
Mar 20, 2026Changwoo Jeon, Rishi Upadhyay, Achuta Kadambi3D Object Detection Monocular 3D Object Detection
University of California, Los Angeles · Yonsei University
LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection
Dec 5, 2025Johannes Meier, Jonathan Michel, Oussema Dhaouadi +7Efficient Object Detection Real-Time Object Detection
DeepScenario · ETH Zurich · TU Munich +5
MonoPRIO: Adaptive Prior Conditioning for Unified Monocular 3D Object Detection
May 14, 2026Leon Davies, Qinggang Meng, Mohamad Saada +23D Object Detection Monocular 3D Object Detection
Department of Computer Science, Loughborough University, Epinal Way, Loughborough, LE11 3TU, Leicestershire, United Kingdom · European Center for Risk & Resilience Studies, University of Southern Denmark, Degnevej 14, Esbjerg, 6705, Denmark

Mar 20, 2026J/K move · Enter open · S save

MoCA3D: Monocular 3D Bounding Box Prediction in the Image Plane

Changwoo Jeon, Rishi Upadhyay, Achuta Kadambi

University of California, Los Angeles · Yonsei University

3D Object Detection Monocular 3D Object Detection

Open paper AskPDF arXiv

Monocular 3D object understanding has largely been cast as a 2D RoI-to-3D box lifting problem. However, emerging downstream applications require image-plane geometry (e.g., projected 3D box corners) which cannot be easily obtained without known intrinsics, a problem for object detection in the wild. We introduce MoCA3D, a Monocular, Class-Agnostic 3D model that predicts projected 3D bounding box corners and per-corner depths without requiring camera intrinsics at inference time. MoCA3D formulates pixel-space localization and depth assignment as dense prediction via corner heatmaps and depth maps. To evaluate image-plane geometric fidelity, we propose Pixel-Aligned Geometry (PAG), which directly measures image-plane corner and depth consistency. Extensive experiments demonstrate that MoCA3D achieves state-of-the-art performance, improving image-plane corner PAG by 22.8% while remaining comparable on 3D IoU, using up to 57 times fewer trainable parameters. Finally, we apply MoCA3D to downstream tasks which were previously impractical under unknown intrinsics, highlighting its utility beyond standard baseline models.

Abstract

3D Object Detection

Explore similar work

MoCA3D: Monocular 3D Bounding Box Prediction in the Image Plane

LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection

MonoPRIO: Adaptive Prior Conditioning for Unified Monocular 3D Object Detection