MoCA3D: Monocular 3D Bounding Box Prediction in the Image Plane

Changwoo Jeon; Rishi Upadhyay; Achuta Kadambi

cs.CVMar 20, 2026

MoCA3D: Monocular 3D Bounding Box Prediction in the Image Plane

Authors: Changwoo Jeon, Rishi Upadhyay, Achuta Kadambi

Organizations: University of California, Los Angeles · Yonsei University

Abstract

Monocular 3D object understanding has largely been cast as a 2D RoI-to-3D box lifting problem. However, emerging downstream applications require image-plane geometry (e.g., projected 3D box corners) which cannot be easily obtained without known intrinsics, a problem for object detection in the wild. We introduce MoCA3D, a Monocular, Class-Agnostic 3D model that predicts projected 3D bounding box corners and per-corner depths without requiring camera intrinsics at inference time. MoCA3D formulates pixel-space localization and depth assignment as dense prediction via corner heatmaps and depth maps. To evaluate image-plane geometric fidelity, we propose Pixel-Aligned Geometry (PAG), which directly measures image-plane corner and depth consistency. Extensive experiments demonstrate that MoCA3D achieves state-of-the-art performance, improving image-plane corner PAG by 22.8% while remaining comparable on 3D IoU, using up to 57 times fewer trainable parameters. Finally, we apply MoCA3D to downstream tasks which were previously impractical under unknown intrinsics, highlighting its utility beyond standard baseline models.

Figures & tables

Figure 1: MoCA3D architecture. Given an RGB image and a tight oracle 2D bounding box, MoCA3D uses a frozen DINOv3 backbone and a box-conditioned 3D Geometry Transformer with dense modules to predict eight corner heatmaps and per-corner depth maps, yielding pixel-aligned projected 3D box corners and depths.

Figure 2: Heatmap Comparison. Predicted corner heatmaps with (a) peak weight $=50.0$ and (b) $=1.0$ (uniform). Larger peak weight sharpens and localizes responses near GT corners, improving soft-argmax stability, while uniform weighting yields flatter heatmaps.

Figure 3: Qualitative Results. MoCA3D vs. DetAny3D predictions under oracle 2D boxes on samples from the KITTI, Omni3D, and Hypersim datasets. Detections in green have a 3D IoU of less than 0.1, making them low-quality detections.

PAG uv (px) $\downarrow$
Model	kitti	nuscenes	objectron	arkitscenes	sunrgbd	hypersim
Cube R-CNN	10.10	11.73	23.57	22.47	23.17	13.92
Cube R-CNN ∗	10.38	11.86	31.28	23.75	24.06	14.17
OVMono3D-LIFT ∗	10.29	11.82	23.55	22.55	22.36	13.02
DetAny3D	4.90	6.75	27.41	17.15	21.06	10.71
MoCA3D	3.12	5.63	18.60	14.90	16.99	9.56

Table 1: Image-plane geometry metrics: MoCA3D outperforms or is comparable to prior models across all datasets. All results were computed with oracle 2D bounding boxes and ground truth intrinsics were used to calculate pixel-space coordinates for models other than MoCA3D. Green indicates the best and orange indicates the second-best results. Cube R-CNN ∗ is trained in a class-agnostic setting.

NHD $\downarrow$
Model	kitti	nuscenes	objectron	arkitscenes	sunrgbd	hypersim
Cube R-CNN	0.2811	0.3067	0.2462	0.3590	0.4466	1.0603
Cube R-CNN ∗	0.3504	0.3758	0.5282	0.4074	0.5642	1.1875
OVMono3D-LIFT ∗	0.3177	0.3458	0.2132	0.3527	0.4864	1.0216
DetAny3D	0.1911	0.2428	0.2107	0.2593	0.3026	0.8970
MoCA3D	0.1902	0.3002	0.2519	0.2529	0.3381	1.0406

Table 2: 3D camera space metrics: MoCA3D is competitive with prior work such as DetAny3D despite a significantly smaller parameter count and compute requirements. Metrics were computed using ground truth camera intrinsics and oracle 2D bounding boxes. Green indicates the best and orange indicates the second-best results.

Figure 4: Efficiency of MoCA3D. We compare (a) trainable parameters and (b) trade-off between efficiency and performance (PAG uv ). Efficiency is defined as the inverse of end-to-end inference time per example on CV-Bench [ 43 ] .

Method	PAG uv $\downarrow$ (px)	PAG d $\downarrow$ (%)	IoU 3D $\uparrow$	#Params (M)	Train (GPU-hrs)
MoCA3D (baseline)	16.05	10.83	0.3768	$19.0$	$27.0$
MoCA3D w/ DA	$16.14$	$10.92$	$0.3682$	$19.8$	–
$\Delta$ vs. MoCA3D	+0.09	+0.09	-0.0086
MoCA3D w/o box prior	$16.35$	$10.88$	$0.3736$	$19.0$	$27.0$
$\Delta$ vs. MoCA3D	+0.29	+0.05	-0.0032
Direct Regressor	$17.38$	$14.89$	$0.3064$	$22.1$	$19.2$

Table 3: MoCA3D ablations. We report PAG uv , PAG d (lower is better) and IoU 3D (higher is better);

\Delta

denotes method

-

MoCA3D.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: MoCA3D-Cube Adapter. MoCA3D-Cube combines MoCA3D’s predicted corner-depth pairs with an RoI-aligned decoder feature and camera intrinsics $K$ to regress a parametric 3D bounding box. The adapter reuses image-plane geometry as the primary cue while adding a lightweight Cube MLP for conventional box prediction.

Dataset	Total	Train	Val	Test
KITTI	7,481	3,321	391	3,769
SUN RGB-D	10,335	4,929	356	5,050
nuScenes	34,149	26,215	1,915	6,019
Objectron	46,644	33,519	3,811	9,314
ARKitScenes	60,924	48,046	5,268	7,610
Hypersim	74,619	59,543	7,386	7,690

Appendix

Table 4: Image statistics of the Omni3D source datasets.

Figure 7: Qualitative comparisons under oracle 2D boxes. MoCA3D is compared with DetAny3D, OVMono3D, and Cube R-CNN on representative samples. Each prediction is shown together with its 3D IoU score, which is also displayed inside the corresponding image. Across diverse scenes, MoCA3D generally yields more accurate and visually consistent 3D geometry.

Figure 8: Additional driving scene generation results. Additional examples of controllable driving-scene editing using MoCA3D-recovered image-plane geometry.

Figure 9: Failure Cases. We show representative failure cases of MoCA3D. For each example, the left panel shows the input image with the tight oracle 2D box, and the right panel shows the corresponding MoCA3D prediction.

3D Object Detection

3D object detection identifies objects and estimates their locations, sizes, and orientations from images, point clouds, radar, or fused sensors.

See all 181 papers

Towards Intrinsic-Aware Monocular 3D Object Detection
Mar 28, 2026Zhihao Zhang, Abhinav Kumar, Xiaoming Liu3D Object Detection Monocular 3D Object Detection
Michigan State University · University of North Carolina at Chapel Hill
LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection
Dec 5, 2025Johannes Meier, Jonathan Michel, Oussema Dhaouadi +7Efficient Object Detection Real-Time Object Detection
DeepScenario · ETH Zurich · TU Munich +5
Towards Camera-Robust 3D Localization: Equation-Anchored Tool-Use for MLLMs
May 19, 2026Xueying Jiang, Wenhao Li, Quanhao Qian +43D Object Detection 3D VLMs
1Nanyang Technological University · 2DAMO Academy, Alibaba Group · 3HuPan Lab +1

Mar 28, 2026J/K move · Enter open · S save

Towards Intrinsic-Aware Monocular 3D Object Detection

Zhihao Zhang, Abhinav Kumar, Xiaoming Liu

Michigan State University · University of North Carolina at Chapel Hill

3D Object Detection Monocular 3D Object Detection Domain Generalization Representation Learning

Open paper AskPDF arXiv

Monocular 3D object detection (Mono3D) aims to infer object locations and dimensions in 3D space from a single RGB image. Despite recent progress, existing methods remain highly sensitive to camera intrinsics and struggle to generalize across diverse settings, since intrinsics govern how 3D scenes are projected onto the image plane. We propose MonoIA, a unified intrinsic-aware framework that models and adapts to intrinsic variation through a language-grounded representation. The key insight is that intrinsic variation is not a numeric difference but a perceptual transformation that alters apparent scale, perspective, and spatial geometry. To capture this effect, MonoIA employs large language models and vision-language models to generate intrinsic embeddings that encode the visual and geometric implications of camera parameters. These embeddings are hierarchically integrated into the detection network via an Intrinsic Adaptation Module, allowing the model to modulate its feature representations according to camera-specific configurations and maintain consistent 3D detection across intrinsics. This shifts intrinsic modeling from numeric conditioning to semantic representation, enabling robust and unified perception across cameras. Extensive experiments show that MonoIA achieves new state-of-the-art results on standard benchmarks including KITTI, Waymo, and nuScenes (e.g., +1.18% on the KITTI leaderboard), and further improves performance under multi-dataset training (e.g., +4.46% on KITTI Val).

Abstract

3D Object Detection

Explore similar work

Towards Intrinsic-Aware Monocular 3D Object Detection

LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection

Towards Camera-Robust 3D Localization: Equation-Anchored Tool-Use for MLLMs