MonoMIP: Multi-Height Inverse Perspective Mapping for Monocular 3D Object Detection

Emir Ulurak; Furkan Kıraç

cs.CVOct 13, 2026

MonoMIP: Multi-Height Inverse Perspective Mapping for Monocular 3D Object Detection

Authors: Emir Ulurak, Furkan Kıraç

Organizations: Ozyegin University, Istanbul, Turkey

Abstract

Monocular 3D object detection methods increasingly rely on depth-guided transformers, but they operate entirely in the perspective image plane where spatial relationships are distorted by foreshortening. We propose MonoMIP, a dual-decoder framework that adds complementary 3D geometric reasoning through Bird's-Eye View (BEV) features constructed via Multi-Height Inverse Perspective Mapping. We perform IPM at K elevation planes spanning ground to rooftop, producing an ordered stack of height-conditioned metric BEV features that reduces above-ground misprojection and exposes depth-dependent appearance variation across planes. A BEV decoder with multi-scale deformable attention aggregates this multi-height evidence per query, while a depth-guided 3D decoder provides complementary perspective-space predictions. Without extra training data, MonoMIP attains 25.84% Moderate AP3D|R40 on KITTI val and 21.66% on the official test set for Car, outperforming recent published monocular methods on both the KITTI and Waymo benchmarks.

Figures & tables

Figure 1: Cross-height agreement follows the projection geometry. The row offset between height planes decreases as $1/Z$ , so cross-height feature similarity is expected to rise with depth. Learned BEV features preserve this: mean cross-height cosine similarity increases with ground-truth depth ( $r{=}0.75$ ), while wider pairs remain less aligned than adjacent body-level planes.

Figure 2: Overall architecture of MonoMIP. A shared backbone extracts multi-scale features. The image-space decoder produces depth-aware object proposals, while the BEV decoder builds multi-height IPM features and refines metric boxes through deformable attention across height planes. Both decoders share queries and are trained jointly. The schematic’s depth expression includes the learned residual and denotes $\hat{z}$ .

Figure 3: Multi-Height IPM pipeline. Bottom-left: Multi-scale encoded features ( $f_{0}$ – $f_{3}$ ) are fused via learned attention into a single feature map ( $F_{\mathrm{fused}}$ ). Top: For each of $K{=}4$ height planes, the rectified KITTI projection matrix $\mathbf{P}_{2}$ provides intrinsic parameters $\mathbf{K}$ and small offsets $t_{x},t_{y}$ ; together with the fixed height planes $Y_{k}$ , they define a mapping $\pi(X,Z,Y_{k};\mathbf{K},t)$ from each BEV grid cell to an image coordinate $(u,v_{k})$ . Features are sampled via grid_sample (bilinear interpolation), producing $K$ BEV feature maps. Right: Reference contrasts (Eq. ( 5 ), omitted from the schematic) are normalized via GroupNorm and stacked as multi-scale inputs to the BEV decoder. The IPM sampling grid is geometry-based with no learned parameters; only the feature fusion and normalization layers are trainable.

Figure 4: Cross-height geometry on a KITTI scene. The four height planes sample the same image column at depth-dependent rows. Near objects produce distinct roof-to-ground samples, while far objects produce more similar samples. The row shift follows the $1/Z$ law, matching Eq. ( 4 ).

Method	Extra data	Venue	Val. $\mathrm{AP_{3D\|R40}}$			Val. $\mathrm{AP_{BEV\|R40}}$			Test $\mathrm{AP_{3D\|R40}}$			Test $\mathrm{AP_{BEV\|R40}}$
			Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
CaDDN [ 6 ]	LiDAR	CVPR’21	23.57	16.31	13.84	–	–	–	19.17	13.41	11.46	27.94	18.91	17.19
OccupancyM3D [ 33 ]		CVPR’24	26.87	19.96	17.15	35.72	26.60	23.68	25.55	17.02	14.79	35.38	24.18	21.37
OPA-3D [ 44 ]	Depth	ICRA’23	24.97	19.40	16.59	33.80	25.51	22.13	24.60	17.05	14.25	33.54	22.53	19.22
MonoFlex [ 53 ]	None	CVPR’21	23.64	17.51	14.83	–	–	–	19.94	13.89	12.07	28.23	19.75	16.89
GUPNet [ 3 ]		ICCV’21	22.76	16.46	13.72	31.07	22.94	19.75	20.11	14.20	11.77	–	–	–

Table 1: KITTI Car comparison at IoU

=

0.7. R40 3D/BEV AP on val and official test . Bold / underline : best/second-best without extra sensor data. ‡ : Car-only; † : all-class training (Pedestrian/Cyclist in Table 2 ). MonoIA also uses language/foundation-model conditioning.

Method	Ped. (Mod.)	Cyc. (Mod.)	Params (M)	Time (ms)
MonoDGP [ 5 ]	9.89	2.82	42.16	25.6
MonoCoP [ 9 ]	10.33	5.08	42.53	25.3
MonoMIP (Ours)	12.41	6.01	39.86	27.9

Table 2: Multi-class detection and efficiency. Pedestrian and Cyclist KITTI test

\mathrm{AP_{3D|R40}}

(Moderate, IoU

=

0.5; all classes trained jointly, no extra data); we bold the best and underline the second-best AP. Parameters and inference time are measured by us for all methods under one protocol on a single RTX 3090 (ResNet-50 backbone, batch size 1, forward only, FP32).

Figure 5: Per-distance Car

\mathrm{AP_{3D|R40}}

(Moderate), KITTI val . Panels use separate

y

-axis scales. Gains concentrate in the

20

–

40

m band, where cross-height row separation stays visible but monocular depth is hard.

(a) Number of height levels $K$
$K$	Heights (m)	$\mathrm{AP_{3D\|R40}}$			$\mathrm{AP_{BEV\|R40}}$
		Easy	Mod.	Hard	Easy	Mod.	Hard
1	{1.65}	32.83	24.83	21.78	41.26	31.18	27.65
2	{0.25, 1.65}	33.04	25.34	22.46	41.68	31.92	28.78
3	{0.25, 1.0, 1.65}	33.16	25.64	22.89	41.92	32.36	29.45
4	{0.25, 0.75, 1.25, 1.65}	33.24	25.84	23.17	42.09	32.66	29.90

Table 3: Ablations on KITTI val (Car,

\mathrm{AP_{R40}}

; ResNet-50). (a) More height levels

K

improve AP. (b) Components added to the base image-space decoder (IoU

=

0.7).

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Setting
Backbone	ResNet-50
Input resolution	$1280\times 384$ (W $\times$ H)
Feature levels	$L=4$
Hidden dimension	$256$
Feed-forward width	$256$
Attention heads	$8$

Appendix

Table S1 : Full implementation details of the MonoMIP model.

Figure S1 : Per-level BEV feature magnitude after reference-anchored cross-height contrast (ground in default) (Eq. ( S13 )). Panels correspond to the ground feature $\mathbf{B}^{\prime}_{0}$ and the height-minus-ground contrast levels $\mathbf{B}^{\prime}_{k}$ supplied to the BEV decoder. Each panel is normalized independently.

Height set (m)	Interpretation	$\mathrm{AP_{3D\|R40}}$ Mod.	$\mathrm{AP_{BEV\|R40}}$ Mod.
$\{1.65\}$	single ground plane	24.83	31.18
$\{0.25,1.65\}$	top + ground	25.34	31.92
$\{0.25,1.0,1.65\}$	coarse 3-plane	25.64	32.36
$\{0.25,0.75,1.25,1.65\}$	default ( $K{=}4$ )	25.84	32.66
$\{0.45,0.85,1.25,1.65\}$	lower range	25.77	32.52
$\{0.10,0.60,1.10,1.65\}$	higher range	25.80	32.57

Appendix

Table S2 : Height-plane sensitivity of trained models (KITTI val , Car, Moderate). Top : number of planes

K

(context; matches the main-paper

K

-ablation). Bottom : two placement changes retaining the

1.65

m reference, and one random ordered set. The two small changes stay within

0.1

AP of the default, shown in bold .

Intervention	Type	$\mathrm{AP_{3D\|R40}}$ Mod.	$\Delta$
Full MonoMIP (ordered)	—	25.84	—
Test-time shuffle (mean of 4)	inference	$14.35$ $\,\pm 3.9$	$-11.49$
Test-time reverse	inference	11.17	$-14.67$

Appendix

Table S3 : Height-layout robustness (KITTI val , Car,

\mathrm{AP_{3D|R40}}

Mod.). Inference-only permutations of the trained level order; the checkpoint and weights are unchanged.

Method	Min	Max	Mean $\pm$ Std
MonoMIP ( $K{=}4$ , full)	24.80	25.84	$25.27\pm 0.30$

Appendix

Table S4 : Training stability of MonoMIP across 10 seeds on KITTI val (Car,

\mathrm{AP_{3D|R40}}

Moderate, IoU

0.7

). Min, max, and mean

\pm

std over ten independent training runs of the full

K{=}4

model; the best run corresponds to our reported result.

Table 14

(a) IoU ${}_{\mathrm{3D}}\geq 0.5$
Method	Diff.	$\mathrm{APH_{3D}}$				$\mathrm{AP_{3D}}$
		All	0–30	30–50	50– $\infty$	All	0–30	30–50	50– $\infty$
GUP Net [ 3 ]	L1	9.94	24.59	4.78	0.22	10.02	24.78	4.84	0.22
DEVIANT [ 2 ]		10.89	26.64	5.08	0.18	10.98	26.85	5.13	0.18
MonoDETR † [ 8 ]		9.60	23.58	4.67	0.99	9.68	23.78	4.72	1.00
MonoUNI [ 1 ]		10.73	26.30	3.98	0.55	10.98	26.63	4.04	0.57

Appendix

Table S7 : Waymo Open Dataset val Vehicle results. All methods are trained on the Waymo front-camera split (DEVIANT [ 2 ] protocol;

52{,}386

train /

39{,}848

val) and scored with the official Waymo

\mathrm{AP_{3D}}

/ heading-weighted

\mathrm{APH_{3D}}

at Level 1 / Level 2 and IoU

{}_{\mathrm{3D}}\!\in\!\{0.5,0.7\}

, broken down by range (All /

[0,30)

[30,50)

[50,\infty)

m). MonoMIP uses the KITTI-default metric height planes and leads on the mid-far-range bins at both thresholds. [Key: first , second ; † : retrained by [ 9 ] .]

Figure S2 : Qualitative KITTI val comparison: MonoMIP vs. MonoCoP. Six validation scenes (rows), each with the image-space 3D-detection panel ( left ) and the metric BEV panel ( right , depth ticks: $10$ – $50$ m). Green dashed boxes are ground truth, blue boxes are MonoMIP, and orange boxes are MonoCoP; dashed magenta circles mark telling objects. MonoMIP’s blue boxes consistently sit closer to the green ground truth in the BEV panel than MonoCoP’s orange. This is clearest on the highlighted mid- and far-range vehicles, where MonoCoP misplaces depth along the viewing ray while MonoMIP keeps the metric footprint aligned.

Figure S3 : Qualitative MonoMIP detections on the Waymo Open Dataset (front camera, val ). Predicted 3D boxes ( blue ) from our reported model on six validation scenes. MonoMIP localizes dense traffic across highway and urban scenes under the KITTI-default metric height planes.

Panel of Figure 18 — Figure S4 : Different plane placements on a Waymo scene. Predicted 3D boxes ( blue ) from (a) the KITTI-default and (b) shifted-plane models. The bottom-left vehicle is missed in (a) but detected in (b). This qualitative example motivates examining vertical coverage, but different trained checkpoints prevent attributing the detection change to planes alone.

Perturbation	$\mathrm{AP_{3D\|R40}}$ Mod.	$\Delta$
none	25.84	—
$H_{\mathrm{cam}}-0.20$ m	21.98	$-3.86$
$H_{\mathrm{cam}}-0.10$ m	23.54	$-2.30$
$H_{\mathrm{cam}}+0.10$ m	22.66	$-3.17$
$H_{\mathrm{cam}}+0.20$ m	19.21	$-6.63$

Appendix

Table S8 : Camera-height sensitivity under inference-time perturbations (KITTI val , Car,

\mathrm{AP_{3D|R40}}

Mod.), measured on the MonoMIP checkpoint. All IPM planes are offset together without retraining. Focal perturbations are reported separately in Table S9 .

Model / intervention	$-2.5\%$ AP	$+2.5\%$ AP
MonoDGP, full network	15.82	14.25
MonoCoP, full network	15.92	14.73
MonoMIP, full network	15.42	14.24
MonoMIP, geometric depth only	17.32	17.20
MonoMIP, MH-IPM only	24.37	23.62

Appendix

Table S9 : Focal-sensitivity attribution. Absolute Car Moderate

\mathrm{AP_{3D|R40}}

, inference only. Top: both network-input focal axes perturbed. Bottom: only the geometric-depth

f_{x}

term or MH-IPM

(f_{x},f_{y})

perturbed.

Variant	3D AP	Params (M)	ms/image	MiB
2D + BEV ∗	21.02	40.389	26.939	459.269
3D + BEV †	25.84	39.856	28.137	457.230
2D + 3D + BEV ∗	24.99	41.736	29.174	464.403
3D + BEV, LSS ∗	24.05	39.856	30.545	457.230

Appendix

Table S10 : Decoder and view-transform ablations. Car Moderate

\mathrm{AP_{3D|R40}}

. ∗ : three-seed mean; † : reported model reference (Sec. F ). Costs are from a separate RTX 3090 profiling session, batch 1, FP32, forward only. Memory is peak allocated VRAM.

Monocular 3D Object Detection

Monocular 3D object detection estimates objects’ positions, dimensions, and orientations in 3D from one camera, without depth sensors.

See all 22 papers

Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs
Aug 12, 2026Yung-Hsu Yang, Luigi Piccinelli, Samuel Rota Bulò +7Multi-View 3D Object Detection 3D Object Detection
ETH Zürich, Switzerland · Meta Reality Labs Zürich, Switzerland · University of Bonn, Germany
LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection
Dec 5, 2025Johannes Meier, Jonathan Michel, Oussema Dhaouadi +7Efficient Object Detection Real-Time Object Detection
DeepScenario · ETH Zurich · TU Munich +5
MonoPRIO: Adaptive Prior Conditioning for Unified Monocular 3D Object Detection
May 14, 2026Leon Davies, Qinggang Meng, Mohamad Saada +23D Object Detection Monocular 3D Object Detection
Department of Computer Science, Loughborough University, Epinal Way, Loughborough, LE11 3TU, Leicestershire, United Kingdom · European Center for Risk & Resilience Studies, University of Southern Denmark, Degnevej 14, Esbjerg, 6705, Denmark

Aug 12, 2026J/K move · Enter open · S save

Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs

Yung-Hsu Yang, Luigi Piccinelli, Samuel Rota Bulò, Sunghwan Hong+6

ETH Zürich, Switzerland · Meta Reality Labs Zürich, Switzerland · University of Bonn, Germany

Multi-View 3D Object Detection 3D Object Detection Feed-Forward 3D Reconstruction Monocular 3D Object Detection

Open paper AskPDF arXiv

Metric 3D object detection is a core capability for embodied agents, yet most reliable systems lean on depth sensors, trading away cost, power, and integration simplicity. This motivates monocular 3D detection, which avoids additional constraints, yet it faces a major obstacle: from a single image, depth, and especially absolute scale, are underconstrained. As a result, the prevailing pattern of detecting in 2D and then predicting 3D attributes is often brittle, since modest range errors can dominate 3D localization, and the learned scale prior can fail when cameras, motion, or environments undergo domain shifts. To address this, we propose Map-Det3D, an online multi-view 3D object detection model that brings detection directly into a 3D space reconstructed from RGB. We map a short temporal window into multiple views and repurpose a feed-forward metric 3D reconstruction model as our geometric backbone while tuning its object-aware capabilities. Building on this representation, Map-Det3D directly predicts boxes in metric 3D space, without the widely used 2D-to-3D lifting. Experiments across different benchmarks show that this design supports strong online performance and robust transfer without adaptation, suggesting that training reconstruction priors for detection is a practical route to stable metric 3D detection from monocular video. Code and models are available at https://royyang0714.github.io/Map-Det3D.

Abstract

Monocular 3D Object Detection

Explore similar work

Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs

LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection

MonoPRIO: Adaptive Prior Conditioning for Unified Monocular 3D Object Detection