MAGneT-3D: Monocular and Domain-Generalizable Temporal 3D Detection

Mohamed Kotb; Johannes Meier; Christoph Reich; Oussema Dhaouadi; Luis Denninger; Daniel Cremers

cs.CVAug 14, 2026

MAGneT-3D: Monocular and Domain-Generalizable Temporal 3D Detection

Authors: Mohamed Kotb, Johannes Meier, Christoph Reich, Oussema Dhaouadi, Luis Denninger, Daniel Cremers

Organizations: TU Munich · ETH Zurich · MCML · TU Darmstadt ·

Abstract

Monocular temporal 3D detection aims to detect objects in 3D, given a monocular video. Query-based 3D detectors unify detection and cross-view association, but their learnable queries fit the spatial distribution of the training data (e.g., field-of-view). We show that this issue is especially severe when these models are applied to monocular video, hindering generalization to unseen datasets and environments. To address this limitation, we introduce MAGneT-3D, the first method for domain-generalized monocular temporal 3D object detection. Instead of relying on static learnable queries, we propose a Domain-Robust Anchor Generator (DRAG) approach that adaptively derives 3D proposals during inference. To further enable domain generalization, we propose a Temporal Refinement and Identity Merging (TRIM) strategy, reducing dependence on specific 3D proposals. To enable comprehensive domain-generalization evaluation, we establish a cross-dataset benchmark spanning nuScenes, Waymo, Lyft, and ONCE. Under zero-shot domain shifts, MAGneT-3D outperforms all baselines, improving NDS from 12.1% to 18.6% while also increasing in-domain accuracy.

Figures & tables

Panel of Figure 1 — Figure 1: MAGneT-3D overview. MAGneT-3D bridges monocular, temporal modeling and domain generalization (left) for strong cross-dataset 3D detection (right) . Prior detectors instead treat these settings as distinct or paired, leaving the intersection unexplored (left) . On our new cross-dataset benchmark (right) , MAGneT-3D trained on nuScenes outperforms on both in-domain (source) and all unseen (target) datasets. We report the nuScenes Detection Score (NDS, in %, $\uparrow$ ).

Property	nuScenes [ 4 ]	Waymo [ 53 ]	Lyft L5 [ 13 ]	ONCE [ 40 ]
Annotated frames (train / val)	$28\,130$ / $6019$	$53\,000$ / $13\,280$	$15\,498$ / $3150$	$5000$ / $3000$
Front camera resolution (in px)	1600 $\times$ 900	1920 $\times$ 1280	1224 $\times$ 1024	1920 $\times$ 1020
Front focal length $f_{x}$ (in px)	1266	2059	844	960
Front horizontal FOV (in °)	70.0	50.0	70.0	90.0
Number of object categories	10	4	9	5
Country	USA & Singapore	USA	USA	China

Table 1: Cross-dataset benchmark overview. Our cross-dataset domain generalization benchmark comprises four 3D monocular detection datasets, differing markedly in their field-of-view (FOV), intrinsics, resolution & geographic coverage.

Panel of Figure 2 — Figure 2: Object locations across datasets. Top-view heatmaps of annotated object locations for Waymo [ 53 ] , nuScenes [ 4 ] , Lyft L5 [ 13 ] , and ONCE [ 40 ] . A larger saturation corresponds to a higher density. Object locations differ significantly between datasets.

Panel of Figure 3 — Figure 3: Query-based 3D detectors [ 59 , 30 , 31 , 32 ] . The current image is fed into a backbone. The resulting image features interact with the learnable queries (bottom) . A propagation transformer takes in query features and historically tracked queries, producing refined 3D detections. Queries (bottom) are learned during training but static during inference and introduce training dataset dependent biases, introducing issues under domain shifts, e.g . , when the field-of-view (FOV) is changing.

Panel of Figure 5 — Figure 5: MAGneT-3D. Instead of static, learnable queries, we generate dynamic proposals (left top) based on image features (DRAG; cf . Sec. 4.3 ), eliminating source domain biases. The proposals interact spatially with the image features and are fed, alongside historic queries, into the temporal prediction refiner. During training, our soft assigner matches each target to multiple proposals for dense supervision. During inference, duplicate objects are merged via clustering (TRIM; cf . Sec. 4.4 ).

Method	Val $\xrightarrow{}$	mAP $\uparrow$					NDS $\uparrow$
Method	Train $\downarrow$	nuSCN	Waymo	Lyft	ONCE	Avg. Cross	nuSCN	Waymo	Lyft	ONCE	Avg. Cross
StreamPETR [ 59 ]	nuSCN	26.5	$4.8$	$5.0$	$8.2$	$6.0$	28.5	13.1	$10.6$	$12.7$	$12.1$
	Waymo	$4.2$	34.5	$0.8$	5.4	$3.4$	9.3	38.0	8.9	$10.2$	9.5
	Lyft	$1.0$	$0.0$	27.8	$5.4$	$2.1$	$5.4$	$4.4$	32.6	$11.2$	$7.0$
Sparse4D v3 [ 32 ]	nuSCN	24.8	$4.4$	8.6	11.1	8.0	27.0	$9.5$	13.1	14.7	12.4
	Waymo	4.9	32.8	2.0	$4.8$	3.9	$8.9$	38.9	$7.8$	$9.1$	$8.6$

Table 2: Comparison against the state of the art. Each model is trained on a single source dataset (vertical) and evaluated on all target datasets (horizontal) . Avg. Cross is the mean over the three unseen datasets. The best result per training dataset is highlighted in green

\blacksquare\!\!\!\!\!\blacksquare

; the second best in yellow

\blacksquare\!\!\!\!\!\blacksquare

; in-domain results in gray

\blacksquare\!\!\!\!\!\blacksquare

. mAP & NDS are both in % (

\uparrow

). nuScenes is abbreviated by nuSCN.

Configuration	DRAG	TRIM	mAP $\uparrow$					NDS $\uparrow$
Configuration	DRAG	TRIM	nuSCN	Waymo	Lyft	ONCE	Avg. Cross	nuSCN	Waymo	Lyft	ONCE	Avg. Cross
1. StreamPETR			26.5	$4.8$	$5.0$	$8.2$	$6.0$	28.5	$13.1$	$10.6$	$12.7$	$12.1$
	✓		20.0	7.9	8.6	$8.8$	$8.4$	25.6	16.3	13.9	$13.9$	14.7
		✓	27.1	$6.9$	$8.6$	11.9	9.1	27.0	$11.7$	$13.6$	16.4	$13.9$
4. MAGneT-3D	✓	✓	28.5	10.1	14.3	12.4	12.3	32.9	17.2	19.5	19.1	18.6

Table 3: Main ablation. We start with the StreamPETR baseline ablate both DRAG and TRIM. We train on nuScenes [ 4 ] (nuSCN). mAP & NDS are both in % (

\uparrow

). The best result is highlighted in

\blacksquare\!\!\!\!\!\blacksquare

; the second best in

\blacksquare\!\!\!\!\!\blacksquare

; in-domain results in

\blacksquare\!\!\!\!\!\blacksquare

Setting	mAP $\uparrow$					NDS $\uparrow$
Setting	nuSCN	Waymo	Lyft	ONCE	Avg. Cross	nuSCN	Waymo	Lyft	ONCE	Avg. Cross
MAGneT-3D (Full)	28.5	10.1	14.3	12.4	12.3	32.9	17.2	19.5	19.1	18.6
w/o Two-phase training	26.9	8.2	12.2	12.0	10.8	30.7	16.2	18.3	18.9	17.8
w/o Virtual depth	27.5	6.2	11.6	8.9	8.9	31.2	13.9	17.4	13.0	14.8
w/o Scale invariant feat. assn.	17.6	6.5	5.3	7.1	6.3	23.7	14.5	12.5	11.2	12.7

Table 4: DRAG ablation. We ablate the core components of DRAG. Each row removes one component from the full model, trained on nuScenes [ 4 ] (nuSCN). mAP & NDS are both in % (

\uparrow

). The best result is highlighted in

\blacksquare\!\!\!\!\!\blacksquare

; the second best in

\blacksquare\!\!\!\!\!\blacksquare

; in-domain results in

\blacksquare\!\!\!\!\!\blacksquare

Setting	mAP $\uparrow$					NDS $\uparrow$
Setting	nuSCN	Waymo	Lyft	ONCE	Avg. Cross	nuSCN	Waymo	Lyft	ONCE	Avg. Cross
MAGneT-3D (Full, $N_{\mathrm{keep}}\!=\!1$ )	28.5	10.1	14.3	12.4	12.3	32.9	17.2	19.5	19.1	18.6
w/ Top- $N_{\mathrm{keep}}$ clust. ( $N_{\mathrm{keep}}\!=\!3$ )	26.2	8.9	13.2	11.0	11.0	30.6	16.8	18.5	16.3	17.2
w/ Top- $N_{\mathrm{keep}}$ clust. ( $N_{\mathrm{keep}}\!=\!5$ )	25.7	$8.5$	$13.1$	$10.5$	$10.7$	29.9	$16.5$	$18.4$	$15.6$	$16.8$
w/o Clustering	25.6	$8.4$	$13.1$	$10.2$	$10.6$	29.4	$16.3$	$18.4$	$15.0$	$16.6$
w/ Hungarian assigner	20.0	$7.9$	$8.6$	$8.8$	$8.4$	25.6	$16.3$	$13.9$	$13.9$	$14.7$

Table 5: TRIM analysis. We analyze the assigner setting and the inference-time clustering of TRIM, trained on nuScenes [ 4 ] (nuSCN). mAP & NDS are both in % (

\uparrow

). The best result is highlighted in

\blacksquare\!\!\!\!\!\blacksquare

; the second best in

\blacksquare\!\!\!\!\!\blacksquare

; in-domain results in

\blacksquare\!\!\!\!\!\blacksquare

Panel of Figure 6 — Figure 6: Spatial distribution results. Top-view distribution of 3D detections from MAGneT-3D and our baseline StreamPETR [ 59 ] . Both are trained on nuScenes [ 4 ] (source dataset). The angles represent the prediction range (max-min azimuth). MAGneT-3D adapts to different angles of the target datasets, while StreamPETR fails to adapt to the angle of the target datasets. More saturated colors indicate a higher density. Color coding: ground truth $\blacksquare\!\!\!\!\!\blacksquare$ (low density) $\xrightarrow{}$ $\blacksquare\!\!\!\!\!\blacksquare$ (high density); MAGneT-3D $\blacksquare\!\!\!\!\!\blacksquare$ $\xrightarrow{}$ $\blacksquare\!\!\!\!\!\blacksquare$ ; StreamPETR $\blacksquare\!\!\!\!\!\blacksquare$ $\xrightarrow{}$ $\blacksquare\!\!\!\!\!\blacksquare$ . Best viewed in color; zoom in for details.

Panel of Figure 7 — Figure 7: Qualitative results. MAGneT-3D yields fewer false positives and more accurate predictions than StreamPETR [ 59 ] . All models are trained on nuScenes [ 4 ] and evaluated across nuScenes, Lyft [ 13 ] , and Waymo [ 53 ] . Best viewed in color; zoom in for details. BEV color coding: Ground truth $\blacksquare\!\!\!\!\!\blacksquare$ , StreamPETR $\blacksquare\!\!\!\!\!\blacksquare$ , and MAGneT-3D $\blacksquare\!\!\!\!\!\blacksquare$ .

3D Object Detection

3D object detection identifies objects and estimates their locations, sizes, and orientations from images, point clouds, radar, or fused sensors.

See all 181 papers

PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection
Jul 3, 2025Seokyeong Lee, Sithu Aung, Junyong Choi +33D Object Detection Pseudo-Labeling
KAIST AI · VRG, FEE, Czech Technical University in Prague · Samsung Research +3
PQR3D: Progressive Query Refinement over Reference-Conditioned Temporal Windows for Multi-View 3D Object Detection
Sep 26, 2026Hui Ye, Yudong Liu, Yiran Chen +2Multi-View 3D Object Detection 3D Object Detection
Department of Computer Science, Georgia State University · Department of Electrical and Computer Engineering, Duke University · School of Computing, University of Connecticut
LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection
Dec 5, 2025Johannes Meier, Jonathan Michel, Oussema Dhaouadi +7Efficient Object Detection Real-Time Object Detection
DeepScenario · ETH Zurich · TU Munich +5

Jul 3, 2025J/K move · Enter open · S save

PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection

Seokyeong Lee, Sithu Aung, Junyong Choi, Seungryong Kim+2

KAIST AI · VRG, FEE, Czech Technical University in Prague · Samsung Research+3

3D Object Detection Pseudo-Labeling Monocular 3D Object Detection

Open paper AskPDF arXiv

Monocular 3D object detection is crucial for scalable perception across fields like autonomous driving, robotics, and surveillance. However, progress is hindered by limited 3D annotations and the inherent ambiguity of single-image geometry. Existing methods often rely on strong geometric assumptions or carefully curated datasets, which limit their applicability to real-world scenarios. In this paper, we present PLOT (Pseudo-Labeling via Object Tracking), a framework that generates 3D annotations from monocular videos without auxiliary sensors or model retraining. PLOT tracks object and background trajectories to estimate camera motion and perform object association in pose-unknown settings. These trajectories provide point correspondences that align frame-wise pseudo-LiDARs, which are then fused via simple optimization into a unified object shape robust to occlusion and viewpoint shifts. Recognizing temporal coherence as a fundamental requirement for reliable shape fusion and video perception, we design a global object memory that preserves consistent object identities across frames. PLOT achieves robust annotation quality and strong generalization on both M3OD video benchmarks and in-the-wild videos, proving its effectiveness across diverse and unconstrained domains. Project page: https://plot-eccv.github.io.

Abstract

3D Object Detection

Explore similar work

PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection

PQR3D: Progressive Query Refinement over Reference-Conditioned Temporal Windows for Multi-View 3D Object Detection

LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection