MonoSAOD: Monocular 3D Object Detection with Sparsely Annotated Label

Junyoung Jung; Seokwon Kim; Jung Uk Kim

cs.CVApr 2, 2026

MonoSAOD: Monocular 3D Object Detection with Sparsely Annotated Label

Authors: Junyoung Jung, Seokwon Kim, Jung Uk Kim

Organizations: Kyung Hee University

Abstract

Monocular 3D object detection has achieved impressive performance on densely annotated datasets. However, it struggles when only a fraction of objects are labeled due to the high cost of 3D annotation. This sparsely annotated setting is common in real-world scenarios where annotating every object is impractical. To address this, we propose a novel framework for sparsely annotated monocular 3D object detection with two key modules. First, we propose Road-Aware Patch Augmentation (RAPA), which leverages sparse annotations by augmenting segmented object patches onto road regions while preserving 3D geometric consistency. Second, we propose Prototype-Based Filtering (PBF), which generates high-quality pseudo-labels by filtering predictions through prototype similarity and depth uncertainty. It maintains global 2D RoI feature prototypes and selects pseudo-labels that are both feature-consistent with learned prototypes and have reliable depth estimates. Our training strategy combines geometry-preserving augmentation with prototype-guided pseudo-labeling to achieve robust detection under sparse supervision. Extensive experiments demonstrate the effectiveness of the proposed method. The source code is available at https://github.com/VisualAIKHU/MonoSAOD .

Figures & tables

Figure 1: (a) Visible objects are annotated in one scene but missed in another due to the difficulty of 3D bounding box annotation and human error, resulting in inconsistent supervision. (b) Comparison between fully annotated and sparsely annotated labels. In the sparse annotation setting, only a subset of objects is labeled while many valid objects remain unlabeled.

Figure 2: Overall architecture of the proposed framework. A teacher–student structure integrates Road-Aware Patch Augmentation (RAPA) and Prototype-Based Filtering (PBF) for robust training under sparse annotations.

Method	30%			50%			70%
Method	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
Baseline [ 38 ]	11.17	8.73	7.56	20.36	15.25	12.76	22.78	17.83	14.97
Co-mining [ 30 ]	16.01	12.62	10.38	21.47	16.22	13.61	24.10	18.21	15.16
SparseDet [ 27 ]	16.95	13.30	10.97	21.99	16.13	13.43	23.58	18.39	15.54
Calibrated Teacher [ 29 ]	17.14	12.96	10.58	22.04	16.03	13.36	25.39	18.94	15.86
Co-student [ 33 ]	15.99	12.67	10.38	23.06	16.39	13.45	23.71	18.11	15.12

Table 1: Detection results of car category on the KITTI validation set under sparse annotation ratios of 30%, 50%, and 70%. We compare our method with existing Sparsely Annotated Object Detection (SAOD) approaches reproduced using their official implementations for fair comparison. Bold / underline fonts indicate the best/second-best results.

Method	Easy	Mod.	Hard
Baseline [ 38 ]	7.12	5.12	3.90
Co-mining [ 30 ]	9.93	6.41	5.25
SparseDet [ 27 ]	10.76	7.42	6.16
Calibrated Teacher [ 29 ]	9.26	6.09	5.02
Co-student [ 33 ]	9.96	6.97	5.33
Proposed Method	17.47	11.36	8.83

Table 2: Detection results of car category on the KITTI test set under 30% annotation setting. Our method achieves large improvements over existing SAOD methods. Bold / underline fonts indicate the best/second-best results.

Conf.	RAPA	PBF	Easy	Mod.	Hard
-	-	-	11.17	8.73	7.56
✓	-	-	12.39	9.68	8.18
✓	-	✓	16.49	12.65	10.32
✓	✓	-	20.31	14.51	11.72
✓	✓	✓	21.28	15.60	12.79

Table 3: Ablation study of the proposed modules on the KITTI validation set for car category under 30% annotation ratio. ‘Conf.’: confidence score-based pseudo-labeling. ‘RAPA’: Road-Aware Patch Augmentation, ‘PBF’: Prototype-Based Filtering.

Figure 3: Progression of pseudo-labels selected by the proposed PBF module for GT Bank enrichment. Green boxes denote sparse ground truths and previously accumulated pseudo-labels, while red boxes indicate high-quality pseudo-labels newly selected at the current step. The consistent selection of geometrically and semantically reliable pseudo-labels highlights the effectiveness of the PBF module.

Figure 4: Line plot of GT annotation growth, where 30% (blue), 50% (green), and 70% (red) represent different sparsity levels.

Method	Easy	Mod.	Hard
MonoDETR (ICCV’23) [ 38 ]	11.17	8.73	7.56
MonoDETR + Ours	21.28	15.60	12.79
MonoDGP (CVPR’25) [ 22 ]	15.56	11.70	9.53
MonoDGP + Ours	22.62	16.79	14.07

Table 4: Generalization results of our framework applied as a plug-in to different monocular 3D detectors on the KITTI validation set(30% annotation). Incorporating our method yields consistent improvements across architectures.

Method	Easy	Mod.	Hard
Baseline	11.10	7.43	5.81
Co-mining [ 30 ]	11.22	7.91	6.20
Sparsedet [ 27 ]	11.40	7.93	6.52
Calibrated Teacher [ 29 ]	11.97	8.65	7.26
Co-student [ 33 ]	11.57	7.97	6.20
Proposed Method	19.11	13.72	10.35

Table 5: Performance under foggy KITTI (fog density 0.1) validation set, evaluated under the 30% annotation setting. Our method demonstrates improved robustness in adverse weather conditions compared to existing SAOD methods.

Figure 5: Examples of the RAPA module. Objects segmented from a source image are geometrically transformed and placed onto plausible road regions in the target image. It produces visually realistic and geometrically consistent augmentations.

Figure S.1 : Conceptual visualization of 3D geometric translation of RAPA. The coordinates $(x_{t}^{\prime},z_{t}^{\prime})$ represent the new 3D position of the object after applying the horizontal translation. Car A is translated to position B while preserving the observation angle $\alpha$ to maintain consistent patch appearance. The global orientation $r_{y}$ is updated to $r_{y}^{\prime}$ based on the new camera viewing angle ( $\theta$ to $\theta_{new}$ ), ensuring 3D geometric consistency.

Method	Clear Image (30%)						Foggy Image (30%)
	$\text{AP}_{3D}$			$\text{AP}_{BEV}$			$\text{AP}_{3D}$			$\text{AP}_{BEV}$
	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
Baseline	11.17	8.73	7.56	17.24	13.62	11.49	11.10	7.43	5.81	16.78	11.99	9.91
Co-mining	16.01	12.62	10.38	24.81	18.31	15.28	11.22	7.91	6.20	17.04	12.09	9.76
SparseDet	16.95	13.30	10.97	24.78	19.41	16.52	11.40	7.93	6.52	17.64	12.77	10.41
Calibrated Teacher	17.14	12.96	10.58	24.35	18.04	15.01	11.97	8.65	7.26	18.99	13.86	11.53

Table S.1 : Detection comparison on the Clear and Foggy KITTI images under the 30% annotation ratio. We report both

\text{AP}_{3D}

and

\text{AP}_{BEV}

Method	10%			20%			30%
Method	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
Baseline [ 8 ]	1.04	1.15	0.15	5.38	4.70	3.70	11.17	8.73	7.56
Co-mining [ 5 ]	0.00	2.50	2.50	3.88	3.90	3.26	16.01	12.62	10.38
SparseDet [ 3 ]	1.88	1.67	1.67	8.83	7.07	5.59	16.95	13.30	10.97
Calibrated Teacher [ 4 ]	1.62	1.55	1.05	6.04	4.88	4.17	17.14	12.96	10.58
Co-student [ 6 ]	0.00	2.50	2.50	1.25	1.66	1.66	15.99	12.67	10.38

Table S.2 : Detection results of car category on the KITTI validation set under the lower annotation ratios of 10%, 20%, and 30%. We compare our method with existing SAOD approaches reproduced using their official implementations for fair comparison. Bold / underline fonts indicate the best/second-best results.

RAPA	$\bm{S_{\text{proto}}}$	$\bm{S_{\text{depth}}}$	Easy	Mod.	Hard
-	-	-	11.17	8.73	7.56
-	✓	-	16.07	12.26	9.90
-	✓	✓	16.49	12.65	10.32
✓	✓	-	18.73	14.85	12.13
✓	✓	✓	21.28	15.60	12.79

Table S.3 : Ablation study of PBF modules on the KITTI validation set for the car category under 30% annotation ratio.

$\bm{\tau_{\text{depth}}}$	$\bm{\tau_{\text{proto}}}$	Easy	Mod.	Hard
0.8		21.75	15.45	12.48
1.0	0.85	21.28	15.60	12.79
1.2		22.12	15.47	12.32
	0.75	21.56	15.57	12.62
1.0	0.85	21.28	15.60	12.79
	0.95	20.46	14.58	11.80

Table S.4 : Hyperparameter sensitivity analysis on KITTI (30%).

Figure S.2 : Visualization of PBF filtering results. In the image view, green boxes indicate the sparsely annotated ground-truth labels, while blue boxes denote the selected pseudo-labels and red boxes indicate predictions filtered out by PBF. In the BEV view (right), green boxes represent the original full ground-truth annotations, and blue boxes correspond to the retained pseudo-labels. The BEV visualization highlights how well the selected pseudo-labels align with the original ground-truth geometry, demonstrating the effectiveness of PBF in filtering out predictions.

Method	Easy	Mod.	Hard
Mix-Teaching [ 7 ]	15.42	13.26	11.17
CMAug [ 9 ]	17.15	13.30	11.44
RAPA	20.31	14.51	11.72
Mix-Teaching + PBF	18.96	14.77	12.22
CMAug + PBF	18.93	14.79	12.61
RAPA + PBF	21.28	15.60	12.79

Table S.5 : Comparison with other patch augmentation methods used in monocular 3D object detection tasks, on the KITTI validation set under the 30% annotation ratio. Results are reported with and without PBF to show the isolated contribution of each augmentation approach.

Figure S.3 : Examples of simulated segmentation errors: dilation, erosion, and boundary approximation.

Method	Easy	Mod.	Hard
Baseline (No Aug)	11.17	8.73	7.56
Scale (Dilation)	18.47	14.07	11.38
Scale (Erosion)	18.04	13.86	11.19
Boundary Approx.	20.04	14.30	11.54
SAM (Ours)	20.31	14.51	11.72

Table S.6: Robustness to segmentation noise on KITTI with 30% annotations.

Figure S.4: Visualization of pseudo-labels added during training. (a) shows the sparsely annotated ground-truth labels. (b) presents the combined set of ground-truth and newly added pseudo-labels, where blue boxes denote the newly added pseudo-labels and green boxes represent all available labels (sparse ground-truth + accumulated pseudo-labels). (c) visualizes the BEV alignment between the newly added pseudo-labels (blue) and the original full ground-truth annotations (green), illustrating that the generated pseudo-labels are geometrically consistent with the true object locations.

Figure S.5: Progression of pseudo-labels selected by the proposed PBF module for GT Bank enrichment on foggy images. Green boxes denote sparse ground truths and previously accumulated pseudo-labels, while red boxes indicate high-quality pseudo-labels newly selected at the current step. The consistent selection of geometrically and semantically reliable pseudo-labels highlights the effectiveness of the PBF module.

Figure S.6: Qualitative comparison of augmentation methods. Proposed method (RAPA) produces realistic augmentations by placing objects at valid locations with geometrically consistent 3D bounding boxes.

3D Object Detection

3D object detection identifies objects and estimates their locations, sizes, and orientations from images, point clouds, radar, or fused sensors.

See all 176 papers

Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision
Mar 18, 2026Nikhil Gosala, B. Ravi Kiran, Senthil Yogamani +1Visual Object Tracking 3D MOT
University of Freiburg, Germany · Qualcomm SARL France · Automated Driving, Qualcomm Technologies, Inc., USA
PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection
Jul 3, 2025Seokyeong Lee, Sithu Aung, Junyong Choi +33D Object Detection Pseudo-Labeling
KAIST AI · VRG, FEE, Czech Technical University in Prague · Samsung Research +3
Unified Unsupervised and Sparsely-Supervised 3D Object Detection by Semantic Pseudo-Labeling and Prototype Learning
Feb 25, 2026Yushen He, Lei Zhao, Weidong Chen3D Object Detection Pseudo-Labeling
School of Automation and Intelligent Sensing and Institute of Medical Robotics, Shanghai Jiao Tong University, and Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China.

Mar 18, 2026J/K move · Enter open · S save

Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision

Nikhil Gosala, B. Ravi Kiran, Senthil Yogamani, Abhinav Valada

University of Freiburg, Germany · Qualcomm SARL France · Automated Driving, Qualcomm Technologies, Inc., USA

Visual Object Tracking 3D MOT Weakly Supervised Learning Self-Training

Open paper AskPDF arXiv

Monocular 3D object tracking aims to estimate temporally consistent 3D object poses across video frames, enabling autonomous agents to reason about scene dynamics. However, existing state-of-the-art approaches are fully supervised and rely on dense 3D annotations over long video sequences, which are expensive to obtain and difficult to scale. In this work, we address this fundamental limitation by proposing the first sparsely supervised framework for monocular 3D object tracking. Our approach decomposes the task into two sequential sub-problems: 2D query matching and 3D geometry estimation. Both components leverage the spatio-temporal consistency of image sequences to augment a sparse set of labeled samples and learn rich 2D and 3D representations of the scene. Leveraging these learned cues, our model automatically generates high-quality 3D pseudolabels across entire videos, effectively transforming sparse supervision into dense 3D track annotations. This enables existing fully-supervised trackers to effectively operate under extreme label sparsity. Extensive experiments on the KITTI and nuScenes datasets demonstrate that our method significantly improves tracking performance, achieving an improvement of up to 15.50 p.p. while using at most four ground truth annotations per track.

Abstract

3D Object Detection

Explore similar work

Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision

PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection

Unified Unsupervised and Sparsely-Supervised 3D Object Detection by Semantic Pseudo-Labeling and Prototype Learning