Monocular 3D object detection offers a low-cost route to 3D perception, but it remains fundamentally ill-posed, since depth must be inferred from a single view. Recent transformer-based detectors learn geometry purely from image features and 3D box annotations, lacking the high-level semantic priors that vision-language models (VLMs) acquire from large-scale pretraining. VLMs offer such priors, but deploying one directly as a detector is impractical due to its heavy inference cost. We propose MonoLaDN, a language-guided denoising framework that transfers the geometric reasoning of a VLM into a monocular 3D detector while keeping the deployed model purely vision-based. During training, a frozen VLM produces structured reasoning about each object's occlusion, depth, dimension, and angle from the image and its ground-truth boxes. We encode this reasoning into queries, perturb them with noise, and train the detector to reconstruct the clean reasoning. Reconstructing the corrupted reasoning forces the detector's depth-guided features to internalize the VLM's geometric priors, so this knowledge remains available even after the reasoning path is discarded at inference, adding no latency or parameters to the deployed model. Experiments on KITTI 3D and nuScenes show that MonoLaDN consistently outperforms state-of-the-art detectors and generalizes well to new domains.
Figures & tables
Figure 1: Comparison of paradigms for monocular 3D detection. (a) VLM-based methods carry rich semantic and geometric priors but are too costly for real-time inference. (b) Traditional vision-only detectors are efficient yet lack such priors, learning geometry only from image features and box annotations. (c) Our MonoLaDN injects VLM-derived 3D reasoning into a depth-guided detector during training, so it gains these priors while keeping a vision-only, efficient model at inference.
Figure 2: Overview of MonoLaDN. During training, a 3D Reasoning Generation stage queries a frozen VLM with the image and ground-truth 3D boxes to produce structured reasoning about each object’s occlusion, depth, dimension, and angle. A text encoder and noise injector turn this reasoning into perturbed queries, decoded jointly with the N learnable object queries under a Language-Guided Mask and trained to reconstruct the clean attributes, alongside the detection loss. At inference, the reasoning path is discarded, leaving a purely vision-based detector with no added cost.
Figure 3: 3D Reasoning Generation. The object-centric visual input and a geometry-to-language mapping are fed to a frozen VLM. The mapping discretizes each attribute into an ordinal label: depth by distance binning, angle by viewpoint sectoring, dimension by shape-archetype matching, and occlusion by combining truncation and occlusion flags. The VLM then produces structured reasoning per attribute.
Figure 4: Object reconstruction heads. Each attribute output E^a is decoded into the box component it reasons about. The occlusion output E^occ has no box target of its own and is broadcast as shared visibility context (dashed) to the dimension, depth, and angle heads. When a head takes more than one embedding, the inputs are averaged before the head; the 2D center head uses the mean of all four outputs. The reconstructed components are supervised by the same 2D and 3D objectives as the detection loss.
Test, AP3D∣R40
Test, APBEV∣R40
Val, AP3D∣R40
Val, APBEV∣R40
Methods
Extra data
Venue
Easy
Mod.
Hard
Easy
Mod.
Hard
Easy
Mod.
Hard
Easy
Mod.
Hard
MonoDTR [ 9 ]
CVPR 2022
21.99
15.39
12.73
28.59
20.38
17.14
24.52
18.57
15.51
33.33
25.35
21.68
DID-M3D [ 24 ]
ECCV 2022
24.40
16.29
13.75
32.95
22.76
19.83
22.98
16.12
14.03
31.10
22.76
19.50
OccupancyM3D [ 25 ]
LiDAR
CVPR 2024
25.55
17.02
14.79
35.38
24.18
21.37
26.87
19.96
17.15
35.72
26.60
23.68
MonoPGC [ 37 ]
ICRA 2023
24.68
17.17
14.14
32.50
23.14
20.30
25.67
18.63
15.65
34.06
24.26
20.78
OPA-3D [ 33 ]
Depth
RAL 2023
24.60
17.05
14.25
33.54
22.53
19.22
24.97
19.40
16.59
33.80
25.51
22.13
Table 1: Comparisons with state-of-the-art monocular methods on the KITTI test and val sets for the car category. We bold the best results and underline the second-best results. The blue refers to the gain and the red is the decrease of our method.
Table 2: Cross-dataset depth accuracy and computational complexity. (a) Depth prediction MAE ( ↓ ) on KITTI and nuScenes frontal validation. (b) Complexity on a single RTX 3090 GPU with batch size 1. Best in bold, second best underlined.
Ablation
Val, AP3D∣R40
Lemb
Lobj
Alp
Asr
Text Encoder
VLM
Easy
Mod.
Hard
(a)
×
×
×
×
×
×
30.76
22.34
19.02
(b)
✓
×
✓
×
CLIP [ 27 ]
Qwen2.5-VL [ 1 ]
31.41
22.55
19.34
(c)
✓
×
✓
×
SigLIP [ 43 ]
Qwen2.5-VL [ 1 ]
31.63
22.72
19.53
(d)
×
✓
✓
×
SigLIP [ 43 ]
Qwen2.5-VL [ 1 ]
32.07
23.15
19.98
(e)
✓
✓
✓
×
SigLIP [ 43 ]
Qwen2.5-VL [ 1 ]
33.46
23.61
20.57
Table 3: Ablation of each component on the KITTI val set. Lemb / Lobj are the embedding and box reconstruction losses. Alp / Asr the leakage-prevention and separated-reasoning masks. Text Encoder embeds the reasoning text, and VLM is the model that generates it. The best results are in bold.
Figure 5: Qualitative results on KITTI val set. (a) MonoDETR [ 45 ] . (b) MonoDGP [ 26 ] . (c) MonoLaDN (Ours). For each image set, the top row presents the camera-view visualization, while the bottom row offers the corresponding bird’s-eye view. Ground-truth bounding boxes are rendered in green , and predictions are shown in order: red , yellow , blue . We also circle some objects to highlight the difference between other state-of-the-art methods and our method.
Figure 6: Deformable cross-attention of the reasoning queries on the KITTI val set. Each row is one object, and the four columns show the occlusion, depth, dimension, and angle reasoning queries. For each query, we accumulate its deformable sampling locations at the final decoder layer, weighted by the corresponding attention weights, into a Gaussian heatmap overlaid on the image (red high, blue low). The target object is marked in green , and the generated reasoning for each attribute is shown beneath its column.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Decoder self-attention under the Language-Guided Mask on the KITTI val set. Each row is one scene. The left column shows the image with all projected ground-truth 3D boxes (the target object in orange , others in green ) and the generated reasoning for its four attributes. The right column shows the decoder self-attention matrix over the W=4M+N queries, ordered as the 4M reasoning queries followed by the N=50 learnable queries; the red lines separate the two groups and the thin gray lines delimit the four-attribute block of each object.
Attribute
Rule
Label
Occlusion
trunc ≤0.15 , occ =0
Fully visible
occ >0
Partially occluded
trunc >0.15
Truncated by border
trunc >0.15 , occ >0
Truncated and occluded
Depth ( z )
z≤10
Very near
10<z≤20
Near
Appendix
Table 4: Geometry-to-language label rules. Each attribute is mapped from the ground-truth annotation to a discrete ordinal label. α is the observation angle, z the camera-frame depth, and h/w/l the box height/width/length.
Methods
Pedestrian
Cyclist
Easy
Mod.
Hard
Easy
Mod.
Hard
GUPNet [ 21 ]
14.72
9.53
7.87
4.18
2.65
2.09
MonoCon [ 19 ]
13.10
8.41
6.94
2.80
1.92
1.55
DEVIANT [ 12 ]
13.43
8.65
7.69
5.05
3.13
2.59
MonoDDE [ 17 ]
11.13
7.32
6.67
5.94
3.78
3.33
MonoDETR [ 45 ]
12.65
7.19
6.72
5.12
2.74
2.02
Appendix
Table 5: Comparison on the KITTI test set for the pedestrian and cyclist categories at IoU = 0.5, reporting AP3D∣R40 . Best in bold.
Method
Val, AP3D∣R40
Training time
Memory
Easy
Mod.
Hard
(min/epoch) ↓
Usage (GB) ↓
MonoDGP
30.76
22.34
19.02
4.68
14.35
MonoDGP + Ours
35.21
25.03
21.60
5.23
20.45
Appendix
Table 6: Training Complexity.
VLM
Val, AP3D∣R40
Total time
Memory
Easy
Mod.
Hard
(h) ↓
Usage (GB) ↓
Qwen2.5-VL-7B [ 1 ]
34.78
24.52
21.13
220.7
9.35
InternVL3.5-8B [ 36 ]
35.04
24.83
21.47
73.6
17.45
SpaceThinker-3B [ 28 ]
35.21
25.03
21.60
230.6
9.35
Appendix
Table 7: 3D Reasoning Generation Complexity. Comparison of the VLMs used to generate the reasoning text, reporting val AP3D∣R40 for the Car category alongside the total generation time over the dataset and the memory cost. The reasoning is generated offline during training only, so it adds no inference cost.
Figure 8: Designs of the language-guided attention mask. Black entries are blocked and white entries are permitted. The leakage-prevention mask Alp stops the learnable queries from attending to any reasoning query. The separated-reasoning mask Asr lets each reasoning query attend to all reasoning queries of the same object and to those of the same attribute across objects. The attribute-isolated mask Aai is a stricter variant that additionally blocks different attributes of the same object from attending to each other.
Alp
Asr
Aai
Easy
Mod.
Hard
×
×
×
17.25
10.92
9.87
✓
×
×
33.46
23.61
20.57
✓
✓
×
34.78
24.52
21.13
✓
×
✓
33.64
23.81
20.72
✓
✓
✓
33.05
23.42
20.11
Appendix
Table 8: Ablation on the language-guided mask design on the KITTI val set, reporting AP3D∣R40 for the Car category. All rows use SigLIP as the text encoder and Qwen2.5-VL-7B as the reasoning VLM. Alp , Asr , and Aai denote the leakage-prevention, separated-reasoning, and attribute-isolated masks. The best results are in bold .
Figure 9: Strategies for forming the perturbed queries. All three start from the ground-truth 3D boxes and end with perturbed queries, but inject noise at different stages. BBox perturbation perturbs the boxes before the VLM, so the reasoning is generated from corrupted geometry. Reasoning perturbation generates clean reasoning and then applies text augmentation to the reasoning text before encoding. Embedding perturbation (ours) encodes the clean reasoning and injects learned noise in the embedding space.
Noise Injection Strategy
Easy
Mod.
Hard
BBox perturbation
33.15
22.83
19.65
Reasoning perturbation
34.05
23.69
20.34
Embedding perturbation (ours)
34.78
24.52
21.13
Appendix
Table 9: Ablation on the noise injection strategy on the KITTI val set, reporting AP3D∣R40 for the Car category. All rows use SigLIP as the text encoder and Qwen2.5-VL-7B as the reasoning VLM. The best results are in bold .
Figure 10: Additional qualitative results on KITTI val set. (a) MonoDETR [ 45 ] . (b) MonoDGP [ 26 ] . (c) MonoLaDN (Ours). For each image set, the top row presents the camera-view visualization, while the bottom row offers the corresponding bird’s-eye view. Ground-truth bounding boxes are rendered in green , and predictions are shown in order: red , yellow , blue . We also circle some objects to highlight the difference between other state-of-the-art methods and our method.