VERIA: Verification-Centric Multimodal Instance Augmentation for Long-Tailed 3D Object Detection

Jumin Lee; Siyeong Lee; Namil Kim; Sung-Eui Yoon

cs.CVMar 25, 2026

VERIA: Verification-Centric Multimodal Instance Augmentation for Long-Tailed 3D Object Detection

Authors: Jumin Lee, Siyeong Lee, Namil Kim, Sung-Eui Yoon

Organizations: KAIST · Naver Labs

Abstract

Long-tail distributions in driving datasets pose a fundamental challenge for 3D perception, as rare classes exhibit substantial intra-class diversity yet available samples cover this variation space only sparsely. Existing instance augmentation methods based on copy-paste or asset libraries improve rare-class exposure but are often limited in fine-grained diversity and scene-context placement. We propose VERIA, an image-first multimodal augmentation framework that synthesizes synchronized RGB--LiDAR instances using off-the-shelf foundation models and curates them with sequential semantic and geometric verification. This verification-centric design tends to select instances that better match real LiDAR statistics while spanning a wider range of intra-class variation. Stage-wise yield decomposition provides a log-based diagnostic of pipeline reliability. On nuScenes and Lyft, VERIA improves rare-class 3D object detection in both LiDAR-only and multimodal settings. Our code is available at https://sgvr.kaist.ac.kr/VERIA/.

Figures & tables

Fig. 1: Motivation for VERIA. (a) Driving datasets exhibit long-tail distributions, limiting 3D perception performance. (b) LiDAR point returns grow sparser with range, amplifying intra-class geometric variation. (c) Existing methods operate in the LiDAR domain and place objects without scene context, constraining diversity to curated asset libraries. (d) VERIA synthesizes objects conditioned on RGB context using foundation models, supporting subclass-level diversity with synchronized pseudo-LiDAR.

Fig. 2: Overview of VERIA. (a) Given a target category $\mathcal{C}$ , a VLM generates a subclass-level description $\mathcal{T}_{c}$ and physical size priors; a 3D bounding box is sampled and projected to define the inpainting region for RGB-context-conditioned synthesis. Semantic verification retains candidates that pass category correctness, scene-level plausibility, and artifact severity checks. (b) Verified RGB instances are converted to synchronized pseudo-LiDAR via segmentation, depth estimation, and spherical projection. Geometric verification further filters implausible reconstructions, yielding verified RGB–LiDAR pairs for downstream training.

Fig. 3: Qualitative augmentations on nuScenes and Lyft with paired RGB and LiDAR. For each dataset, we show an original scene (a) and the corresponding augmented scene (b), alongside the individually synthesized instances. On nuScenes, we augment construction vehicle (I1), motorcycle (I2), and bicycle (I3); on Lyft, we augment bicycle (I1) and motorcycle (I2). Each instance is composited using a collision-aware strategy; in RGB, instances are layered in depth order, while in LiDAR, occluded background points are removed from the sensor origin. Red 3D bounding boxes indicate the augmented objects, which are placed at plausible locations within the scene. Both datasets show decreasing point density with range, consistent with real LiDAR characteristics, though Lyft’s 64-beam sensor yields comparatively denser returns at distance than nuScenes’ 32-beam configuration.

Fig. 4: Yield versus geometric tolerance $\lambda$ on nuScenes and Lyft. Yield increases monotonically with $\lambda$ as relaxing the consistency check admits a larger fraction of candidates.

Dataset (N)	Metric	Method
		InternVL3	InternVL3	Qwen3VL	Qwen3VL
		/UniDepth2	/MoGe2	/UniDepth2	/MoGe2
nuScenes (550,098)	$\mathbb{P}(S_{\text{sem}})$	81.29	81.29	91.71	91.71
	$\mathbb{P}(S_{\text{geo}})$	81.73	83.60	81.73	83.60
	$\mathbb{P}(S_{\text{sem}}\cap S_{\text{geo}})$	71.42	72.05	75.99	76.76
Lyft (209,270)	$\mathbb{P}(S_{\text{sem}})$	89.33	89.33	86.24	86.24

TABLE I: Semantic and geometric acceptance rates and final yield over the

N

synthesized candidates on nuScenes and Lyft, reported across four verifier–depth configurations.

Modality	Method	Per-class AP (%)										mAP (%)
Modality	Method	Car	Pedestrian	Barrier	Truck	Traffic cone	Trailer	Bus	Constr. Veh.	Motorcycle	Bicycle	mAP (%)
LiDAR- only	CenterPoint
	+ GT-Aug	84.86	85.50	67.40	58.33	70.77	39.65	70.59	21.93	68.67	57.10	62.48
	+ PGT-Aug	85.16	85.49	68.43	57.59	70.73	39.14	70.25	24.23	68.70	59.94	62.97
	+ Text3DAug	85.02	85.02	68.05	57.77	71.17	40.68	71.91	23.39	69.05	58.79	63.09
	+ VERIA (InternVL3/MoGe2)	84.68	85.46	68.44	59.03	69.99	40.52	70.23	24.33	69.71	58.84	63.12

TABLE II: 3D object detection on nuScenes. Overall mAP and per-class AP, with augmented categories highlighted in bold. VERIA is comparable to LiDAR-only augmentation baselines and shows larger gains in the multimodal setting.

Modality	Method	Per-class AP (%)							mAP (%)
Modality	Method	Car	Other Veh.	Pedestrian	Bicycle	Truck	Bus	Motorcycle	mAP (%)
LiDAR- only	CenterPoint
	+ GT-Aug	36.40	30.60	5.80	5.10	18.60	20.70	4.40	17.37
	+ Text3DAug	36.20	30.70	6.10	5.40	19.10	21.90	4.50	17.70
	+ VERIA (InternVL3/MoGe2)	36.69	30.73	6.02	6.02	19.17	21.27	5.52	17.92
	+ VERIA (Qwen3VL/MoGe2)	36.60	30.80	6.10	5.90	20.70	21.13	4.60	17.98

TABLE III: 3D object detection on Lyft. Overall mAP and per-class AP, with augmented categories highlighted in bold. VERIA improves bicycle and motorcycle AP in both LiDAR-only and multimodal settings, consistent with the nuScenes results.

Module	Model	Time(s)
Inpainter	PowerPaint [ 33 ]	1.08
VLM	InternVL3-14B [ 10 ] / Qwen3VL-32B [ 12 ]	2.18 / 2.36
Segmentation	SAM2 [ 41 ]	0.14
Depth estimator	MoGe2 [ 35 ] / UniDepth2 [ 39 ]	0.39 / 0.37

TABLE IV: Model and inference time (s) measured on an A100 GPU.

Fig. 5: Qualitative pseudo-LiDAR comparison. VERIA against PGT-Aug and Text3DAug using MoGe2 (first row) and UniDepth2 (second row). All visualized instances contain at least 64 points. Despite relying on depth-based reconstruction, VERIA produces beam-pattern-consistent pseudo-LiDAR comparable to mesh-based outputs on nuScenes (32-beam) and Lyft (64-beam). Mesh-based instances can appear visually cleaner than real scans, as they often lack sensor noise and irregular returns; Tab. V provides complementary quantitative evaluation.

Class	Method	FID $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
Constr. Veh.	PGT-Aug	2.96	0.95	0.81	0.87
	Text3dAug	2.54	0.92	0.87	0.89
	VERIA (InternVL3/UniDepth2)	2.15	0.94	0.89	0.91
	VERIA (InternVL3/MoGe2)	2.17	0.94	0.85	0.89
	VERIA (Qwen3VL/UniDepth2)	2.12	0.94	0.87	0.90
	VERIA (Qwen3VL/MoGe2)	2.16	0.94	0.83	0.88

TABLE V: Pseudo-LiDAR instance quality on nuScenes , measured by FID for distributional fidelity and

k

-NN precision and recall for sample-level authenticity and diversity.

Data	VLM	Sem. Verif.	Category CLIP $\uparrow$ [IQR]	Scale CLIP $\uparrow$ [IQR]
nuScenes	Intern VL3	PASS	26.28 [24.30, 27.71]	28.83 [27.09, 30.28]
		FAIL (Category)	24.23 [22.80, 25.54]	27.86 [25.57, 29.75]
		FAIL (Scale)	25.53 [23.02, 27.66]	27.04 [25.49, 28.43]
	Qwen 3VL	PASS	26.29 [24.25, 27.75]	28.84 [27.05, 30.30]
		FAIL (Category)	24.49 [22.91, 25.82]	27.18 [25.59, 28.58]
		FAIL (Scale)	25.14 [22.63, 27.36]	27.10 [24.71, 29.08]

TABLE VI: Image-level CLIPScore ablation by semantic verification.

Dataset	VLM	Sem. Verif.	NIQE $\downarrow$ [IQR]	BRISQUE $\downarrow$ [IQR]
nuScenes	Intern VL3	PASS	3.99 [3.63, 4.39]	28.26 [23.07, 33.79]
	Intern VL3	FAIL (Artifact)	4.10 [3.72, 4.51]	30.00 [24.46, 35.72]
	Qwen 3VL	PASS	3.99 [3.62, 4.39]	28.15 [22.95, 33.69]
	Qwen 3VL	FAIL (Artifact)	4.16 [3.72, 4.72]	30.41 [25.18, 36.42]
Lyft	Intern VL3	PASS	3.85 [3.40, 4.35]	25.15 [20.00, 30.73]
	Intern VL3	FAIL (Artifact)	4.06 [3.53, 4.66]	25.76 [20.36, 31.99]

TABLE VII: NIQE and BRISQUE scores ablation by semantic verification.

Fig. 6: Examples rejected by semantic verification on nuScenes and Lyft. Representative samples filtered by the semantic verifier across categories, with the verifier’s diagnostic comment shown below each image. Rejections include category mismatches, implausible scale or placement, and artifacts.

		Per-class AP (%)
Modality	Method	Constr. Veh.				Motorcycle				Bicycle				mAP (%)
	Sem. Verif.	✓	–	✓	–	✓	–	✓	–	✓	–	✓	–	✓	–	✓	–
	Geo. Verif.	✓	✓	–	–	✓	✓	–	–	✓	✓	–	–	✓	✓	–	–
LiDAR-only (CenterPoint)	InternVL3/MoGe2	24.33	23.14	23.12	22.79	69.71	69.35	67.29	67.29	58.84	57.65	57.47	57.41	63.12	62.96	62.74	62.43
	Qwen3VL/MoGe2	24.82	22.82	22.40	22.10	69.54	69.33	69.06	67.62	58.13	57.48	57.38	55.92	62.90	62.87	62.49	62.39
	InternVL3/UniDepth2	23.17	22.89	22.88	22.27	69.33	69.29	67.89	67.89	59.83	58.53	57.34	56.09	62.99	62.75	62.35	62.48

TABLE VIII: 3D object detection ablation on nuScenes. Augmented-category AP and mAP across semantic and geometric verification configurations, showing how verification choices affect downstream performance.

		FID $\downarrow$
Class	Method	Sem. ✓	Sem. $\times$	Sem. ✓	Sem. $\times$
		Geo. ✓	Geo. ✓	Geo. $\times$	Geo. $\times$
Constr. Veh.	InternVL3/UniDepth2	2.15	2.16	2.18	2.22
	InternVL3/MoGe2	2.17	2.19	2.18	2.32
	Qwen3VL/UniDepth2	2.12	2.19	2.21	2.22
	Qwen3VL/MoGe2	2.16	2.16	2.21	2.31

TABLE IX: FID ablation of point-cloud instances on nuScenes. FID under semantic and geometric verification configurations.

Fig. 7: FID vs. geometric tolerance $\lambda$ on nuScenes. FID is lowest at moderate $\lambda$ and generally increases as $\lambda$ becomes more permissive; overly strict tolerances can also raise FID.

Data Augmentation

Data augmentation expands or transforms training examples to improve learning, data efficiency, and robustness to distribution shifts.

See all 249 papers

Long-Tailed 3D Detection via Multi-Modal Fusion
Dec 18, 2023Yechi Ma, Neehar Peri, Achal Dave +33D Object Detection Long-Tail Autonomous Driving
Department of Computer Science, Zhejiang University. · Computing & Mathematical Sciences, Caltech. · Toyota Research Institute. +2
SemLT3D: Semantic-Guided Expert Distillation for Camera-only Long-Tailed 3D Object Detection
Apr 20, 2026Hao Vo, Khoa Vo, Thinh Phan +5Autonomous Driving Perception 3D Object Detection
AICV Lab, University of Arkansas, USA · University of Utah, USA · University of Houston, USA +1
FOMO-3D: Using Vision Foundation Models for Long-Tailed 3D Object Detection
Mar 9, 2026Anqi Joyce Yang, James Tu, Nikita Dvornik +23D Object Detection Autonomous Driving Perception
Waabi · University of Toronto

Dec 18, 2023J/K move · Enter open · S save

Long-Tailed 3D Detection via Multi-Modal Fusion

Yechi Ma, Neehar Peri, Achal Dave, Wei Hua+2

Department of Computer Science, Zhejiang University. · Computing & Mathematical Sciences, Caltech. · Toyota Research Institute.+2

3D Object Detection Long-Tail Autonomous Driving Multimodal Object Detection Camera-LiDAR Fusion

Open paper AskPDF arXiv

Contemporary autonomous vehicle (AV) benchmarks have significantly advanced multimodal (LiDAR+RGB) 3D detection. However, despite the naturally long-tailed distribution of object classes, existing benchmarking protocols primarily focus on frequent categories (e.g., pedestrian and car), largely overlooking rare but safety-critical classes such as stroller and emergency vehicle. In practice, reliable detection of both common and rare classes is essential for safe autonomous driving. We formalize this problem as Long-Tailed 3D Detection (LT3D), where evaluation encompasses all annotated classes, including rare ones. To address LT3D, we introduce hierarchical losses that promote feature sharing across classes, diagnostic metrics that assign partial credit to semantically reasonable mistakes with respect to the semantic hierarchy (e.g., confusing a child with an adult), and a multimodal late-fusion (MMLF) framework to fuse detections. In particular, we show that rare-class accuracy benefits substantially from MMLF of independently trained uni-modal LiDAR and RGB detectors. Because of the modular design, unlike prevailing end-to-end trained multi-modal detectors that require paired LiDAR-RGB data, MMLF enables the use of advanced unimodal detectors that are trained on large-scale uni-modal datasets with sufficient data for rare classes. Lastly, we examine three fundamental design choices in MMLF, including the RGB detector representation (2D vs. 3D), cross-modal association (3D vs. image plane), and fusion strategy. We find that 2D RGB detectors recognize rare classes more reliably than 3D RGB detectors, image-plane association is more robust to depth estimation errors, and probabilistic score-calibrated fusion consistently yields the best performance. Extensive experiments on nuScenes and Argoverse2 demonstrate substantial improvements of MMLF, establishing a new state of the art.

Abstract

Data Augmentation

Explore similar work

Long-Tailed 3D Detection via Multi-Modal Fusion

SemLT3D: Semantic-Guided Expert Distillation for Camera-only Long-Tailed 3D Object Detection

FOMO-3D: Using Vision Foundation Models for Long-Tailed 3D Object Detection