Modeling and Measuring Redundancy in Multisource Multimodal Data for Autonomous Driving

Yuhan Zhou; Mehri Sattari; Haihua Chen; Kewei Sha

cs.CVMar 6, 2026

Modeling and Measuring Redundancy in Multisource Multimodal Data for Autonomous Driving

Authors: Yuhan Zhou, Mehri Sattari, Haihua Chen, Kewei Sha

Organizations: Dept. of Information Science, University of North Texas, Denton, Texas, USA · Dept. of Data Science, University of North Texas, Denton, Texas, USA

Abstract

Next-generation autonomous vehicles (AVs) rely on large volumes of multisource and multimodal ( $M^2$ ) data to support real-time decision-making. In practice, data quality (DQ) varies across sources and modalities due to environmental conditions and sensor limitations, yet AV research has largely prioritized algorithm design over DQ analysis. This work focuses on redundancy as a fundamental but underexplored DQ issue in AV datasets. Using the nuScenes and Argoverse 2 (AV2) datasets, we model and measure redundancy in multisource camera data and multimodal image-LiDAR data, and evaluate how removing redundant labels affects the YOLOv8 object detection task. Experimental results show that selectively removing redundant multisource image object labels from cameras with shared fields of view improves detection. In nuScenes, mAP ${50}$ gains from $0.66$ to $0.70$ , $0.64$ to $0.67$ , and from $0.53$ to $0.55$ , on three representative overlap regions, while detection on other overlapping camera pairs remains at the baseline even under stronger pruning. In AV2, $4.1$ - $8.6\%$ of labels are removed, and mAP ${50}$ stays near the $0.64$ baseline. Multimodal analysis also reveals substantial redundancy between image and LiDAR data. These findings demonstrate that redundancy is a measurable and actionable DQ factor with direct implications for AV performance. This work highlights the role of redundancy as a data quality factor in AV perception and motivates a data-centric perspective for evaluating and improving AV datasets. Code, data, and implementation details are publicly available at: https://github.com/yhZHOU515/RedundancyAD

Figures & tables

Fig. 1 : Illustration of multisource and multimodal data in autonomous vehicles. LiDAR point clouds, multi-camera images, RADAR, vehicle dynamics, user-related data, and external contextual inputs feed into the AV system. Redundancy arises from overlapping fields of view among multiple onboard cameras (green), as well as from multimodal sensing in which cameras and LiDAR (orange) observe the same objects. The rest colored blocks represent additional data sources and modalities.

Fig. 2 : Illustration of research design, using nuScenes dataset as examples. We first quantify redundancy in multisource and multimodal autonomous driving datasets (RQ1), then design data pruning to remove redundant observations (RQ2), and finally evaluate how redundancy removal affects object detection performance (RQ3).

Fig. 3 : The workflow of the research design on redundancy evaluation. nuScenes emphasizes 2D labeling and AV2 adopts 3D annotations. Beyond dataset-specific configurations, this workflow can be generalized to other AD datasets.

Dataset	Year	Sensors	Annotations	Size	Modalities
nuScenes-mini [ 13 ]	2020	6 cameras (12 Hz), 1 LiDAR, RADAR, GPS/IMU	3D bounding boxes, object attributes, tracking, HD maps	10 scenes, 404 frames	Multisource camera images
nuScenes-in-KITTI [ 63 ]	2021	Cameras, 1 LiDAR, GPS/IMU	2D/3D bounding boxes, calibration, tracking	85 scenes in KITTI format	Image and LiDAR
Argoverse 2 [ 64 ]	2023	9 cameras, 2 LiDARs (10 Hz), GPS/IMU	3D bounding boxes, tracking, map annotations, motion labels	1,000 scenes, 20,000 LiDAR sequences	Multisource camera images

TABLE I : Summary of datasets used for redundancy evaluation

Fig. 4 : Camera settings and overlapping FoVs in nuScenes. For datasets with different sensor layouts, the same redundancy modeling and measurement approach remains applicable by adjusting the geometric parameters.

Fig. 5 : Illustration of cropping based on overlapping angles of the cameras.

Aspect	nuScenes multisource	AV2 multisource
Sensor layout	6 calibrated surround-view cameras with predefined overlapping FoVs.	9 calibrated cameras (7 ring + 2 front stereo); surround-view configuration with denser overlap.
Source of supervision	Object identity inferred implicitly from dataset annotations and temporal alignment.	Explicit object identity via track_uuid from 3D ego-frame cuboids.
Annotation space	2D image-space object-centric observations derived from overlap regions.	3D-first cuboids in a unified ego frame, projected into each camera view to obtain 2D observations.
Redundancy unit	Multiple 2D observations of the same object instance across overlapping cameras.	Multiple projected 2D observations originating from the same 3D object instance across cameras.
Overlap pairing	Fixed camera overlap pairs defined by dataset specification.	Camera overlap pairs computed from calibration (yaw center and horizontal FoV) using angular overlap.
BCS computation	Boundary-aware 2D coverage score computed on valid image regions.	Boundary-aware 2D coverage score computed after projecting 3D cuboids and clipping to image bounds.

TABLE II : Comparison between the redundancy experiments on nuScenes and AV2 datasets. Both apply instance-level, BCS-guided pruning; AV2 adds calibration-driven overlap pairing and 3D-to-2D projection. All the experiments were conducted on a PC equipped with one NVIDIA GeForce 4090 RTX GPU and 24 GB of memory. Model configurations are fixed across experiments.

Fig. 6 : Example of cross-modal redundancy between the front camera (upper) and the top LiDAR (lower) in the nuScenes dataset. Colored lines indicate the corresponding objects observed by both sensors.

Dataset	Experiment	Prec.	Rec.	mAP $50$	mAP $50:95$
nuScenes	Train unpruned, Eval unpruned	0.780	0.630	0.720	0.450
nuScenes	Train pruned ( $\tau_{\mathrm{BCS}}{=}0.8$ on pair 4), Eval unpruned	0.867	0.800	0.770	0.638
AV2	Train unpruned, Eval unpruned	0.815	0.565	0.640	0.447
AV2	Train pruned ( $\tau_{\mathrm{BCS}}{=}0.5$ ), Eval unpruned	0.818	0.557	0.622	0.433

TABLE III : Multisource detection performance with and without BCS-guided pruning

Fig. 7 : Examples of high and low similarity samples in nuScenes.

Fig. 8 : Effects of removing redundant instances in overlapping FoV images on mAP $50$ in nuScenes.

Train	$\tau_{\mathrm{BCS}}$	Del.	Rem.	Tracks	Prec.	Rec.	mAP ${50}$
Baseline	–	0	187,265	95,266	0.815	0.565	0.640
Pruned	0.1	16,056	171,209	95,266	0.811	0.531	0.604
Pruned	0.2	14,345	172,920	95,266	0.788	0.544	0.605
Pruned	0.3	12,810	174,455	95,266	0.826	0.538	0.617
Pruned	0.4	11,018	176,247	95,266	0.823	0.545	0.618
Pruned	0.5	9,442	177,823	95,266	0.818	0.557	0.622

TABLE IV : AV2 multisource BCS pruning sweep. Training uses different

\tau_{\mathrm{BCS}}

values while evaluation is fixed to the same unpruned validation split. “Del.” and “Rem.” denote deleted and remaining camera-level 2D labels in the training set. “Tracks” is the number of unique 3D object tracks observed during training.

Fig. 9 : Redundancy in LiDAR and image data in nuScenes.

Fig. 10 : T-test of distance threshold and cross-modal redundancy in nuScenes.

Fig. 11 : The effects of removing close-range redundant LiDAR data in nuScenes on detection performance.

Autonomous Driving Datasets

Autonomous driving datasets collect and organize driving-scene data to train and evaluate perception, prediction, planning, and safety systems.

See all 141 papers

DriveXQA: Cross-modal Visual Question Answering for Adverse Driving Scene Understanding
Mar 11, 2026Mingzhe Tao, Ruiping Liu, Junwei Zheng +6VLMs for Autonomous Driving Multimodal Large Language Models
Karlsruhe Institute of Technology · Mercedes-Benz Tech Innovation · Hunan University
123D: Unifying Multi-Modal Autonomous Driving Data at Scale
May 8, 2026Daniel Dauner, Valentin Charraut, Bastian Berle +10Autonomous Driving Datasets Autonomous Driving
1KE:SAI · University of Tübingen, Tübingen AI Center · 4Valeo Brain +4
V2X-QA: A Comprehensive Reasoning Dataset and Benchmark for Multimodal Large Language Models in Autonomous Driving Across Ego, Infrastructure, and Cooperative Views
Apr 3, 2026Junwei You, Pei Li, Zhuoyu Jiang +7Autonomous Driving Benchmarks VLMs for Autonomous Driving
Department of Civil and Environmental Engineering, University of Wisconsin–Madison, Madison, WI, 53706, USA · Department of Civil and Architectural Engineering and Construction Management, University of Wyoming, Laramie, WY, 82071, USA · Kuaishou Technology, Beijing, 100085, China +1

Mar 11, 2026J/K move · Enter open · S save

DriveXQA: Cross-modal Visual Question Answering for Adverse Driving Scene Understanding

Mingzhe Tao, Ruiping Liu, Junwei Zheng, Yufan Chen+5

Karlsruhe Institute of Technology · Mercedes-Benz Tech Innovation · Hunan University

VLMs for Autonomous Driving Multimodal Large Language Models Autonomous Driving Datasets Multimodal Sensor Fusion Efficient Multimodal Inference

Open paper AskPDF arXiv

Fusing sensors with complementary modalities is crucial for maintaining a stable and comprehensive understanding of abnormal driving scenes. However, Multimodal Large Language Models (MLLMs) are underexplored for leveraging multi-sensor information to understand adverse driving scenarios in autonomous vehicles. To address this gap, we propose the DriveXQA, a multimodal dataset for autonomous driving VQA. In addition to four visual modalities, five sensor failure cases, and five weather conditions, it includes

102,505

QA pairs categorized into three types: global scene level, allocentric level, and ego-vehicle centric level. Since no existing MLLM framework adopts multiple complementary visual modalities as input, we design MVX-LLM, a token-efficient architecture with a Dual Cross-Attention (DCA) projector that fuses the modalities to alleviate information redundancy. Experiments demonstrate that our DCA achieves improved performance under challenging conditions such as foggy (GPTScore:

53.5

vs.

25.1

for the baseline).

Abstract

Autonomous Driving Datasets

Explore similar work

DriveXQA: Cross-modal Visual Question Answering for Adverse Driving Scene Understanding

123D: Unifying Multi-Modal Autonomous Driving Data at Scale

V2X-QA: A Comprehensive Reasoning Dataset and Benchmark for Multimodal Large Language Models in Autonomous Driving Across Ego, Infrastructure, and Cooperative Views