cs.CVApr 2, 2026

Safety-Aligned 3D Object Detection: Single-Vehicle, Cooperative, and End-to-End Perspectives

Authors: Brian Hsuan-Cheng Liao, Chih-Hong Cheng, Hasan Esen, Alois Knoll

Organizations: DENSO AUTOMOTIVE Deutschland GmbH, 85386 Eching, Germany. · Department of Computer Engineering, Technical University of Munich, 85748 Garching, Germany. · Department of Computer Science, Carl von Ossietzky University of Oldenburg, 26129 Oldenburg, Germany.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Perception plays a central role in connected and autonomous vehicles (CAVs), underpinning not only conventional modular driving stacks, but also cooperative perception systems and recent end-to-end driving models. While deep learning has greatly improved perception performance, its statistical nature makes perfect predictions difficult to attain. Meanwhile, standard training objectives and evaluation benchmarks treat all perception errors equally, even though only a subset is safety-critical. In this paper, we investigate safety-aligned evaluation and optimization for 3D object detection that explicitly characterize high-impact errors. Building on our previously proposed safety-oriented metric, NDS-USC, and safety-aware loss function, EC-IoU, we make three contributions. First, we present an expanded study of single-vehicle 3D object detection models across diverse neural network architectures and sensing modalities, showing that gains under standard metrics such as mAP and NDS may not translate to safety-oriented criteria represented by NDS-USC. With EC-IoU, we reaffirm the benefit of safety-aware fine-tuning for improving safety-critical detection performance. Second, we conduct an ego-centric, safety-oriented evaluation of AV-infrastructure cooperative object detection models, underscoring its superiority over vehicle-only models and demonstrating a safety impact analysis that illustrates the potential contribution of cooperative models to "Vision Zero." Third, we integrate EC-IoU into SparseDrive and show that safety-aware perception hardening can reduce collision rate by nearly 30% and improve system-level safety directly in an end-to-end perception-to-planning framework. Overall, our results indicate that safety-aligned perception evaluation and optimization offer a practical path toward enhancing CAV safety across single-vehicle, cooperative, and end-to-end autonomy settings.

Figures & tables

Fig. 1: Asymmetry in the safety consequences of perception errors: The red prediction overestimates the truck’s longitudinal distance and under-covers it relative to the blue prediction, making it more safety-critical. Whereas standard metrics such as IoU and translation error (TE) assign identical values to both predictions, our safety-aligned metrics—USC and EC-IoU—favor the blue prediction as the safer one.
Fig. 2: USC applies perspective-view (PV) and bird’s-eye-view (BEV) projections to enforce two safety-relevant constraints. Adapted from Fig. 2 in [ 5 ] © 2024 IEEE.
MetricCorrelation
mAP0.699
NDS0.806
mAUSC0.814
NDS-USC0.925
TABLE I: Absolute Pearson correlation coefficients between model-level metrics and simulated system-level collision rate. Higher values indicate stronger correlation with collision risk.
Fig. 3: IoU and EC-IoU evaluation (with different α\alpha values) as a prediction moves over the ground truth along the x-axis. The ego vehicle is assumed at x=0x=0 , the ground truth GG is centered at x=10x=10 , and the blue box denotes an example prediction PP centered at x=7x=7 . Adapted from Fig. 3 in [ 6 ] © 2024 IEEE.
ModelKey Contributions
FCOS3D [ 30 ]Projects 3D coordinates onto 2D feature maps and adapts a fully convolutional one-stage detector for 3D box regression.
PGD [ 19 ]Extends FCOS3D with probabilistic depth estimation and geometric relation graphs, enabling uncertainty-aware depth refinement for more accurate localization.
DETR3D [ 31 ]Adopts a Transformer architecture in which sparse 3D queries retrieve encoded 2D image features and decode them directly into 3D bounding boxes.
PETR [ 32 ]Improves DETR3D by injecting 3D coordinate information into 2D image features, yielding 3D position-aware feature maps for more accurate localization.
PointPillars [ 33 ]Converts point clouds into pseudo-images using vertical pillars, enabling efficient processing by CNN backbones.
SSN [ 34 ]Extends PointPillars by incorporating explicit shape information from point clouds through an additional shape-aware loss.
TABLE II: Representative models included in our benchmark and their key contributions.
Fig. 4: Benchmarking results of representative 3D object detectors on nuScenes, labeled by the sensor modality (C: camera, L: lidar, and F: fusion). Top: true-positive error measures (mAIoU, mATE ′ , mAUSC). Bottom: overall metrics (mAP, NDS, NDS-USC). Safety-oriented metrics tolerate certain non-critical errors but emphasize safety-critical mislocalizations, changing the performance picture compared with standard metrics.
ModelTP measures ↑\uparrowOverall (%) ↑\uparrow
IoUEC-IoUTE ′USCmAPNDSNDS-USC
PGD0.4040.3970.3740.80146.8648.2964.19
w/ EC-IoU-20.4190.4080.4080.81747.3448.8765.28
w/ EC-IoU-40.4170.4050.3810.81947.6248.8265.36
PETR0.3890.3600.3400.76152.8352.0764.10
w/ EC-IoU-20.3930.3630.3470.77053.2552.4064.68
TABLE III: Fine-tuning PGD and PETR with standard and EC-IoU losses (higher is better).
Fig. 5: Qualitative comparison of PETR fine-tuned with the standard loss (red) and our safety-aware EC-IoU loss (blue). EC-IoU shifts predictions toward the ego vehicle, improving coverage of safety-critical regions.
Fig. 6: TUMTraf cooperative perception benchmark [ 8 ] . Top: three roadside camera views and the ego-vehicle camera view. Bottom: Overview of the intersection covered by vehicle and infrastructure sensors. Adapted from [ 8 ] © 2024 IEEE.
DomainMod.RV-mAPEV-mAPEC-mAP
VehicleCam.16.8528.5630.98
Vehiclelidar25.1256.9855.09
VehicleFusion34.9061.9260.94
Coop.Cam.33.0054.1143.33
Coop.lidar38.5367.4955.76
Coop.Fusion46.3571.1168.44
TABLE IV: CoopDet3D performance ( %\% ) under roadside-view, ego-view, and ego-centric safety-oriented evaluation on TUMTraf [ 8 ] .
Fig. 7: Top: vehicle-only lidar-based model. Bottom: cooperative lidar-based model. The cooperative model reduces false positives, but slightly overestimates the lead vehicle’s longitudinal distance from the ego perspective. In the bird’s-eye view, the roadside lidar point cloud is shown in red, while the ego vehicle appears on the right in white and is moving leftward.
StageCollisions/year
HDVs only10.95
HDVs + AVs5.48
HDVs + AVs + CP2.67
TABLE V: Estimated annual collisions at an intersection with human-driven vehicles (HDVs) only, mixed traffic with HDVs and AVs, and mixed traffic with AVs supported by cooperative perception (CP).
Fig. 8: SparseDrive architecture [ 10 ] , featuring sparse query-based perception and parallel motion prediction and planning. We inject EC-IoU into the detection loss while keeping the remaining modules and losses unchanged. Adapted from Fig. 2 in [ 10 ] © 2025 IEEE.
ModelDetection ↑\uparrowTrack. ↑\uparrowPred. ↓\downarrowPlanning ↓\downarrow
NDS (%)IoU (%)EC-IoU (%)AMOTA (%)ADE (m)Col. (%)L2 (m)
UniAD [ 28 ]49.80––35.900.710.6100.73
SparseDrive [ 10 ]52.4343.9742.4636.970.620.1110.59
w/ EC-IoU-251.6843.6743.1536.470.630.0860.60
w/ EC-IoU-451.4643.5143.2536.230.630.0780.60
TABLE VI: End-to-end driving performance with standard ℓ1\ell_{1} loss or EC-IoU loss applied to SparseDrive fine-tuning. UniAD is included for reference. Performance is evaluated for detection, tracking, prediction, and planning. Lower is better for ADE, Col., and L2.
Fig. 9: Qualitative comparison of baseline and EC-IoU-optimized SparseDrive variants. In the same cut-in scenario as Fig. 5 , the EC-IoU-based model decelerates earlier and accelerates more decisively, more closely matching the human reference.
More in this topic

3D Object Detection

3D object detection identifies objects and estimates their locations, sizes, and orientations from images, point clouds, radar, or fused sensors.

See all 181 papers

Explore similar work

CardsList
Oct 6, 2026

Sparse2comm: Towards Robust Cooperative 3D Object Detection

Cooperative perception improves autonomous driving by sharing complementary observations among vehicles and roadside infrastructure for 3D object detection. However, practical deployment is constrained by limited bandwidth and unreliable cooperation, where packet loss, transmission delay, and spatial misalignment jointly degrade the cooperative feature stream. Existing methods often reduce communication cost or compensate for one degradation type, leaving coupled disturbances insufficiently addressed. To address this problem, we propose Sparse2comm, a bandwidth-efficient and robust cooperative 3D object detection framework that treats unreliable cooperation as progressive restoration over degraded cooperative features. Sparse Feature Encoding first encodes communication as randomly mask-sampled foreground features transmitted by collaborating agents, from which the ego vehicle reconstructs dense semantic representations. This sparse-to-dense mechanism learns to infer missing object-centric content from sparse observations, enabling ultra-low-bandwidth communication and packet-loss recovery within the same representation. On the semantically restored features, Latency-Aware Alignment predicts motion flow to compensate delayed messages, and Self-Calibrating Fusion estimates residual spatial offsets in a self-supervised manner before adaptive cross-agent fusion. Sparse2comm therefore restores semantic completeness, temporal consistency, and spatial alignment in an ordered pipeline. Extensive experiments on DAIR-V2X, OpenV2V, and V2V4Real show that Sparse2comm maintains competitive clean accuracy and consistently improves robustness under individual and mixed real-world degradations. Compared with the selective feature communication baseline Where2comm, Sparse2comm improves mixed-setting [email protected]/[email protected] by +20.15/+11.79, +12.66/+11.07, and +15.36/+12.61 on the three datasets, respectively.

Lei Yang, Boqi Li, Chunmian Lin +6

University of Hong Kong, China

Communication-Efficient Collaborative PerceptionAutonomous Driving Perception3D Object DetectionCooperative Perception
AskPDF
Jul 21, 2026

CoGoal3D: Collaborative 3D Object Detection with 3D-Aware Fusion and Refinement

V2X collaborative object detection features overcoming the limitations of single-vehicle systems by aggregating environmental features from multiple collaborative agents. However, existing mainstream V2X perception methods mainly focus on 2D BEV object detection. When 3D detection task is concerned, inferior results are obtained because they ignore the 3D spatial misalignment caused by differing height and attitude among the collaborators. In this paper, we propose a novel collaborative 3D object detection framework called CoGoal3D, which extracts and refines the 3D feature gradually in a two-stage pipeline. In the first stage, a multiscale 3D-aware global fusion module is designed to mitigate the 3D spatial misalignment. The resulting proposals are then refined in the second stage with an auxiliary task of 3D point reconstruction. An effective multi-agent collaborative data augmentation strategy is further proposed to enrich the training data while minimizing information loss. Extensive experiments on public real-world datasets demonstrate that our CoGoal3D achieves new state-of-the-art performance, with 3D [email protected] improvements of 10.86%, 10.34%, and 10.18% on the DAIR-V2X, V2V4Real, and V2X-Real datasets, respectively. Code is available at https://github.com/Megalo-f/CoGoal3D.

Zhihao Yang, Zhiyu Xiang, Peng Xu +5

Zhejiang University, Hangzhou, China · Zhejiang Provincial Key Laboratory of Multi-Modal Communication Networks and Intelligent Information Processing, Hangzhou, China · China Jiliang University, Hangzhou, China

Point Cloud Surface Reconstruction3D Object DetectionCooperative PerceptionFeature Fusion
AskPDF
May 1, 2026

Robust Fusion of Object-Level V2X for Learned 3D Object Detection

Perception for automated driving is largely based on onboard environmental sensors, such as cameras and radar, which are cost-effective but limited by line-of-sight and field-of-view constraints. These inherent limitations may cause onboard perception to fail under occlusions or poor visibility conditions. In parallel, cooperative awareness via vehicle-to-everything (V2X) communication is becoming increasingly available, enabling vehicles and infrastructure to share their own state as object-level information that complements onboard perception. In this work, we study how such V2X information can be integrated into 3D object detection and how robust the resulting system is to realistic V2X imperfections. Using the nuScenes dataset, we emulate object-level cooperative awareness messages from ground truth, injecting controlled noise and object dropout to mimic real-world conditions such as latency, localization errors, and low V2X penetration rates. We convert these messages into a dedicated bird's-eye view (BEV) input and fuse them into a BEVFusion-style detector. Our results demonstrate that while object-level cooperative information can substantially improve detection performance, achieving an NDS of 0.80 under favorable conditions, models trained on idealized data become fragile and over-reliant on V2X. Conversely, our proposed noise-aware training strategy, coupled with explicit confidence encoding, enhances robustness, maintaining performance gains even under severe noise and reduced V2X penetration.

Lukas Ostendorf, Lennart Reiher, Onn Haran +1

Institute for Automotive Engineering, RWTH Aachen University, 52074 Aachen, Germany · Qualcomm Technologies, Inc., Kfar-Netter, Israel

3D Object DetectionCooperative Perception
AskPDF