3D Object Detection

Follow this topic Ask about this topic

3D object detection predicts what objects are present and where they are in three-dimensional space, often as labeled bounding boxes. Methods include LiDAR-based point, voxel, and pillar networks; camera systems that lift image features into bird’s-eye view; and fusion of cameras, LiDAR, and radar, sometimes with temporal or cooperative data from multiple vehicles. Researchers evaluate these approaches on driving benchmarks such as KITTI, nuScenes, and Waymo, as well as indoor, medical, aerial, and other specialized datasets. A central challenge is balancing accurate detection of small, distant, or occluded objects with efficient inference, limited labels, and reliability under changes in sensors, weather, and deployment conditions.

Latest papers

181 papers14 papers in the last four weeks, down 30% on the four weeks before. 0.1% of all new papers.

May 30, 2026

Shape-Prior-Based Point Cloud Completion for Single-Stage Fully Sparse 3D Object Detection

Single-stage fully sparse 3D object detectors rely on point clouds data to detect objects in autonomous driving scenarios. However, the sparsity and incompleteness of point clouds significantly limit the performance of 3D object detection. To address this issue, this paper proposes a point clouds completion method specifically designed for single-stage fully sparse detectors. The entire shape-prior-based completion process consists of two consecutive steps. In the first step, we design a novel Instance Selection module, which is capable of identifying point clouds corresponding to foreground objects even when the baseline model does not generate proposals, while effectively ignoring the point clouds of background regions. In the second step, we introduce a novel Alignment-Based Point Completion module, which aligns the point clouds of foreground objects with prototypes in terms of both their centers and orientations. Subsequently, points are selected from the prototype to fill in the missing parts of the foreground object. We evaluated our method on two single-stage fully sparse detectors using the KITTI dataset. The experimental results demonstrate that the proposed method significantly improves the detection performance, confirming its effectiveness and generalizability.

Kaizheng Wang, Mingqian Ji, Jian Yang +1

School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

Point Cloud Completion 3D Reconstruction 3D Object Detection

May 26, 2026

Adaptation-Free Heterogeneous Collaborative Perception with Unseen Agent Configurations

Collaborative perception improves 3D object detection by enabling agents to share complementary observations, but most existing methods assume fixed or known collaborator encoder configurations, limiting deployment in practice. In this work, we consider an open-world setting in which auxiliary agents with unseen configurations may appear after deployment, such as different LiDAR beam counts or encoder architectures. To address this challenge, we propose ALF, a collaborative perception framework that enables zero-adaptation collaboration with unseen agent configurations by lifting lightweight box-level messages into ego-compatible auxiliary features. ALF converts auxiliary box-level messages into pseudo-BEV maps and synthesizes ego-compatible latent features by combining object-centric cues with scene context from the ego feature. On V2X-Real, under a zero-shot evaluation across 64 case studies, ALF outperforms the strongest prior baseline by 35.91% in relative [email protected] while requiring only 120 bytes per agent per frame (approximately 9.6 Kbps bandwidth at 10 Hz).

Hyunchul Bae, Heejin Ahn

School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon, Korea 34141

Zero-Shot Learning Communication-Efficient Collaborative Perception 3D Object Detection V2X Communication Cooperative Perception Heterogeneous Collaborative Perception

May 24, 2026

Neuromorphic LiDAR-based Bird's Eye View Object Detection using Energy-efficient Spiking Neural Networks

Autonomous driving perception demands accurate and efficient processing of three-dimensional sensor data under strict power constraints. Traditional convolutional neural networks achieve strong detection accuracy but are computationally intensive, limiting their suitability for deployment on resource-constrained neuromorphic platforms. Spiking neural networks offer a compelling alternative through event-driven sparse computation, yet their application to complex real-world perception tasks such as three-dimensional object detection remains limited. In this work, we propose an end-to-end spiking encoder-decoder network for object detection in bird's eye view representations of LiDAR point clouds, trained using surrogate gradient backpropagation. We train two variants: a membrane potential variant that reads continuous neuron state at the output stage for maximum accuracy, achieving

92.05

/

87.04

/

86.51

AP at

\mathrm{IoU}\!=\!0.5

(Easy/Moderate/Hard), and, a fully binary spiking variant that operates exclusively on spike trains at every layer for direct neuromorphic deployment. We evaluate four input spike encoding strategies and demonstrate that allowing the network to learn spike representations directly from data outperforms hand-crafted Poisson, latency, and z-axis encoding schemes on the KITTI benchmark, where sequential frames are unavailable and the BEV input is presented repeatedly across timesteps as a proxy for temporal streaming. A block-wise energy analysis demonstrates a

3.33\times

reduction in synaptic operation energy over an equivalent CNN under conservative loop-based operation. Together, these results demonstrate the viability of spiking neural networks for accurate and energy-efficient neuromorphic perception in autonomous driving.

Sambit Mohapatra, Senthil Yogamani, Heinrich Gotzig +1

Valeo, Germany · TU Ilmenau, Germany · Valeo, Ireland

Lidar-Based 3D Object Detection Autonomous Driving Perception Spiking Neural Networks 3D Object Detection Neuromorphic Computing

May 24, 2026

Semantics-Guided Multimodal Masked Autoencoder Pretraining for 3D BEV Object Detection

Accurate 3D bird's-eye view (BEV) object detection is essential for autonomous driving, and depends strongly on effective multimodal representations from complementary sensors such as cameras and LiDAR. Multimodal masked autoencoders have shown strong potential for learning such representations for downstream 3D BEV object detection. However, existing methods typically apply uniform random masking to camera and LiDAR inputs, treating all regions equally, and learn representations only through masked reconstruction. We propose a semantics-guided multimodal masked autoencoder framework that introduces semantic information during pretraining through two separate components: (i) semantics-guided LiDAR voxel masking, which preserves semantically important LiDAR regions more strongly, and (ii) an auxiliary point-wise LiDAR semantic decoder branch that injects semantic guidance in addition to reconstruction. On BEVFusion 3D object detection, our semantics-guided pretraining strategy improves performance on the nuScenes mini validation set compared to the standard UniM2AE baseline: semantics-guided LiDAR voxel masking yields +1.49% mean Average Precision (mAP) and +1.66% nuScenes Detection Score (NDS), while decoder-side point semantic supervision yields +1.39% mAP and +3.22% NDS over the baseline.

Prabuddhi Wariyapperuma, Rajitha de Silva, Marc Hanheide +2

University of Lincoln, Lincoln Centre for Autonomous Systems, Lincoln, UK. · University of Cambridge, Institute for Manufacturing, Department of Engineering, Cambridge, UK.

Lidar-Based 3D Object Detection Masked Autoencoders Multimodal Pretraining Camera-LiDAR Fusion Autonomous Driving Perception 3D Object Detection

May 23, 2026

MR-LiDAR: A Multi-Resolution Roadside LiDAR Benchmark for Perception Diagnostics and Deployment Guidance

LiDAR configuration selection in roadside sensing remains constrained by limited controlled evidence across sensors. To address this gap, we present MR-LiDAR, a controlled cross-configuration benchmark comparing four LiDAR configurations from 16 to 128 beams on dynamic vehicle and vulnerable road user (VRU) targets. Its cross-configuration main set covers matched daytime, nighttime, and vehicle-occlusion conditions, while three deployment and operating condition subsets examine mounting height, target speed, and curved-road geometry. By reducing confounding from scene and trajectory variation, MR-LiDAR links sensor configuration to target point acquisition and algorithmic detection outcomes. We characterize distance-dependent point loss, define effective beam count as the number of vertical beams intersecting the target's angular extent, and introduce a point-based Occlusion Ratio. The results show that 16- and 32-beam configurations degrade earlier with distance, while VRUs exhibit greater point sparsity than motor vehicles. Nominal beam count alone is insufficient to explain roadside scanning capability. In the cross-configuration main set, the 80- and 128-beam configurations achieve comparable results, consistent with their nearly identical downward beam layouts. A uniform 80-beam counterfactual yields only about one third of the effective beam count of the actual non-uniform configuration. Illumination and the tested speed range have little influence on target-point-count patterns, whereas mounting height and near-field road geometry can substantially alter configuration ordering. Finally, an empirical minimal-sufficient configuration matrix combines frame-level geometric sufficiency with matched-box quality and includes threshold sensitivity analysis, providing bounded configuration guidance for the tested conditions.

Shunlai Cui, Peng Cao, Yuan Zhu +5

School of Transportation and Logistics, Southwest Jiaotong University, Chengdu, 610031, China · Department of Electrical and Computer Engineering, Old Dominion University, Norfolk, VA, 23529, USA · School of Transportation and Logistics, Southwest Jiaotong University +5

Benchmark Design 3D Object Detection

May 21, 2026

Scene Reconstruction as Mapping Priors for 3D Detection

In autonomous driving, mapping is critical for motion planning but remains an under-utilized resource for perception tasks such as 3D object detection. Maps can provide robust structural priors of the static environment, helping resolve ambiguities and correct for sensor data sparsity or noise, especially for distant objects or under adverse weather conditions. However, conventional High-Definition (HD) maps are resource-intensive to obtain and maintain, which presents a challenge for efficient, large-scale deployment. In this paper, we propose a scalable solution to systematically leverage mapping to improve 3D detection by overcoming two primary challenges. First, we introduce a pipeline to automatically build dense mapping priors from aggregated sensor data, eliminating the need for human labeling. Second, we design a novel Mapping Priors Augmented 3D Detection (MPA3D) framework to effectively integrate mapping priors with different sensor modalities. Extensive experiments on the Waymo Open Dataset demonstrate that our approach achieves new state-of-the-art results, proving the effectiveness of scalable reconstructed scene priors for enhancing 3D detection.

Yang Fu, Yuliang Zou, Hao Xiang +8

Waymo LLC · UC San Deigo

Autonomous Driving Autonomous Driving Perception 3D Object Detection 3D Scene Representation

May 20, 2026

RCGDet3D: Rethinking 4D Radar-Camera Fusion-based 3D Object Detection with Enhanced Radar Feature Encoding

4D automotive radar is indispensable for autonomous driving due to its low cost and robustness, yet its point cloud sparsity challenges 3D object detection. Existing 4D radar-camera fusion methods focus on complex fusion strategies, trading inference speed for marginal gains. This trade-off hinders real-time deployment due to heavy computation on dense feature maps. In contrast, feature extraction from sparse radar points is less time-consuming but remains under-explored. This work uncovers that simply enhancing radar feature extraction can achieve comparable or even higher performance than elaborate fusion modules, while maintaining real-time performance. Based on this finding, we propose RCGDet3D, which centers on radar feature encoding and simplifies multi-modal fusion. Its encoder inherits from the efficient Gaussian Splatting-based Point Gaussian Encoder (PGE) in RadarGaussianDet3D with two key improvements. First, the Ray-centric PGE (R-PGE) predicts Gaussian attributes in ray-aligned coordinate systems before unifying them to Bird's-Eye View (BEV) space, significantly improving geometric consistency and reducing learning difficulty by decoupling the coordinate transformation from representation learning. Second, a Semantic Injection (SI) module incorporates visual cues from images, producing more geometrically accurate and semantically enriched radar features. Experiments on View-of-Delft (VoD) and TJ4DRadSet show that RCGDet3D outperforms state-of-the-art methods in both accuracy and speed, setting a new benchmark for real-time deployment.

Weiyi Xiong, Bing Zhu

School of Automation Science and Electrical Engineering, Beihang University, Beijing, P.R. China

Camera-Radar Fusion 3D Object Detection Object Detection Real-Time Object Detection

May 19, 2026

Co-Fusion4D: Spatio-temporal Collaborative Fusion for Robust 3D Object Detection

In autonomous driving, 3D object detection is essential for accurate perception and reliable decision-making. However, object motion and ego-motion often induce cross-frame spatiotemporal inconsistencies in BEV-based detectors, leading to temporal BEV feature misalignment and degraded spatiotemporal consistency. To address these challenges, we propose Co-Fusion4D, a unified framework that explicitly preserves cross-frame spatiotemporal consistency and suppresses temporal feature drift. Co-Fusion4D adopts a current-frame-centric strategy, treating the current frame as the primary source of information while selectively incorporating historical frames after spatiotemporal filtering and alignment. This dominant-complementary mechanism effectively mitigates cumulative alignment errors, suppresses noisy feature propagation, and exploits reliable temporal cues for a more consistent BEV representation. In addition, Co-Fusion4D integrates a Dual Attention Fusion (DAF) module to further enhance spatiotemporal feature interaction. DAF jointly leverages intra-frame spatial attention and inter-frame temporal attention to adaptively align and fuse multi-frame features, emphasizing motion-consistent regions while suppressing spurious correlations. By departing from conventional uniform fusion paradigms, this design substantially improves the temporal stability and discriminative capability of BEV representations. Extensive experiments on the nuScenes benchmark demonstrate that Co-Fusion4D achieves state-of-the-art performance, with 74.9% mAP and 75.6% NDS, without relying on test-time augmentation or external data.

Wenxuan Li, Qin Zou, Shoubing Chen +3

Autonomous Driving Perception 3D Object Detection

May 19, 2026

Towards Camera-Robust 3D Localization: Equation-Anchored Tool-Use for MLLMs

3D localization in Multimodal Large Language Models (MLLMs), including 3D object detection and 3D visual grounding, is fundamentally limited by camera intrinsic ambiguity: the same image admits different 3D scenes under different cameras. Existing MLLMs either ignore camera parameters and overfit to a canonical training intrinsic, or retrieve depth and 3D cues from external tools but treat the returned values as reference cues (numerical hints that the model is free to interpret implicitly), both preventing camera information from being deterministically propagated into the prediction. We propose an equation-anchored tool-use framework that re-purposes spatial tools as formula variables. The proposed framework proactively retrieves camera intrinsics and samples multi-point metric depths, writes the pinhole back-projection equation

\hat{X} = (u_c - c_x)\bar{Z}/f_x

explicitly in Chain-of-Thought (CoT), and substitutes tool outputs into the formula before regressing the final 9-DoF bounding box. On both 3D object detection and 3D visual grounding tasks under rescaled camera intrinsics from

0.5\times

to

1.5\times

, our method outperforms RGB-only and tool-augmented baselines, with significant gains where the camera deviates most from the training scale. Code and data will be released.

Xueying Jiang, Wenhao Li, Quanhao Qian +4

1Nanyang Technological University · 2DAMO Academy, Alibaba Group · 3HuPan Lab +1

3D Object Detection 3D VLMs 3D Visual Grounding

May 18, 2026

4DLidarOpen: An Open 4D FMCW Lidar Dataset for Motion-Aware Autonomous Driving

We present 4DLidarOpen, a large-scale open multi-modal dataset for autonomous driving, centered on 4D frequency-modulated continuous-wave (FMCW) Lidar sensing. Unlike conventional time-of-flight Lidar datasets that mainly provide geometric measurements, 4DLidarOpen includes point-wise radial velocity measurements from a forward-facing 4D FMCW Lidar, together with multiple Lidars of different types, including rotating, solid-state, and blind-spot variants, surround-view cameras, and 6-DOF ego-vehicle poses. The dataset was collected in complex urban environments in Beijing and covers dense pedestrian interactions, congested traffic, high-speed driving, and unprotected maneuvers. 4DLidarOpen provides synchronized multi-sensor data and 3D bounding-box annotations with persistent track IDs across five object categories. A hybrid annotation strategy is adopted, where large-scale auto-labeled data support scalable training and human experts refine annotations for the human-annotated training and validation sets. Based on this dataset, we establish benchmarks for 3D object detection, birds-eye view (BEV) segmentation and flow prediction, and motion forecasting with planning. Extensive experiments show that direct velocity measurements from 4D FMCW Lidar provide complementary motion cues for dynamic-scene understanding. Compared with geometric-only sensing, the velocity-aware representation improves motion-related perception and downstream forecasting and planning, especially in scenarios involving vulnerable road users and fast-moving objects. These results indicate that 4D FMCW Lidar is a promising sensing modality for motion-aware autonomous driving. The dataset and evaluation toolkit are publicly released to support research on 4D scene understanding, multi-Lidar fusion, and velocity-aware perception and planning.

Kane Qian, Xin Zhao, Yining Shi +7

1Tsinghua University. · 2Hesai Technology Co., Ltd.

Autonomous Driving Datasets Autonomous Driving Autonomous Driving Perception 3D Object Detection Bird's-Eye-View Segmentation Spatiotemporal Forecasting

May 16, 2026

Coarse Semantic Injection for LLM-Conditioned Structured Indoor Prediction

Large language models (LLMs) have recently been used as structured decoders for indoor understanding from 3D point-token inputs. However, point cloud encoders often under-represent thin structural elements such as doors and windows after voxelization and sparse pooling, and may miss individual furniture instances in cluttered scenes. We propose an interface-preserving semantic augmentation for LLM-conditioned structured decoding. The key idea is to associate semantic evidence with the point-cloud representation, reduce it to a coarse four-group code (furniture, walls, openings, and others), and encode it as an RGBB point interface: red for furniture, green for walls, blue for openings, and black for others, where RGBB denotes four semantic color states represented in three RGB channels rather than an additional fourth channel. This semantic color code is appended to the original raw point attributes before tokenization, so geometry and semantics share the same sparse tokenization path while the downstream language model decoder and output serialization remain unchanged. We further introduce a lightweight routed semantic shift module, with an auxiliary head used only for training-time ratio/budget regularization and analysis, to strengthen semantic cues after sparse pooling. The overall pipeline can use RGB-derived semantic evidence. Under these controlled semantic-source settings, the reported metrics improve across Structured3D, the SpatialLM dataset, and ARKitScenes, especially for opening localization and per-instance furniture detection in cluttered scenes. Ablations clarify the roles of semantic source, color coding, token fusion, and shift injection, while also showing that color/entropy effects remain nontrivial.

Shuliang Zhu, Tomiwa Adey, Jinjia Zhou

3D Scene Prediction Point Cloud Learning 3D Object Detection 3D Scene Understanding

May 15, 2026

3DTMDet: A Dual-Path Synergy Network of Transformer and SSM for 3D Object Detection in Point Clouds

A fundamental challenge in point cloud object detection lies in the conflict between the extreme sparsity of distant points and the need for remote context understanding. The existing methods typically use 1D serialization to expand the receptive field, which inevitably discards already scarce local geometric details and reduces detection of distant and small objects. To address this issue, we propose 3DTMDet, a novel detection network that synergistically combines state space models (Mamba) with Transformers. The core idea is to utilize SSM's linear complexity and advantages in long sequence modeling to effectively capture global interactions between sparse and distant points, while using Transformer modules with local attention to encode fine-grained geometric structures in local point sets, preserving accurate shape information. We propose the 3D Hybrid Mamba Transformer (3DHMT) block, which uses an SSM-Attention-SSM pipeline to balance global context understanding and local detail preservation, effectively alleviating the tension between receptive field enlargement and geometric preservation in remote detection. In addition, we introduced a voxel generation block inspired by LiDAR physics, which diffuses features along the sensor observation direction to reconstruct the complete object structure of occlusion and distant areas. Extensive experiments conducted on the KITTI and ONCE datasets have shown that 3DTMDet outperforms state-of-the-art detectors. The code is available at https://github.com/QiuBingwen/3DTMDet.

Bingwen Qiu, Yuan Liu, Junqi Bai +5

aSchool of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing, 210094, Jiangsu, China · bThe 28th Research Institute of China Electronics Technology Group Corporation, Nanjing, 210007, Jiangsu, China · cCollege of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, Jiangsu, China +2

3D ViTs 3D Object Detection

May 14, 2026

MonoPRIO: Adaptive Prior Conditioning for Unified Monocular 3D Object Detection

Monocular 3D object detection remains challenging because size and depth are underdetermined by single-view evidence, particularly under occlusion, truncation, and scale-depth ambiguity. Although recent methods improve depth and geometric reasoning, metric size remains unstable in unified multi-class settings, where class variability and limited visibility broaden plausible size modes. We propose MonoPRIO, a unified monocular detector that targets this bottleneck through adaptive prior conditioning in the size pathway. MonoPRIO constructs class-aware size prototypes offline, routes each decoder query to a soft mixture prior, applies uncertainty-aware log-space conditioning, and uses Cluster-Aligned Prior (CAP) regularisation on matched positives. On the KITTI test server, among the compared methods reporting all three categories, MonoPRIO achieves the highest Moderate AP for Car, Pedestrian, and Cyclist. In the car-only setting, it obtains the best 3D bounding-box AP across Easy, Moderate, and Hard among the compared methods without additional depth or LiDAR supervision, while requiring lower latency than MonoCLUE under matched measurement conditions. The frozen CLIP RN50 encoder is executed only during offline prior-bank construction and not during detector training or inference. Ablations show that simple prior baselines do not recover the full-model gain and that learned soft routing outperforms uniform and hard routing over the same visual-size bank. Routed injection provides the main improvement; CAP is complementary. MonoPRIO improves over MonoDGP on the nuScenes frontal protocol. The clearest benefits occur in low-data and partially occluded regimes, while atypical geometry or severe visibility loss can cause prior mismatch. Code, trained models, result logs, and reproducibility material are available at https://github.com/leon-davies/MonoPRIO.

Leon Davies, Qinggang Meng, Mohamad Saada +2

Department of Computer Science, Loughborough University, Epinal Way, Loughborough, LE11 3TU, Leicestershire, United Kingdom · European Center for Risk & Resilience Studies, University of Southern Denmark, Degnevej 14, Esbjerg, 6705, Denmark

3D Object Detection Monocular 3D Object Detection

May 14, 2026

Towards Accurate Single Panoramic 3D Detection: A Semantic Gaussian Centric Approach

Three-dimensional object detection in panoramic imagery is crucial for comprehensive scene understanding, yet accurately mapping 2D features to 3D remains a significant challenge. Prevailing methods often project 2D features onto discrete 3D grids, which break geometric continuity and limit representation efficiency. To overcome this limitation, this paper proposes PanoGSDet, a monocular panoramic 3D detection framework built upon continuous semantic 3D Gaussian representations. The proposed framework comprises a panoramic depth estimation component and a semantic Gaussian component. The panoramic depth estimation component extracts the equirectangular semantic and depth features from the monocular panorama input. The semantic Gaussian component includes a semantic Gaussian lifting module that projects spherical features into 3D semantic Gaussians, a semantic Gaussian optimization module that refines these semantic Gaussians, and a Gaussian guided prediction head that generates 3D bounding boxes from optimized Gaussian representations. Extensive experiments on the Structured3D dataset demonstrate that our method significantly outperforms existing methods.

Kanglin Ning, Yiran Zhao, Wenrui Li +3

Harbin Institute of Technology · The Suzhou Research Institute of HIT · The PengChengLab

3D Gaussian Splatting 3D Object Detection Monocular 3D Object Detection

May 13, 2026

SToRe3D: Sparse Token Relevance in ViTs for Efficient Multi-View 3D Object Detection

Vision Transformers (ViTs) enable strong multi-view 3D detection but are limited by high inference latency from dense token and query processing across multiple views and large 3D regions. Existing sparsity methods, designed mainly for 2D vision, prune or merge image tokens but do not extend to full-model sparsity or address 3D object queries. We introduce SToRe3D, a relevance-aligned sparsity framework that jointly selects 2D image tokens and 3D object queries while storing filtered features for reactivation. Mutual 2D-3D relevance heads allocate compute to driving-critical content and preserve other embeddings. Evaluated on nuScenes and our new nuScenes-Relevance benchmark, SToRe3D achieves up to 3x faster inference with marginal accuracy loss, establishing real-time large-scale ViT-based 3D detection while maintaining accuracy on planning-critical agents.

Sandro Papais, Lezhou Feng, Charles Cossette +1

University of Toronto · Zoox Inc.

Multi-View 3D Object Detection Token Pruning Efficient ViTs 3D Object Detection Real-Time Object Detection

May 13, 2026

Generative Texture Diversification of 3D Pedestrians for Robust Autonomous Driving Perception

In recent years, autonomous driving has significantly in creased the demand for high-quality data to train 2D and 3D perception models for safety-critical scenarios. Real world datasets struggle to meet this demand as require ments continuously evolve and large-scale annotated data collection remains costly and time-consuming making syn thetic data a scalable, practical and controllable alterna tive. Pedestrian detection is among the most safety-critical tasks in autonomous driving. In this paper, we propose a simple yet effective method for scaling variability in 3D pedestrian assets for synthetic scene generation. Starting from a single 3D base asset, we generate multiple distinct pedestrian instances by synthesizing diverse facial textures and identity-level appearance variations using StyleGAN2 and automatically mapping them onto 3D meshes. This ap proach enables scalable appearance-level asset diversifica tion without requiring the design of new geometries for each instance. Using the assets, we construct synthetic datasets and study the impact of mixing real and synthetic data for RGB-based object detection. Through complementary ex periments, we analyze geometry-driven distribution shifts in point cloud perception for 3D object detection. Our findings demonstrate that controlled synthetic diversifica tion improves robustness in 2D detection while revealing the sensitivity of 3D perception models to geometric domain gaps. Overall, this work highlights how generative AI en ables scalable, simulation-ready pedestrian diversification through controlled facial texture synthesis, along with the benefits and limitations of cross-domain training strategies in autonomous driving pipelines.

Arka Bhowmick, Enes Ozeren, Ahmed Abdullah +1

1BIT Technology Solutions GmbH, Germany · 2Mannheim University of Applied Sciences, Germany

3D Asset Generation Synthetic Data Generation Autonomous Driving Perception 3D Object Detection Object Detection

May 12, 2026

TriBand-BEV: Real-Time LiDAR-Only 3D Pedestrian Detection via Height-Aware BEV and High-Resolution Feature Fusion

Safe autonomous agents and mobile robots need fast real time 3D perception, especially for vulnerable road users (VRUs) such as pedestrians. We introduce a new bird's eye view (BEV) encoding, which maps the full 3D LiDAR point cloud into a light-weight 2D BEV tensor with three height bands. We explicitly reformulate 3D detection as a 2D detection problem and then reconstruct 3D boxes from the BEV outputs. A single network detects cars, pedestrians, and cyclists in one pass. The backbone uses area attention at deep stages, a hierarchical bidirectional neck over P1 to P4 fuses context and detail, and the head predicts oriented boxes with distribution focal learning for side offsets and a rotated IoU loss. Training applies a small vertical re bin and a mild reflectance jitter in channel space to resist memorization. We use an interquartile range (IQR) filter to remove noisy and outlier LiDAR points during 3D reconstruction. On KITTI dataset, TriBand-BEV attains 58.7/52.6/47.2 pedestrian BEV AP(%) for easy, moderate, and hard at 49 FPS on a single consumer GPU, surpassing Complex-YOLO, with gains of +12.6%, +7.5%, and +3.1%. Qualitative scenes show stable detection under occlusion. The pipeline is compact and ready for real time robotic deployment. Our source code is publicly available on GitHub.

Mohammad Khoshkdahan, Alexey Vinel

Karlsruhe Institute of Technology · Karlsruhe, Germany

Lidar-Based 3D Object Detection Bird's-Eye-View Representation 3D Object Detection Real-Time Object Detection

May 12, 2026

SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions

Multimodal sensor fusion has demonstrated remarkable performance improvements over unimodal approaches in 3D object detection for autonomous vehicles. Typically, existing methods transform multimodal data from independent sensors, such as camera and LiDAR, into a unified bird's-eye view (BEV) representation for fusion. Although effective in ideal conditions, this strategy suffers from substantial performance deterioration when camera or LiDAR data are missing, corrupted, or noisy. To address this vulnerability, we develop a framework-agnostic fusion module for camera and LiDAR data that allows for handling cases when one of the two modalities is missing or corrupted. To demonstrate the effectiveness of our module, we instantiate it in BEVFusion [1], a well-established framework to combine camera and LiDAR data for 3D object detection. By means of quantitative experiments on the MultiCorrupt dataset, we demonstrate that our module achieves favorable performance improvements under scenarios of missing and corrupted modalities, substantially outperforming existing unified representation approaches across a wide range of sensor deterioration scenarios and reaching state-of-the-art performance in scenarios of corrupted modality due to extreme weather conditions and sensor failure.

Markus Essl, Marta Moscati, Mubashir Noman +4

Johannes Kepler University Linz, Austria · MBZUAI, UAE · Macquarie University, Sydney, Australia +1

Multimodal Robustness Camera-LiDAR Fusion Autonomous Driving 3D Object Detection

May 11, 2026

Think as Needed: Geometry-Driven Adaptive Perception for Autonomous Driving

Autonomous driving scenes range from empty highways to dense intersections with dozens of interacting road users, yet current 3D detection models apply a fixed computation budget to every frame, wasting resources on simple scenes while lacking capacity for complex ones. Existing approaches compound this problem: Transformer-based interaction models scale quadratically with the number of detected objects, and frame-by-frame processing causes the system to immediately forget objects the moment they become occluded. We propose Enhanced HOPE, an adaptive perception architecture that measures the geometric complexity of each incoming LiDAR frame using an unsupervised statistical estimator and routes it through a shallow or deep processing path accordingly, requiring no manual scene labels. To keep interaction modeling efficient, we replace quadratic pairwise attention with a linear-time subspace-based network that groups nearby objects into clusters and processes them jointly. The computational savings from these two mechanisms free up resources for a persistent temporal memory module that retains previously detected objects and traffic rules across frames, enabling the system to recall occluded objects seconds after they disappear from view. On the nuScenes and CARLA benchmarks, Enhanced HOPE reduces latency by 38% on simple scenes with no accuracy loss, improves mean Average Precision by 2.7 points on rare long-tail scenarios, and tracks objects through occlusions lasting over 5 seconds, where all tested baselines fail.

Donghyun Kim, Jaehyoung Park

Stony Brook University

Adaptive Inference Lidar-Based 3D Object Detection Autonomous Driving Perception 3D Object Detection Multi-Object Tracking

May 11, 2026

MUSDA: Multi-source Multi-modality Unsupervised Domain Adaptive 3D Object Detection for Autonomous Driving

With the advancement of autonomous driving, numerous annotated multi-modality datasets have become available. This presents an opportunity to develop domain-adaptive 3D object detectors for new environments without relying on labor-intensive manual annotations. However, traditional domain adaptation methods typically focus on a single source domain or a single modality, limiting their effectiveness in multi-source, multi-modality scenarios. In this paper, we propose a novel framework for multi-source, multi-modality unsupervised domain adaptation in 3D object detection for autonomous driving. Given multiple labeled source domains and one unlabeled target domain, our framework first introduces hierarchical spatially-conditioned (HSC) domain classifiers, which jointly align features from both camera and LiDAR modalities at two distinct levels for each source-target domain pair. To effectively leverage information from multiple source domains, we construct a prototype graph between each pair of domains. Based on this, we develop a prototype graph weighted (PGW) multi-source fusion strategy to aggregate predictions from multiple source detection heads. Experimental results on three widely used 3D object detection datasets - Waymo, nuScenes, and Lyft - demonstrate that our proposed framework effectively integrates information across both modalities and source domains, consistently outperforming state-of-the-art methods.

Xiaohu Lu, Hamed Khatounabadi, Hayder Radha

Electrical and Computer Engineering Michigan State University East Lansing, USA

Multi-Source Domain Adaptation Unsupervised Domain Adaptation Camera-LiDAR Fusion Autonomous Driving Perception 3D Object Detection

May 11, 2026

Hyperbolic Distillation: Geometry-Guided Cross-Modal Transfer for Robust 3D Object Detection

Cross-modal knowledge distillation has emerged as an effective strategy for integrating point cloud and image features in 3D perception tasks. However, the modality heterogeneity, spatial misalignment, and the representation crisis of multiple modalities often limit the efficient of these cross-modal distillation methods. To address these limitations in existing approaches, we propose a hyperbolic constrained cross-modal distillation method for multimodal 3D object detection (HGC-Det). The proposed HGC-Det framework includes an image branch and a point cloud branch to extract semantic features from two different modalities. The point cloud branch comprises three core components: a 2D semantic-guided voxel optimization component (SGVO), a hyperbolic geometry constrained cross-modal feature transfer component (HFT), and a feature aggregation-based geometry optimization component (FAGO). Specifically, the SGVO component adaptively refines the spatial representation of the 3D branch by leveraging semantic cues from the image branch, thereby mitigating the issue of inadequate representation fusion. The HFT component exploits the intrinsic geometric properties of hyperbolic space to alleviate semantic loss during the fusion of high-dimensional image features and low-dimensional point cloud features. Finally, the FAGO compensates for potential spatial feature degradation introduced by the 2D semantic-guided voxel optimization component. Extensive experiments on indoor datasets (SUN RGB-D, ARKitScenes) and outdoor datasets (KITTI, nuScenes) demonstrate that our method achieves a better trade-off between detection accuracy and computational cost.

Kanglin Ning, Wenrui Li, Houde Quan +3

Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China · Suzhou Research Institute of HIT

Cross-Modal Knowledge Distillation 3D Object Detection Multimodal Fusion

May 6, 2026

Query2Uncertainty: Robust Uncertainty Quantification and Calibration for 3D Object Detection under Distribution Shift

Reliable uncertainty estimation for 3D object detection is critical for deploying safe autonomous systems, yet modern detectors remain poorly calibrated, especially under distribution shifts. Although post-hoc calibration methods address this issue and provide improved calibration for in-distribution tests, they fail to adapt in distribution-shifted scenarios. In this work, we address this issue and introduce a density-aware calibration method that couples post-hoc calibrators with the feature density of latent object queries from DETR-style 3D object detectors. These queries form a compact, location and class-aware feature, ideal for density estimation, allowing our approach to adjust model confidences in distribution-shift scenarios. By fitting a density estimator on these query features, our approach jointly recalibrates both classification and bounding box regression uncertainties. On both a multi-view camera and LiDAR-based detector, our approach consistently outperforms standard post-hoc methods in both in-distribution and distribution-shifted scenarios. Code available https://tillbeemelmanns.github.io/query2uncertainty/ .

Till Beemelmanns, Alexey Nekrasov, Stefan Vilceanu +4

Institute for Automotive Engineering, RWTH Aachen · Computer Vision Institute, RWTH Aachen

Distribution Shift 3D Object Detection Density Estimation Uncertainty Calibration

May 3, 2026

SimPB++: Simultaneously Detecting 2D and 3D Objects from Multiple Cameras

Simultaneous perception of 2D objects in perspective view and 3D objects in Bird's Eye View (BEV) is challenging for multi-camera autonomous driving. Existing two-stage pipelines use 2D results only as a one-time cue for 3D detection. We propose SimPB++, which simultaneously detects 2D objects in perspective and 3D objects in BEV from multiple cameras. It unifies both tasks into an end-to-end model with a hybrid decoder architecture, coupling multi-view 2D and 3D decoders interactively. Two novel modules enable deep interaction: Dynamic Query Allocation adaptively assigns 2D queries to 3D candidates, and Adaptive Query Aggregation refines 3D representations using multi-view 2D features, forming a cyclic 3D-2D-3D refinement. For multi-view 2D detection, we use Query-group Attention for intra-group communication. We also design a Crop-and-Scale strategy for long-range perception and a Propagating Denoising strategy with an auxiliary RoI detector. SimPB++ supports mixed supervision with 2D-only and fully annotated data, reducing reliance on expensive 3D labels. Experiments show state-of-the-art performance on nuScenes for both tasks and strong long-range detection (up to 150m) on Argoverse2.

Yingqi Tang, Zhaotie Meng, Erkang Cheng +1

Nullmax · Westlake University

Autonomous Driving Perception Multi-View 3D Object Detection 3D Object Detection Object Detection

May 3, 2026

AFFormer: Adaptive Feature Fusion Transformer for V2X Cooperative Perception under Channel Impairments

Accurate 3D object detection is essential for ensuring the safety of autonomous vehicles. Cooperative perception, which leverages vehicle-to-everything (V2X) communication to share perceptual data, enhances detection but is vulnerable to channel impairments, such as noise, fading, and interference. To strengthen the reliability of intelligent transportation systems, this work improves the robustness of V2X cooperative perception under communication conditions that reflect common channel impairments. This paper proposes an Adaptive Feature Fusion Transformer (AFFormer), a Transformer-based framework that mitigates the adverse effects of corrupted features by modeling temporal, inter-agent, and spatial correlations. AFFormer introduces three key modules: Multi-Agent and Temporal Aggregation for context-aware fusion across agents and over time, Dual Spatial Attention for efficient modeling of spatial dependencies, and Uncertainty-Guided Fusion for entropy-driven refinement of fused features. A teacher-student knowledge distillation strategy further enhances robustness by aligning fused features with reliable early-collaboration supervision. AFFormer is validated on the V2XSet and DAIR-V2X datasets, where it consistently outperforms existing methods under both ideal and impaired communication conditions, demonstrating improved robustness to communication-induced feature degradation while maintaining a competitive efficiency-accuracy trade-off.

Xi Zhou, Tao Huang, Qing-Long Han +2

College of Science and Engineering, James Cook University, Smithfield, QLD 4878, Australia · School of Engineering, Swinburne University of Technology, Melbourne, VIC 3122, Australia · Transport for NSW, Sydney, NSW, Australia

3D Object Detection V2X Communication Cooperative Perception

May 1, 2026

Robust Fusion of Object-Level V2X for Learned 3D Object Detection

Perception for automated driving is largely based on onboard environmental sensors, such as cameras and radar, which are cost-effective but limited by line-of-sight and field-of-view constraints. These inherent limitations may cause onboard perception to fail under occlusions or poor visibility conditions. In parallel, cooperative awareness via vehicle-to-everything (V2X) communication is becoming increasingly available, enabling vehicles and infrastructure to share their own state as object-level information that complements onboard perception. In this work, we study how such V2X information can be integrated into 3D object detection and how robust the resulting system is to realistic V2X imperfections. Using the nuScenes dataset, we emulate object-level cooperative awareness messages from ground truth, injecting controlled noise and object dropout to mimic real-world conditions such as latency, localization errors, and low V2X penetration rates. We convert these messages into a dedicated bird's-eye view (BEV) input and fuse them into a BEVFusion-style detector. Our results demonstrate that while object-level cooperative information can substantially improve detection performance, achieving an NDS of 0.80 under favorable conditions, models trained on idealized data become fragile and over-reliant on V2X. Conversely, our proposed noise-aware training strategy, coupled with explicit confidence encoding, enhances robustness, maintaining performance gains even under severe noise and reduced V2X penetration.

Lukas Ostendorf, Lennart Reiher, Onn Haran +1

Institute for Automotive Engineering, RWTH Aachen University, 52074 Aachen, Germany · Qualcomm Technologies, Inc., Kfar-Netter, Israel

3D Object Detection Cooperative Perception

Apr 28, 2026

SWAN: World-Aware Adaptive Multimodal Networks for Runtime Variations

Multimodal deep neural networks deployed in realistic environments must contend with runtime variations: changes in modality quality, overall input complexity, and available platform resources. Current networks struggle with such fluctuations -- adaptive networks cannot adhere to a strict compute budget, controller-based networks neglect to consider input complexity, and statically provisioned networks fail at all the above. Consequently, they do not extract maximum utility from the expended computational resources. We present SWAN (Sample and World-Aware Multimodal Network), the first adaptive multimodal network that accomplishes all three goals. SWAN employs a quality-aware controller to assign resources among modalities according to a variable user-specified maximum budget. Within this budget, an adaptive gating module further optimizes efficiency by scaling layer utilization according to sample complexity. For further gains, SWAN also employs a token dropping module that masks semantically irrelevant multimodal features before performing detections. We evaluate SWAN in the domain of autonomous driving with complex multi-object 3D detection, reducing FLOPs by up to 49% with minimal degradation.

Jason Wu, Shir-Kang Scott Jin, Yuyang Yuan +4

University of California, Los Angeles · University of California, Riverside · U.S. Army DEVCOM Army Research Laboratory

Efficient Multimodal Inference Dynamic Neural Networks 3D Object Detection Multimodal Object Detection

Apr 28, 2026

Control Your Queries: Heterogeneous Query Interaction for Camera-Radar Fusion

In autonomous driving, camera-radar fusion offers complementary sensing and low deployment cost. Existing methods perform fusion through input mixing, feature map mixing, or query-based feature sampling. We propose a new fusion paradigm, termed heterogeneous query interaction, and present ConFusion, a camera-radar 3D object detector. ConFusion combines image queries, radar queries, and learnable world queries distributed in 3D space to improve query initialization and object coverage. To encourage cross-type interaction among heterogeneous queries, we introduce heterogeneous query mixing (QMix), which performs dedicated cross-type attention after feature sampling to consolidate complementary object evidence. We further propose interactive query swap sampling (QSwap), which improves feature sampling by allowing related queries to exchange informative feature tokens under attention and geometric constraints. Experiments on the nuScenes dataset show that ConFusion achieves state-of-the-art performance, reaching 59.1 mAP and 65.6 NDS on the validation set, and 61.6 mAP and 67.9 NDS on the test set.

Jialong Wu, Yihan Wang, Matthias Rottmann

1Osnabrück University · 3Aptiv Services Deutschland GmbH · University of Wuppertal

Camera-Radar Fusion Autonomous Driving Perception 3D Object Detection Multimodal Fusion Feature Fusion

Apr 28, 2026

Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Tracking

Camera-based 3D object detection and tracking are central to autonomous driving, yet precise 3D object localization remains fundamentally constrained by depth ambiguity when no expensive, depth-rich online LiDAR is available at inference. In many deployments, however, vehicles repeatedly traverse the same environments, making static point cloud maps from prior traversals a practical source of geometric priors. We propose DualViewMapDet, a camera-only inference framework that retrieves such map priors online and leverages them to mitigate the absence of a LiDAR sensor during deployment. The key idea is a dual-space camera-map fusion strategy that avoids one-sided view conversion. Specifically, we (i) project the map into perspective view (PV) and encode multi-channel geometric cues to enrich image features and support BEV lifting, and (ii) encode the map directly in bird's-eye view (BEV) with a sparse voxel backbone and fuse it with lifted camera features in a shared metric space. Extensive evaluations on nuScenes and Argoverse 2 demonstrate consistent improvements over strong camera-only baselines, with particularly strong gains in object localization. Ablations further validate the contributions of PV/BEV fusion and prior-map coverage. We make the code and pre-trained models available at https://dualviewmapdet.cs.uni-freiburg.de .

Markus Käppeler, Özgün Çiçek, Yakov Miron +1

Department of Computer Science, University of Freiburg, Germany. · Bosch Research, Robert Bosch GmbH, Renningen, Germany. · University of Haifa, Israel

Autonomous Driving Perception 3D Object Detection Multi-Object Tracking

Apr 27, 2026

BifDet: A 3D Bifurcation Detection Dataset for Airway-Tree Modeling

Thoracic Computed Tomography (CT) scans offer detailed insights into the intricate branching network of the airway tree, which is essential for understanding various respiratory diseases. Airway bifurcations, where airway branches split, are crucial landmarks for understanding lung physiology, disease mechanisms and lesion localization. Despite the significance of bifurcation analysis, a notable lack of datasets annotated for this task hinders the development of advanced automated specialized detection or segmentation tools. In this paper, we introduce BifDet, the first publicly-available dataset specialized for 3D airway bifurcation detection, filling a critical gap in existing resources. Our dataset comprises carefully annotated CT scans from the ATM22 open-access cohort with bifurcation bounding boxes covering the parent and daughter branches. As a use-case for demonstrating the potential of BifDet, we fine-tune and evaluate RetinaNet and DETR for 3D airway bifurcations detection on CT scans. We provide detailed pipelines, including preprocessing steps and specific implementation design choices. Results are detailed over various categories of minimal bounding box sizes to serve as baseline to benchmark future research.

Ali Keshavarzi, Quentin Bouniot, Benjamin M. Smith +1

LTCI, Telecom Paris, Insitut Polytechnique de Paris, Palaiseau, France · Department of Medicine, Columbia University, New York, USA

3D Object Detection Medical Image Benchmarks

Apr 27, 2026

WildLIFT: Lifting monocular drone video to 3D for species-agnostic wildlife monitoring

Monocular RGB cameras mounted on drones are widely used for wildlife monitoring, yet most analytical pipelines remain confined to two-dimensional image space, leaving geometric information in video underexploited. We present WildLIFT, a computational framework that integrates three-dimensional scene geometry from monocular drone video with open-vocabulary 2D instance segmentation to enable species-agnostic 3D detection and tracking. Oriented 3D bounding box labels with semantic face information enable quantitative assessment of viewpoint coverage and inter-animal occlusion, producing structured metadata for downstream ecological analyses. We validate the framework on 2,581 manually curated frames comprising over 6,700 3D detections across four large mammal species. WildLIFT maintains high identity consistency in multi-animal scenes and substantially reduces manual 3D annotation effort through keyframe-based refinement. By transforming standard drone footage into structured 3D and viewpoint-aware representations, WildLIFT extends the analytical utility of aerial wildlife datasets for behavioural research and population monitoring.

Vandita Shukla, Fabio Remondino, Blair Costelloe +1

13D Optical Metrology, Fondazione Bruno Kessler, Trento, Italy. · Computer Vision and Machine Learning Systems Group, University of Muenster, Muenster, Germany. · Department of Collective Behaviour, Max Planck Institute of Animal Behaviour, Konstanz, Germany. +2

3D Object Detection Drone-Based Wildlife Monitoring 3D MOT Wildlife Monitoring Multi-Object Tracking