AsyncBEV: Cross-modal Flow Alignment in Asynchronous 3D Object Detection

Shiming Wang; Holger Caesar; Liangliang Nan; Julian F. P. Kooij

cs.CVJan 19, 2026

AsyncBEV: Cross-modal Flow Alignment in Asynchronous 3D Object Detection

Authors: Shiming Wang, Holger Caesar, Liangliang Nan, Julian F. P. Kooij

Organizations: Intelligent Vehicles Group, Delft University of Technology · 3D Geoinformation Group, Delft University of Technology

Abstract

In autonomous driving, multi-modal perception tasks like 3D object detection typically rely on well-synchronized sensors, both at training and inference. However, despite the use of hardware- or software-based synchronization algorithms, perfect synchrony is rarely guaranteed: Sensors may operate at different frequencies, and real-world factors such as network latency, hardware failures, or processing bottlenecks often introduce time offsets between sensors. Such asynchrony degrades perception performance, especially for dynamic objects. To address this challenge, we propose AsyncBEV, a trainable, lightweight and generic module to improve the robustness of 3D Bird's Eye View (BEV) object detection models against sensor asynchrony. Inspired by scene flow estimation, AsyncBEV predicts a dense 2D flow field directly from asynchronous multi-modal BEV features, taking into account the known time offset between these sensor measurements. The predicted feature flow is then used to warp and spatially align the feature maps, which we show can easily be integrated into different current BEV detector architectures (e.g., BEV grid-based and token-based). Extensive experiments demonstrate AsyncBEV improves robustness against both small and large asynchrony between LiDAR or camera sensors in both the token-based CMT and grid-based UniBEV, especially for dynamic objects. We significantly outperform the ego motion compensated CMT and UniBEV baselines, notably by $16.6$ % and $11.9$ % NDS on dynamic objects in the worst-case scenario of a $0.5 s$ time offset. Code is available at https://github.com/tudelft-iv/AsyncBEV.

Figures & tables

Figure 1: Multi-modal 3D object detection under sensor asynchrony. a) shows a general concept of asynchronous 3D object detection. b) shows multi-modal 3D object detection under sensor asynchrony of a pretrained CMT ( ? ). Predictions exhibit large translation errors. c) shows our AsyncBEV improves CMT’s robustness against sensor asynchrony and mostly corrects the spatial misalignment of predicted boxes.

Figure 2: Overview of AsyncBEV . a) demonstrates an example pipeline of asynchronous multi-modal 3D object detection. In this setup, images are sampled at the reference timestamp $t_{1}$ , while LiDAR point clouds were obtained at an earlier timestamp $t_{0}$ . b) shows the $\Delta$ -BEVFlow estimation, the core module of AsyncBEV. It takes asynchronous BEV features as input and predicts the velocity of each BEV cell. The BEV flow is calculated by multiplying the velocity by $\Delta t$ and supervised by the ground truth dense flow representation. Finally, the predicted flow is used to spatially align the features from $t_{0}$ to $t_{1}$ with the detector-specific warper.

Figure 3: Detector-specific Warper. a) Token-based methods rely on 3D coordinates of tokens to provide spatial guidance. Generated

\Delta

-BEVFlow is used to correct the corresponding 3D coordinates of tokens from the asynchronous sensor. b) Grid-based methods project the multi-modal features into predefined BEV grids. Generated

\Delta

-BEVFlow is used to build a look-up table from the reference timestamp to the asynchronous timestamp to warp the asynchronous BEV features to the reference timestamp.

		All Objects			Static Objects			Dynamic Objects
Method		$0\>s$	$0.25\>s$	$0.5\>s$	$0\>s$	$0.25\>s$	$0.5\>s$	$0\>s$	$0.25\>s$	$0.5\>s$	FPS $\uparrow$
	NDS $\uparrow$	72.9	49.6	43.2	69.0	48.7	44.2	47.5	32.2	26.1
CMT	mAP $\uparrow$	70.3	35.9	24.8	61.0	32.2	22.8	36.8	16.4	9.0	6.7
	NDS $\uparrow$	72.9	66.8	63.3	69.0	68.3	67.5	47.5	34.5	26.8
CMT + EMC	mAP $\uparrow$	70.3	60.6	54.1	61.0	59.8	58.6	36.8	21.1	11.9	6.7
	NDS $\uparrow$	71.5 $\downarrow$ 1.4	70.2 $\uparrow$ 3.4	67.6 $\uparrow$ 4.3	68.2 $\downarrow$ 0.8	67.2 $\downarrow$ 1.1	65.4 $\downarrow$ 2.1	45.8 $\downarrow$ 1.7	44.9 $\uparrow$ 10.4	41.5 $\uparrow$ 14.7

Table 1: Evaluation on the nuScenes val set of asynchronous multi modal 3D object detection ( Best , Second Best ). LiDAR is asynchronous. AsyncBEV is only trained once with each model and evaluated on different time offsets. FD: freeze the whole detector and only train AsyncBEV. Performance deltas are measured relative to the EMC baseline. Inference speed in Frames Per Second (FPS) is measured with a single A100 GPU.

Figure 4: Performance of different methods with increasing time offsets under LiDAR asynchrony.

Figure 5: Qualitative Results of AsyncBEV. The left two columns demonstrate the 3D object detection performance of CMT and CMT+AsyncBEV under $0.5s$ LiDAR asynchrony. The right two columns show the output of our predicted $\Delta$ -BEVFlow and the ground truth $\Delta$ -BEVFlow.

$\Delta$ -BEVFlow design			All Objects ( $0\>s$ )		All Objects ( $0.5\>s$ )
EMC+DA	Motion	Velocity	NDS	mAP	NDS	mAP
✗	✗	✗	66.7	62.0	39.7	22.2
✓	✗	✗	63.1 $\downarrow$ 3.6	57.6 $\downarrow$ 4.4	61.1 $\uparrow$ 21.4	54.9 $\uparrow$ 32.7
✓	✓	✗	64.2 $\downarrow$ 2.5	58.6 $\downarrow$ 3.4	62.5 $\uparrow$ 22.8	56.1 $\uparrow$ 33.9
✓	✗	✓	65.7 $\downarrow$ 1.0	60.8 $\downarrow$ 1.2	63.3 $\uparrow$ 23.6	57.1 $\uparrow$ 34.9

Table 2: Ablation study of

\Delta

-BEVFlow design. This table showcases the core design choice for the new task

\Delta

-BEVFlow. EMC+DA : EMC + asynchronous data augmentation training. Motion : BEV-ME. Velocity : BEV-VE, which is our default design.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

		All Objects			Static Objects			Dynamic Objects
Method		$0\>s$	$0.25\>s$	$0.5\>s$	$0\>s$	$0.25\>s$	$0.5\>s$	$0\>s$	$0.25\>s$	$0.5\>s$
	NDS $\uparrow$	72.9	70.7	67.5	69.0	66.9	63.8	47.5	46.9	45.8
CMT	mAP $\uparrow$	70.3	66.7	61.2	61.0	57.9	52.4	36.8	36.0	34.2
	NDS $\uparrow$	72.9	71.4	68.8	69.0	67.6	65.1	47.5	46.9	45.9
CMT + EMC	mAP $\uparrow$	70.3	67.8	63.5	61.0	58.9	54.9	36.8	36.0	34.1
	NDS $\uparrow$	72.7 $\downarrow$ 0.2	72.7 $\uparrow$ 1.3	72.3 $\uparrow$ 3.5	68.8 $\downarrow$ 0.2	68.8 $\uparrow$ 1.2	68.4 $\uparrow$ 3.3	48.1 $\uparrow$ 0.6	48.1 $\uparrow$ 1.2	48.0 $\uparrow$ 2.1

Appendix

Table III: Evaluation on the nuScenes val set of asynchronous multi modal 3D object detection. ( Best , Second Best ). Camera is asynchronous. Trained with one model and evaluated on different time offsets. Performance deltas are measured relative to the EMC baseline.

	All Objects		Static Objects		Dynamic Objects
Method	NDS $\uparrow$	mAP $\uparrow$	NDS $\uparrow$	mAP $\uparrow$	NDS $\uparrow$	mAP $\uparrow$
CMT ( $0.5\>s$ L)	43.2	25.8	44.2	22.8	26.1	9.0
CMT_C	44.7 $\uparrow$ 1.5	38.3 $\uparrow$ 12.5	47.4 $\uparrow$ 3.2	29.5 $\uparrow$ 6.7	28.3 $\uparrow$ 2.2	14.7 $\uparrow$ 5.7
CMT+AsyncBEV ( $0.5\>s$ L)	70.0 $\uparrow$ 25.3	66.4 $\uparrow$ 28.1	67.6 $\uparrow$ 20.2	59.1 $\uparrow$ 29.6	43.4 $\uparrow$ 15.1	32.3 $\uparrow$ 17.6
UniBEV ( $0.5\>s$ L)	39.7	22.2	41.2	19.0	25.3	8.9
UniBEV_C	42.6 $\uparrow$ 2.9	35.4 $\uparrow$ 13.2	46.1 $\uparrow$ 4.9	27.1 $\uparrow$ 8.1	28.2 $\uparrow$ 2.9	13.1 $\uparrow$ 4.2

Appendix

Table IV: Performance of single-modality performance of CMT and UniBEV.

Figure VI: Qualitative Results of AsyncBEV. The left two columns demonstrate the 3D object detection performance of CMT and CMT+ AsyncBEV under LiDAR asynchrony with $0.5\>s$ offset. The right two columns show the output of our BEV-VE module and the ground truth flow in the BEV space.

		All Objects ( $0.5s$ )		Dynamic Objects ( $0.5s$ )
Loss	MM	NDS	mAP	NDS	mAP
✗	✗	63.0	56.8	37.0	25.4
✓	✗	63.2 $\uparrow$ 0.2	57.0 $\uparrow$ 0.2	37.6 $\uparrow$ 0.6	26.0 $\uparrow$ 0.6
✓	✓	63.3 $\uparrow$ 0.3	57.1 $\uparrow$ 0.3	37.8 $\uparrow$ 0.8	26.6 $\uparrow$ 1.2

Appendix

Table V: Ablation of BEV-VE inputs. MM: Multi-modal features, L: Flow loss.

	All Objects ( $0.5\>s$ )		Dynamic Objects( $0.5\>s$ )
SimpleBEV	NDS	mAP	NDS	mAP
✗	62.7	56.1	37.3	25.7
✓	63.3 $\uparrow$ 0.6	57.1 $\uparrow$ 1.0	37.8 $\uparrow$ 0.5	26.6 $\uparrow$ 0.9

Appendix

Table VI: Ablation of SimpleBEV encoder

Figure VII: Performance of CMT, CMT+EMC and CMT+AsyncBEV for different time offsets under LiDAR asynchrony.

Module	# Params	$\times$ AsyncBEV
Camera branch	70, 572, 864	232
LiDAR branch	6, 975, 184	23
AsyncBEV	304, 114	1

Appendix

Table VII: # Params of CMT+AsyncBEV’s modules.

\times

AsyncBEV is the parameter ratio of each module relative to AsyncBEV.

Autonomous Driving Perception

Autonomous driving perception enables vehicles to detect and interpret road users, drivable space, and hazards using onboard and shared sensors.

See all 373 papers

SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions
May 12, 2026Markus Essl, Marta Moscati, Mubashir Noman +4Multimodal Robustness Camera-LiDAR Fusion
Johannes Kepler University Linz, Austria · MBZUAI, UAE · Macquarie University, Sydney, Australia +1
Co-Fusion4D: Spatio-temporal Collaborative Fusion for Robust 3D Object Detection
May 19, 2026Wenxuan Li, Qin Zou, Shoubing Chen +3Autonomous Driving Perception 3D Object Detection
Semantics-Guided Multimodal Masked Autoencoder Pretraining for 3D BEV Object Detection
May 24, 2026Prabuddhi Wariyapperuma, Rajitha de Silva, Marc Hanheide +2Lidar-Based 3D Object Detection Masked Autoencoders
University of Lincoln, Lincoln Centre for Autonomous Systems, Lincoln, UK. · University of Cambridge, Institute for Manufacturing, Department of Engineering, Cambridge, UK.

May 12, 2026J/K move · Enter open · S save

SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions

Markus Essl, Marta Moscati, Mubashir Noman, Muhammad Zaigham Zaheer+3

Johannes Kepler University Linz, Austria · MBZUAI, UAE · Macquarie University, Sydney, Australia+1

Multimodal Robustness Camera-LiDAR Fusion Autonomous Driving 3D Object Detection

Open paper AskPDF arXiv

Multimodal sensor fusion has demonstrated remarkable performance improvements over unimodal approaches in 3D object detection for autonomous vehicles. Typically, existing methods transform multimodal data from independent sensors, such as camera and LiDAR, into a unified bird's-eye view (BEV) representation for fusion. Although effective in ideal conditions, this strategy suffers from substantial performance deterioration when camera or LiDAR data are missing, corrupted, or noisy. To address this vulnerability, we develop a framework-agnostic fusion module for camera and LiDAR data that allows for handling cases when one of the two modalities is missing or corrupted. To demonstrate the effectiveness of our module, we instantiate it in BEVFusion [1], a well-established framework to combine camera and LiDAR data for 3D object detection. By means of quantitative experiments on the MultiCorrupt dataset, we demonstrate that our module achieves favorable performance improvements under scenarios of missing and corrupted modalities, substantially outperforming existing unified representation approaches across a wide range of sensor deterioration scenarios and reaching state-of-the-art performance in scenarios of corrupted modality due to extreme weather conditions and sensor failure.

Abstract

Autonomous Driving Perception

Explore similar work

SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions

Co-Fusion4D: Spatio-temporal Collaborative Fusion for Robust 3D Object Detection

Semantics-Guided Multimodal Masked Autoencoder Pretraining for 3D BEV Object Detection