RESBev: Making BEV Perception More Robust

Lifeng Zhuo; Kefan Jin; Zhe Liu; Hesheng Wang

cs.CVMar 10, 2026

RESBev: Making BEV Perception More Robust

Authors: Lifeng Zhuo, Kefan Jin, Zhe Liu, Hesheng Wang

Organizations: Shanghai Jiao Tong University

Abstract

Bird's-eye-view (BEV) perception has emerged as a cornerstone of autonomous driving systems, providing a structured, ego-centric representation critical for downstream planning and control. However, real-world deployment faces challenges from sensor degradation and adversarial attacks, which can cause severe perceptual anomalies and ultimately compromise the safety of autonomous driving systems. To address this, we propose a resilient and plug-and-play BEV perception method, RESBev, which can be easily applied to existing BEV perception methods to enhance their robustness to diverse disturbances. Specifically, we reframe perception robustness as a latent semantic prediction problem. A latent world model is constructed to extract spatiotemporal correlations across sequential BEV observations, thereby learning the underlying BEV state transitions to predict clean BEV features for reconstructing corrupted observations. The proposed framework operates at the semantic feature level of the Lift-Splat-Shoot pipeline, enabling recovery that generalizes across both natural disturbances and adversarial attacks without modifying the underlying backbone. Extensive experiments on the nuScenes dataset demonstrate that, with few-shot fine-tuning, RESBev significantly improves the robustness of existing BEV perception models against various external disturbances and adversarial attacks.

Figures & tables

Panel of Figure 1 — Figure 1: Overview and robustness comparison of RESBev. (a) RESBev predicts a BEV prior from historical BEV features and ego-motion, and fuses it with corrupted observations to reconstruct the BEV representation. (b) RESBev improves the resilience of the LSS baseline under natural disturbances and adversarial attacks.

Panel of Figure 2 — Figure 2: Diagnostic analysis for robust BEV recovery. (a) BEV features show stronger temporal stability than perspective-view features. (b) Small feature deviations can still cause severe performance degradation, motivating predictive recovery over simple aggregation.

Stage	Rep.	Feature Form	IoU
Lift	Perspective	$C\times H_{img}\times W_{img}$	16.4
Shoot	Task output	$N_{cls}\times X\times Y$	18.7
Splat	BEV feature	$C\times X\times Y$	31.6

Table 1: Recovery-stage diagnosis. BEV features provide the best recovery performance under noise corruption.

Variant	Hist.	Curr.	Prior	IoU
Single frame	No	Yes	No	15.53
Temporal attn.	Yes	Yes	No	20.17
Recurrent fusion	Yes	Yes	No	22.31
Predictive prior	Yes	No	Yes	30.11

Table 2: Temporal-mechanism diagnosis on the same BEV features. Predicting a temporal prior is more effective than directly aggregating corrupted observations.

Figure 3: The overall architecture of our proposed temporal fusion model. The model consists of two core components: Semantic Prior Predictor that predicts the current BEV state from the past, and Anomaly Reconstructor that fuses this prediction with current fused BEV features.

Corruption	LSS [ 12 ]			SimpleBEV [ 5 ]			GaussianLSS [ 15 ]			BEVFormer [ 9 ]			DiffBEV [ 24 ]
	Vanilla	+RESBev	Change	Vanilla	+RESBev	Change	Vanilla	+RESBev	Change	Vanilla	+RESBev	Change	Vanilla
Clean	33.03	33.31	$\uparrow$ 0.28	55.70	55.91	$\uparrow$ 0.21	42.80	43.14	$\uparrow$ 0.34	46.70	47.01	$\uparrow$ 0.31	38.90
FGSM	10.28	28.42	$\uparrow$ 18.14	13.23	31.09	$\uparrow$ 17.86	9.91	31.43	$\uparrow$ 21.52	21.54	38.78	$\uparrow$ 17.24	25.18
PGD	9.17	31.47	$\uparrow$ 22.30	11.89	31.65	$\uparrow$ 19.76	7.72	31.09	$\uparrow$ 23.37	18.68	38.90	$\uparrow$ 20.22	28.11
Fog	9.93	28.39	$\uparrow$ 18.46	17.18	33.76	$\uparrow$ 16.58	24.26	31.95	$\uparrow$ 7.69	25.34	38.91	$\uparrow$ 13.57	29.02
Bright	9.78	28.38	$\uparrow$ 18.60	17.63	33.79	$\uparrow$ 16.16	24.03	31.87	$\uparrow$ 7.85	25.70	38.74	$\uparrow$ 13.04	30.39

Table 3: Average IoU under seen corruptions across three severity levels. The Change column denotes the absolute IoU improvement of RESBev over the vanilla model.

Table 4: Average NDS under seen and unseen corruptions across three severity levels. BEVDiffuser and RESBev are evaluated on the same BEVFormer and BEVFusion backbones.

Corruption	LSS [ 12 ]			SimpleBEV [ 5 ]			GaussianLSS [ 15 ]			BEVFormer [ 9 ]			DiffBEV [ 24 ]
	Vanilla	+RESBev	Change	Vanilla	+RESBev	Change	Vanilla	+RESBev	Change	Vanilla	+RESBev	Change	Vanilla
Clean	33.03	33.31	$\uparrow$ 0.28	55.70	55.91	$\uparrow$ 0.21	42.80	43.14	$\uparrow$ 0.34	46.70	47.01	$\uparrow$ 0.31	38.90
C&W Attack	8.78	30.80	$\uparrow$ 22.02	11.43	30.61	$\uparrow$ 19.18	5.97	31.24	$\uparrow$ 25.27	17.56	39.67	$\uparrow$ 22.11	29.81
Snow	10.26	28.35	$\uparrow$ 18.09	17.40	33.71	$\uparrow$ 16.31	16.08	32.10	$\uparrow$ 16.02	28.11	38.91	$\uparrow$ 10.80	30.14
Camera Crash	10.08	28.34	$\uparrow$ 18.27	17.34	33.74	$\uparrow$ 16.40	13.41	31.56	$\uparrow$ 18.15	27.46	39.72	$\uparrow$ 12.26	29.42
Noise	8.64	28.27	$\uparrow$ 19.63	17.60	33.76	$\uparrow$ 16.16	16.67	31.43	$\uparrow$ 14.76	21.55	39.34	$\uparrow$ 17.79	29.89

Table 5: Average IoU under unseen corruptions across three severity levels.

Config.	LSS	SBEV	GLSS	BEVFormer
Predictor-only	26.67	30.11	29.16	35.48
Predictor + Reconstructor	29.02	32.80	31.59	38.82
Gain	$\uparrow$ 8.8%	$\uparrow$ 8.9%	$\uparrow$ 8.3%	$\uparrow$ 9.4%

Table 6: Ablation study of RESBev components. SBEV denotes SimpleBEV and GLSS denotes GaussianLSS. All results are mean IoU over five seen corruptions, where each sample is evaluated under a randomly selected corruption type and severity. Gain is computed over the Predictor-only baseline.

Corruption	1-Step	10-Step	30-Step	50-Step	Change
FGSM	28.42	28.58	28.12	28.11	$\downarrow$ 1.09%
PGD	31.47	30.93	30.97	30.84	$\downarrow$ 2.00%
Fog	28.39	28.03	28.00	27.89	$\downarrow$ 1.76%
Bright	28.38	27.91	27.88	27.89	$\downarrow$ 1.72%
Frame Lost	28.33	28.48	28.31	28.43	$\uparrow$ 0.35%

Table 7: Long-horizon robustness under consecutive seen corruptions.

Method	Params (M)	FLOPs (G)	FPS (Hz)	Training Memory (GB)	Inference Memory (GB)
BEVFormer-base	69.1	1311	2.7	28.5	8.4
BEVFormer-base + RESBev	83.8	1552	2.4	32.1	9.8

Table 8: Efficiency and GPU memory usage comparison under FP16 inference.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Representation	Shape
Previous reconstructed BEV	$f^{rec}_{t-1}$	$C\times 200\times 200$
Current corrupted BEV	$f^{corrupt}_{t}$	$C\times 200\times 200$
Ego-motion vector	$a_{t-1}$	$15$
Visual latent	$z^{vis}_{t-1}$	$625\times 256$
Action latent	$z^{act}_{t-1}$	$1\times 256$
Predicted latent state	$z^{pred}_{t}$	$625\times 256$

Appendix

Table 9: Representative feature flow in RESBev.

Corruption	Severity	LSS		SimpleBEV		GaussianLSS		BEVFormer
Corruption	Severity	Vanilla	+RESBev	Vanilla	+RESBev	Vanilla	+RESBev	Vanilla	+RESBev
Clean	-	33.03	33.31	55.70	55.91	42.80	43.14	46.70	47.01
	1	13.87	28.58	17.65	31.03	11.76	31.49	29.10	38.86
	2	10.06	28.47	12.51	31.13	9.43	31.42	21.51	38.79
FGSM	3	6.92	28.21	9.54	31.12	8.53	31.37	14.01	38.69
	Avg.	10.28	28.42	13.23	31.09	9.91	31.43	21.54	38.78

Appendix

Table 10: Performance (IoU) of baseline models with and without RESBev across seen corruptions.

Corruption	Severity	LSS		SimpleBEV		GaussianLSS		BEVFormer
Corruption	Severity	Vanilla	+RESBev	Vanilla	+RESBev	Vanilla	+RESBev	Vanilla	+RESBev
Clean	-	33.03	33.31	55.70	55.91	42.80	43.14	46.70	47.01
	1	11.28	30.94	14.65	30.87	8.02	31.38	24.96	39.71
	2	8.54	30.75	11.67	30.74	6.09	31.21	17.54	39.70
C&W Attack	3	6.51	30.70	7.98	30.22	3.80	31.12	10.18	39.60
	Avg.	8.78	30.80	11.43	30.61	5.97	31.24	17.56	39.67

Appendix

Table 11: Performance (IoU) of baseline models with and without RESBev across unseen corruptions.

Corruption	1-Step	10-Step	30-Step	50-Step	Change
C&W	30.80	30.26	30.30	30.13	$\downarrow$ 2.18%
Snow	28.35	27.86	27.81	27.80	$\downarrow$ 1.94%
Camera Crash	28.34	28.00	27.91	27.79	$\downarrow$ 1.94%
Noise	28.27	27.98	27.81	27.80	$\downarrow$ 1.66%
Dark	28.36	27.93	27.95	27.88	$\downarrow$ 1.69%

Appendix

Table 12: Long-horizon robustness under consecutive unseen corruptions.

Autonomous Driving Perception

Autonomous driving perception enables vehicles to detect and interpret road users, drivable space, and hazards using onboard and shared sensors.

See all 379 papers

Can BEV Perception Gracefully Degrade under Sensor Failures?
May 29, 2026Haifa Zhang, Yijing Wang, Haoyu Wang +2Missing-Modality Learning Multimodal Robustness
Tianjin Key Laboratory of Intelligent Unmanned Swarm Technology and System, School of Electrical and Information Engineering, Tianjin University, Tianjin, China · Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai, China
Sparse Recovery Bird's-Eye View Model Soup for Robust Camera-Based 3D Object Detection
Oct 26, 2026Jinwoo Chung, Sangho An, Hyunsung Park +1Corruption Robustness 3D Object Detection
Kookmin University, Seoul, Republic of Korea · Hyundai Motor Company · Corresponding author: Jangho Kim
Towards Effective and Transferable Physical Camouflage against Multi-View BEV-based 3D Perception in Autonomous Driving
Oct 23, 2026Linye Lyu, Jiawei Zhou, Daojing He +1Adversarial Attacks Physical Adversarial Attacks

May 29, 2026J/K move · Enter open · S save

Can BEV Perception Gracefully Degrade under Sensor Failures?

Haifa Zhang, Yijing Wang, Haoyu Wang, Zheng Li+1

Tianjin Key Laboratory of Intelligent Unmanned Swarm Technology and System, School of Electrical and Information Engineering, Tianjin University, Tianjin, China · Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai, China

Missing-Modality Learning Multimodal Robustness Camera-LiDAR Fusion Autonomous Driving Perception Bird's-Eye-View Representation

Open paper AskPDF arXiv

Despite the remarkable success of multi-modal bird's-eye view (BEV) perception in autonomous driving, current systems exhibit a critical vulnerability: existing fusion mechanisms are highly brittle to sensor corruptions, often causing catastrophic performance degradation. This vulnerability largely stems from the fact that standard fusion frameworks typically integrate multi-modal representations in a static manner, leading to a precipitous performance collapse under missing or corrupted modalities. In contrast, we show that graceful degradation is achievable through active modality reliability assessment. To this end, we present Grace-BEV, a lightweight and plug-and-play framework that enforces active reliability awareness during multi-modal fusion. Instead of relying on computationally expensive cross-modal interactions, Grace-BEV leverages the aligned BEV space to explicitly assess modality trustworthiness via a TrustGate Router and dynamically recalibrate feature integration using the FailSafe Fusion Block. Furthermore, we devise a Three-Phase Training strategy with Modality Dropout to prevent modality dominance and encourage balanced cross-modal learning under unreliable inputs. Extensive experiments on nuScenes-R and nuScenes-C show that Grace-BEV maintains robust performance across diverse corruption settings. Notably, under catastrophic LiDAR failures where standard baselines collapse to 0.0% mean Average Precision (mAP), Grace-BEV restores performance to as high as 34.7% mAP. Moreover, it improves clean accuracy by up to 1.4%, achieving a strong trade-off between robustness and efficiency.

Abstract

Autonomous Driving Perception

Explore similar work

Can BEV Perception Gracefully Degrade under Sensor Failures?

Sparse Recovery Bird's-Eye View Model Soup for Robust Camera-Based 3D Object Detection

Towards Effective and Transferable Physical Camouflage against Multi-View BEV-based 3D Perception in Autonomous Driving