The Operator Mismatch Problem: Deploying BEV Perception with Portable GPU Compute

Rohit Verma; Anand V Bodas

cs.AIOct 8, 2026

The Operator Mismatch Problem: Deploying BEV Perception with Portable GPU Compute

Authors: Rohit Verma, Anand V Bodas

Organizations: Intel

Abstract

Modern autonomous driving systems rely on bird's-eye-view (BEV) perception models that fuse camera and LiDAR inputs to detect objects in 3D space. These models are accurate, but they cannot be deployed through standard inference runtimes. The reason is an operator mismatch between dense convolutions (which runtimes handle well), sparse 3D convolutions (which runtimes cannot represent), and geometric scatter operations (which runtimes have no vocabulary for). Today, every sparse convolution library is CUDA-only and PyTorch-coupled, locking BEV deployment to a single vendor's hardware and a single execution framework. We present BEVPIPE, a framework for deploying multimodal BEV perception pipelines using portable GPU compute APIs and integrating them with production inference runtimes. BEVPIPE partitions the model into runtime-managed dense subgraphs and three external operator extensions (voxelizer, sparse encoder, BEV projector), connected through a shared GPU memory space. BEVPIPE achieves a 19.5x end-to-end speedup over conventional deployments while retaining 98.5% of reference mAP. We also showcase that BEVPIPE is portable across different GPU backends.

Figures & tables

Figure 1: System architecture of BEVPIPE .

Stage	% Pipeline
Camera branch (INT8 backbone + BEV proj.)	19.87%
Fusion + detection (FP16)	23.65%
Total runtime-managed	43.52%

Table 1: Dense operator latency breakdown. Values are means over 81 steady-state samples on the 96-EU integrated GPU. “% Pipeline” is the fraction of end-to-end latency.

Phase	% Encoder
Precomputation (CPU hash + neighbor map)	27.29%
Submanifold convolution (GPU kernel)	4.61%
Strided convolution (GPU kernel)	4.04%
Sparse-to-BEV scatter	11.15%
GPU dispatch + sync overhead	52.91%
Total	100%

Table 2: Sparse encoder internal phase breakdown. Values are means over 81 steady-state samples on the 96-EU integrated GPU. “% Encoder” is the fraction of total sparse encoder latency.

Operator	% Pipeline
Voxelization (scatter)	0.934
Voxelization (compact)	0.198
BEV projection (scatter)	1.824
BEV projection (convert)	1.474

Table 3: Geometric operator kernel latencies as % of total pipeline time.

Stage	Fine	Medium	BEVPIPE (Coarse)
LiDAR branch (voxelize + sparse enc.)	$19.2\times$	$24.7\times$	$29.5\times$
Camera branch (backbone + depth + proj.)	$2.6\times$	$5.2\times$	$7.9\times$
Fusion + detection	$2.8\times$	$3.9\times$	$5.6\times$
Total	$10.1\times$	$14.9\times$	$19.5\times$

Table 4: End-to-end pipeline performance of all competing approaches relative to Conventional . All values show the latency speedup compared to Conventional .

Backend	Device	mAP	Max deviation
PyTorch + CUDA	NVIDIA GPU	0.585	—
OpenCL	Intel iGPU	0.576	—
SYCL	Intel iGPU	0.578	$<10^{-4}$
SYCL	NVIDIA GPU	0.578	$<10^{-4}$

Table 5: Functional portability across GPU backends with detection accuracy. mAP is computed on the same 81-sample subset. Max deviation is the maximum absolute difference in predicted output tensors relative to the OpenCL reference.

Design Decision	Relative Effect
Depth-threshold sparsity	$3.0\times$ faster
Int32 atomic accumulation	$12.7\times$ faster
Async queue dispatch	$1.35\times$ faster
CPU–GPU overlap	$1.18\times$ faster

Table 6: Ablation of system-level design decisions. Each row disables one optimization and reports the relative regression (

>1\!\times

means the enabled design is faster).

Explore similar work

FlashBEV: Fast and Memory-Efficient Exact BEV Transformation with IO-Awareness
Jul 11, 2026Shunsuke Yokokawa, Hironori KasaharaGPU Kernel Optimization Autonomous Driving Perception
Waseda University, Tokyo, Japan · T2, Inc., Tokyo, Japan
Fast-BEV++: Fast by Algorithm, Deployable by Design
Dec 9, 2025Yuanpeng Chen, Hui Song, Sheng Yang +5Autonomous Driving Perception Bird's-Eye-View Representation
iMotion Automotive Technology (Suzhou) Co., Ltd · School of Data Science, Fudan University · Independent Researcher
Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints
Jun 23, 2026Hojun Choi, Seulbin Hwang, Dae Jung Kim +3VLMs for Autonomous Driving Open-Vocabulary Semantic Segmentation
KAIST AI, South Korea · NAVER LABS, South Korea

Jul 11, 2026, cs.CVJ/K move · Enter open · S save

FlashBEV: Fast and Memory-Efficient Exact BEV Transformation with IO-Awareness

Shunsuke Yokokawa, Hironori Kasahara

Waseda University, Tokyo, Japan · T2, Inc., Tokyo, Japan

GPU Kernel Optimization Autonomous Driving Perception Bird's-Eye-View Representation

Open paper AskPDF arXiv

Bird's-eye-view (BEV) perception is a core component of camera-based 3D understanding in autonomous driving, where view transformation (VT) maps multi-camera image features into a unified BEV representation. Sampling-based view transformation (Sampling-VT) is attractive because it supports dense and continuous BEV aggregation for high-resolution and long-range perception. Its deployment bottleneck, however, is systems-level: standard tensorized implementations of Sampling-VT -- which we refer to as Tensorized Sampling-VT -- explicitly materialize large height-dependent intermediate tensors, causing memory and latency costs that scale poorly with vertical resolution and the number of cameras. We revisit Tensorized Sampling-VT from an operator-execution perspective and show that it follows a gather-reduction pattern: each BEV query independently accumulates contributions across cameras and height bins, enabling thread-local accumulation with on-the-fly recomputation that eliminates the need to materialize height- and camera-dependent intermediates. Based on this insight, we propose FlashBEV, a fully fused and IO-aware execution strategy mathematically equivalent to Tensorized Sampling-VT (same operator output) while substantially reducing global memory traffic and kernel-launch overhead. Experiments show that FlashBEV achieves more than an order of magnitude lower peak GPU memory and significant inference-latency speedups, with memory effectively independent of the number of height bins, reducing the operator's peak memory to O(BCXY) (output only). This unlocks higher BEV range/resolution and vertical discretization within fixed deployment budgets on memory-constrained devices. Our contribution is an execution redesign -- same math, different execution -- that removes a key scalability barrier for deployment-ready Sampling-VT. Code available at https://github.com/yokosyun/FlashBEV

Abstract

Explore similar work

FlashBEV: Fast and Memory-Efficient Exact BEV Transformation with IO-Awareness

Fast-BEV++: Fast by Algorithm, Deployable by Design

Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints