Towards Real-Time and Adaptable LiDAR Scene Completion

Azhar Hussian; Martin Vossiek; Vasileios Belagiannis

cs.CVAug 17, 2026

Towards Real-Time and Adaptable LiDAR Scene Completion

Authors: Azhar Hussian, Martin Vossiek, Vasileios Belagiannis

Organizations: Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany

Abstract

LiDAR scene completion is a key component of 3D perception in autonomous driving, where the scene must be completed in real time to be usable in downstream tasks. Existing approaches typically follow an initialize-and-refine paradigm, in which a coarse initialization of the scene is first constructed, then refined into complete 3D geometry. Generative models are slower because they iteratively refine random Gaussian noise into the scene, while non-generative methods perturb the partial scene with a fixed noise scale, which limits coverage of large gaps and occluded regions and requires manual recalibration for each new sensor configuration. We present RapidLiDAR, a LiDAR scene completion method that treats the initialization itself as a learned, data-driven component. We propose an adaptive initialization module that predicts a spatially varying displacement for each partial input point, expanding the partial observations into a coarse scene initialization adapted to the local geometry, without requiring manual noise tuning. To refine this coarse initialization into a complete and coherent scene, we additionally propose a multi-scale reconstruction module that further refines point positions by querying multi-scale 3D voxel and 2D BEV feature maps constructed from the input scan. By replacing point-neighborhood operators such as farthest point sampling and $k$ -nearest neighbor search with voxel- and BEV-based feature extraction, our architecture is faster and can handle different input resolutions by design. Experiments on SemanticKITTI and KITTI-360 show that our method achieves completion performance on par with the state of the art while completing a full scene in 0.1 seconds, which is 2.3 times faster than the fastest prior method. This matches the 10 Hz acquisition rate of typical automotive LiDAR sensors, taking a step toward real-time LiDAR scene completion.

Figures & tables

Figure 1: Initialization matters. Top row: each method’s initialization; bottom row: the corresponding refined output. The highlighted box marks a large unobserved region. (a) LiDiff [ 19 ] starts from Gaussian noise that carries no information about the scene; (b) LiNeXt [ 6 ] perturbs the input with a fixed noise variance, so its points stay near the observed surface and never reach across the gap; (c) Our adaptive module learns data-dependent displacements that populate the region from the surrounding geometry, and this coverage is preserved in both the coarse initialization and the final refined result.

Figure 2: Overview of RapidLiDAR. The Multi-Scale Feature Extraction module ( center ) voxelizes the input point cloud $X\in\mathbb{R}^{M\times 3}$ and extracts multi-scale 3D voxel features $(F_{1},F_{2},\dots,F_{n})$ and a dense 2D BEV feature map $B_{\text{dense}}$ . The Adaptive Initialization Module ( top ) predicts a spatially varying displacement $\Delta$ for each point in $\tilde{P}$ to obtain the initialized scene $P_{\text{init}}$ . The Multi-Scale Reconstruction Module ( bottom ) projects voxel features into BEV feature maps and cross-attends per-point features from $P_{\text{init}}$ with the multi-scale BEV feature maps using multi-scale deformable attention. A final MLP predicts a residual displacement that aligns each point with the underlying target surfaces, producing the completed scene.

Figure 3: Illustration of Dense BEV Head. Converts sparse 3D volumetric features into a dense BEV map. The channel and depth dimensions are merged and projected to

C_{\text{out}}

using a 2D convolution. Multi-head self-attention over BEV tokens captures global scene context, followed by residual 2D convolutions for feature refinement.

	SemanticKITTI			KITTI-360
Method	CD $\downarrow$	JSD 3D $\downarrow$	JSD BEV $\downarrow$	CD $\downarrow$	JSD 3D $\downarrow$	JSD BEV $\downarrow$
LMSCNet	0.641	–	0.431	0.979	–	0.496
LODE	1.029	–	0.451	1.565	–	0.483
MID	0.503	–	0.470	0.637	–	0.476
PVD	1.256	–	0.498	–	–	–
LiDiff	0.434	0.564	0.444	0.564	–	0.459

Table 1: Scene Completion on SemanticKITTI and KITTI-360. Quantitative comparison with prior methods.

\dagger

denotes methods with an additional refinement. Best results in each group are highlighted in bold .

Method	CD $\downarrow$	Param (M) $\downarrow$	Time (s) $\downarrow$
LiDiff	0.434	32.67	30.1
ScoreLiDAR	0.406	32.67	7.1
LiNeXt	0.214	1.99	0.23
Ours	0.206	11.8	0.10

Table 2: Computational Efficiency Comparison. We report Chamfer distance, number of learnable parameters, and inference time per scan.

Method	CD $\downarrow$	JSD 3D $\downarrow$	JSD BEV $\downarrow$
Ours	0.206	0.475	0.332
Ours w/o AIM	0.218	0.488	0.345
Ours w/o MSRM	0.215	0.494	0.342

Table 3: Architectural Ablation Study. We evaluate the impact of our core modules on the SemanticKITTI validation set. Best results are in bold .

$S_{\max}$	CD $\downarrow$
50	0.2594
70	0.2592
100	0.2589

Table 4: Effect of Maximum Displacement Bound. Reconstruction performance for different values of

S_{\max}

, evaluated on a downsampled validation set of 180,000 output points.

$\eta$ (m)	CD $\downarrow$	Param (M)	Time (s) $\downarrow$
0.5	0.214	10.0	0.07
0.4	0.210	11.6	0.09
0.3	0.206	11.8	0.10
0.2	0.208	11.9	0.14

Table 5: Voxel Resolution Ablation. Impact of voxel resolution

\eta

on reconstruction performance, parameter count, and inference time.

Figure 4: Qualitative Comparison on SemanticKITTI. Our method produces more complete geometry in large occluded regions compared to prior methods.

3D Scene Completion

3D scene completion infers missing geometry, occupancy, semantics, and appearance from partial images, point clouds, or depth scans.

See all 41 papers

Jun 2, 2026

PatchScene: Patch-based Voxel Diffusion for Large-Scale Scene Completion

We propose PatchScene, a novel diffusion-based framework for large-scale LiDAR scene completion. Unlike existing methods that rely on global latent representations or dense voxel grids, PatchScene adopts a patch-based voxel diffusion paradigm that explicitly generates fine-grained geometry within localized 3D regions. To ensure coherent reconstruction at both spatial and temporal scales, we introduce a confidence-guided spatio-temporal fusion mechanism that integrates overlapping patches and adjacent frames in a unified generative process. Furthermore, we design an Annular-Flow diffusion strategy that leverages the radial density pattern of LiDAR scans to progressively propagate high-fidelity information from near-range to far-range regions, enabling spatially unbounded scene completion. Extensive experiments on the SemanticKITTI benchmark demonstrate that PatchScene achieves state-of-the-art performance across all standard metrics, surpassing previous approaches in both geometric accuracy and temporal consistency. Remarkably, the model trained on 20 m LiDAR ranges generalizes effectively to 50 m scenes without retraining, highlighting its strong scalability and generalization capability for real-world autonomous driving applications.

Qingdong Xu, Jiajun Zhu, Shilin Zhu +4

1MEGVII Technology · 2Qianli Technology · 4Northeastern University, China +2

3D Reconstruction Autonomous Driving Perception 3D Scene Completion 3D Diffusion Models Diffusion Models

Ask PDF

Jun 14, 2026

PointDiffusion: Diffusion-Based Scene Completion in the Point Cloud Domain

Reconstructing dense 3D scenes from sparse LiDAR point clouds is a fundamental challenge in autonomous driving, where latent diffusion models offer a promising solution. However, existing approaches rely on object-level autoencoders that collapse into unstable global representations at outdoor scale and suffer from ground truth data corrupted by odometry drift that systematically degrades supervision quality. Furthermore, multi-step diffusion inference incurs prohibitive latency for real-time deployment. We propose a novel multi-token Gaussian VAE with cross-attention pooling for stable scene-scale LiDAR compression, combined with an anchor-based ICP ground truth refinement pipeline that eliminates drift-induced noise from training supervision. Together, these components enable a scaffold-free single-step diffusion completion model that achieves an approximately 16x reduction in squared Chamfer distance on SemanticKITTI seq. 08 (0.396 m^2 to 0.024 m^2), surpasses LiDiff and ScoreLiDAR by 17-19% and 10-11%, respectively, and operates at 25-143x lower inference latency. Our results demonstrate that data quality dominates model design in this regime and that multi-token latent spaces provide a stable first stage for latent diffusion-based scene completion.

Chidera Agbasiere, Mikhail Sannikov, Faith Ogunwoye +5

Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology, Moscow, Russia

Point Cloud Completion 3D Reconstruction 3D Scene Completion Diffusion Model Inference Acceleration Diffusion Models

Ask PDF

Aug 27, 2026

Generative Semantic Scene Completion

Outdoor LiDAR semantic scene completion (SSC) recovers a dense semantic voxel grid from a scan observing 1% of the target volume, under class imbalance beyond 7,000x. We recast SSC as generative semantic scene completion (GSSC): a single discrete-diffusion formulation in three roles. First, paired sparse-dense scene synthesis (PS

^3

) generates matched sparse LiDAR observations with their dense semantic completions, addressing the long tail at its source and yielding the PS

^3

-SemanticKITTI corpus we train on alongside SemanticKITTI. Second, semantic-guided generative scene completion (SGSC) generates the scene from noise with multinomial discrete diffusion, conditioned on the sparse scan through a bird's-eye-view semantic map and a sparse 3D feature stream. Third, the same framework instead refines an existing completion in one flow-matching step: structured source discrete diffusion (S

^2

^2

). S

^2

^2

improves the mIoU of SGSC's own output and every external SSC base tested, without base retraining or test-time adaptation. On the strongest base, one step without test-time augmentation reaches 38.8% mIoU on the SemanticKITTI hidden test. To our knowledge that is the best causal, single-sweep, single-sample result on that leaderboard, +2.1 pp over the previous best published score under the same restriction. Four correction steps with eight-view test-time augmentation reach 39.2%, outside that restriction.

Shi Chen, Weifeng Ge

College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA

3D Scene Completion 3D Scene Understanding Diffusion Models

Ask PDF

Abstract

3D Scene Completion

Explore similar work

PatchScene: Patch-based Voxel Diffusion for Large-Scale Scene Completion

PointDiffusion: Diffusion-Based Scene Completion in the Point Cloud Domain

Generative Semantic Scene Completion