Spectral-Geometric Neural Fields for Pose-Free LiDAR View Synthesis

Yinuo Jiang; Jun Cheng; Yiran Wang; Cheng Cheng

cs.CVMar 13, 2026

Spectral-Geometric Neural Fields for Pose-Free LiDAR View Synthesis

Authors: Yinuo Jiang, Jun Cheng, Yiran Wang, Cheng Cheng

Organizations: School of AIA, Huazhong University of Science and Technology

Abstract

Neural Radiance Fields (NeRF) have shown remarkable success in image novel view synthesis (NVS), inspiring extensions to LiDAR NVS. However, most methods heavily rely on accurate camera poses for scene reconstruction. The sparsity and textureless nature of LiDAR data also present distinct challenges, leading to geometric holes and discontinuous surfaces. To address these issues, we propose SG-NLF, a pose-free LiDAR NeRF framework that integrates spectral information with geometric consistency. Specifically, we design a hybrid representation based on spectral priors to reconstruct smooth geometry. For pose optimization, we construct a confidence-aware graph based on feature compatibility to achieve global alignment. In addition, an adversarial learning strategy is introduced to enforce cross-frame consistency, thereby enhancing reconstruction quality. Comprehensive experiments demonstrate the effectiveness of our framework, especially in challenging low-frequency scenarios. Compared to previous state-of-the-art methods, SG-NLF improves reconstruction quality and pose accuracy by over 35.8% and 68.8%. Our work can provide a novel perspective for LiDAR view synthesis.

Figures & tables

Figure 1: (a) Reconstruction and synthesis quality. The X-axis shows Chamfer Distance (CD) for point cloud reconstruction on the low-frequency KITTI-360 dataset [ 50 , 26 ] . The Y-axis shows RMSE for range depth and intensity synthesis. Circle area represents inference time. Lower CD and RMSE mean better reconstruction and synthesis. Our framework outperforms prior work by large margins. (b) Pose accuracy. The axis represents Absolute Trajectory Error (ATE) for pose estimation. Our SG-NLF demonstrates higher accuracy compared to existing NeRF-based models.

Figure 2: Geometric Inconsistency [ 39 , 11 , 50 ] . The geometric hole mask is generated by comparing rendered opacity with ground truth LiDAR measurements. White regions in rectangular boxes (first row) highlight areas where these methods fail to reconstruct geometry. Comparisons between second and third rows further show these holes lead to poor-quality synthesized views.

Figure 3: Overview of the SG-NLF. Given multi-view LiDAR sequences $\{\mathcal{S}_{i}\}_{i=0}^{N}$ , SG-NLF employs a hybrid representation that combines discrete geometric encoding with continuous spectral embedding for effective reconstruction. The hybrid features are utilized to construct a confidence-aware graph for global pose optimization. The optimized poses and hybrid features are fed into NeRF to synthesize novel views $\hat{\mathcal{S}}$ . To further enhance cross-frame consistency, SG-NLF distinguishes between the real depth maps (obtained from ground truth point clouds $\mathcal{S}_{ij}$ that are transformed into cross-frame coordinates) and fake depth maps (obtained from synthesized and transformed $\hat{\mathcal{S}}_{ij}$ ).

Method	Point Cloud		Depth					Intensity
Method	CD $\downarrow$	F-score $\uparrow$	RMSE $\downarrow$	MedAE $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	MedAE $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
LiDARsim [ 31 ]	11.0426	0.5975	10.1994	1.3881	0.5588	0.3888	17.9423	0.2089	0.1254	0.6463	0.0824	13.6127
PCGen [ 20 ]	1.0356	0.7862	7.5672	0.7066	0.5333	0.3744	20.6219	0.2139	0.1063	0.5930	0.1070	13.4243
BARF-LN [ 27 , 39 ]	3.1001	0.6156	7.5767	2.0583	0.5779	0.2834	22.5759	0.2121	0.1575	0.7121	0.1468	11.9778
HASH-LN [ 10 , 39 ]	2.6913	0.6082	6.3005	2.1686	0.5176	0.3752	22.6196	0.2404	0.1502	0.6508	0.1602	12.9286
GeoTrans-LN [ 35 , 39 ]	0.5753	0.8116	4.4291	0.2023	0.3896	0.5330	25.6137	0.2709	0.1589	0.5578	0.2578	12.9707

Table 1: Comparisons with state-of-the-art methods on KITTI-360 [ 26 ] dataset with a low-frequency setting [ 50 ] . We compare our method with pose-dependent [ 31 , 20 , 55 ] and pose-free methods [ 27 , 10 , 50 , 35 ] . For approaches not originally designed for LiDAR NVS [ 27 , 10 , 35 ] , we follow GeoNLF [ 50 ] and adopt LiDAR-NeRF [ 39 ] for reconstruction. For pose-dependent methods, we employ ground-truth poses for synthesis. We color the best results as red and the second-best as orange .

Method	Point Cloud		Depth					Intensity
Method	CD $\downarrow$	F-score $\uparrow$	RMSE $\downarrow$	MedAE $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	MedAE $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
LiDARsim [ 31 ]	16.7623	0.4308	12.3483	1.9971	0.3125	0.3889	16.2553	0.0858	0.0355	0.1713	0.2897	21.3989
PCGen [ 20 ]	2.2608	0.6139	12.6586	0.7055	0.2365	0.4389	12.6586	0.0865	0.0235	0.1530	0.3707	21.3643
BARF-LN [ 27 , 39 ]	1.2695	0.7589	8.2414	0.1123	0.1432	0.6856	20.8900	0.3920	0.0144	0.1023	0.6119	26.2330
HASH-LN [ 10 , 39 ]	0.9691	0.8011	7.8353	0.0441	0.1190	0.6543	20.6244	0.0459	0.0135	0.0954	0.6279	26.8870
GeoTrans-LN [ 35 , 39 ]	4.1587	0.2993	10.7899	2.1529	0.1445	0.3671	17.5885	0.0679	0.0256	0.1149	0.3476	23.6211

Table 2: Comparisons with state-of-the-art methods on nuScenes [ 4 ] dataset with a low-frequency setting [ 50 ] . We color the best results as red and the second-best as orange . The notations are consistent with the KITTI-360 [ 26 ] dataset in Table 1 above.

Figure 4: Qualitative comparisons for LiDAR range depth and intensity reconstruction. Both pose-dependent [ 55 , 20 ] and pose-free methods [ 50 ] are compared. Regions with obvious differences are highlighted in the rectangular boxes and arrows.

Method	Point Cloud		Depth					Intensity
Method	CD $\downarrow$	F-score $\uparrow$	RMSE $\downarrow$	MedAE $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	MedAE $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
LiDARsim [ 31 ]	2.2249	0.8667	6.5470	0.0759	0.2289	0.7157	21.7746	0.1532	0.0506	0.2502	0.4479	16.3045
PCGen [ 20 ]	0.2090	0.8597	4.8838	0.1785	0.5210	0.5062	24.3050	0.2005	0.0818	0.6100	0.1248	13.9606
NKSR [ 11 ]	0.5780	0.8685	4.6647	0.0698	0.2295	0.7052	22.5390	0.1565	0.0536	0.2429	0.4200	16.1159
LiDAR-NeRF [ 39 ]	0.0923	0.9226	3.6801	0.0667	0.3523	0.6043	26.7663	0.1557	0.0549	0.4212	0.2768	16.1683
LiDAR4D [ 55 ]	0.0894	0.9264	3.2370	0.0507	0.1313	0.7218	27.8840	0.1343	0.0404	0.2127	0.4698	17.4529

Table 3: Comparisons on the KITTI-360 [ 26 ] dataset with standard-frequency setting [ 39 ] . We compare SG-NLF with state-of-the-art pose-dependent methods [ 31 , 20 , 55 , 11 , 39 ] and pose-free model [ 50 ] . We color the best results as red and the second-best as orange .

Figure 5: Visual comparisons for pose estimation. We transform input sequences into a unified scene using estimated global poses and color-code the composite scene by height. We compare with pose-free methods [ 27 , 50 , 39 ] . Best view zoomed in on-screen for details.

Method	KITTI-360			nuScenes
	ATE	RPE r	RPE t	ATE	RPE r	RPE t
	(m) $\downarrow$	(deg) $\downarrow$	(cm) $\downarrow$	(m) $\downarrow$	(deg) $\downarrow$	(cm) $\downarrow$
ICP [ 3 ]	1.894	1.019	30.383	1.131	0.647	15.410
MICP [ 5 ]	1.483	1.419	35.584	2.519	1.101	38.840
HRegNet [ 30 ]	7.423	9.083	290.160	7.815	2.173	120.913
SGHR [ 40 ]	2.539	0.906	95.576	9.557	0.699	30.383

Table 4: Pose estimation comparisons on KITTI- 360 [ 26 , 50 ] and nuScenes [ 4 , 50 ] . We compare with registration baselines [ 3 , 35 , 30 , 5 , 40 ] and pose-free NeRF methods [ 27 , 10 , 50 , 39 ] .

Panel of Table 5 — Table 5: Hybrid Representation Analysis. Ablation study of geometric encodings (GE) and spectral embeddings (SE) on nuScenes [ 4 ] , with comparisons to GeoNLF [ 50 ] .

Figure 6: Qualitative comparison for hybrid representation.

Panel of Table 6 — Table 6: Efficacy of SG-NLF Components. Ablation on the hybrid spectral-geometric representation (HR), global pose optimization (GP), and cross-frame consistency (CFC) using nuScenes dataset [ 4 ] . We adopt the same baseline as GeoNLF [ 50 ] .

Neural Radiance Fields

Neural radiance fields represent 3D scenes as continuous functions of position and viewing direction to synthesize images from new camera views.

See all 82 papers

Neural LiDAR Bundle Adjustment
Jul 5, 2026Chin Yung Anson Hon, Kaicheng Zhang, Sen Wang3D Mapping Point Cloud Registration
Imperial College London, London, United Kingdom
M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesis
Sep 24, 2026Yang Zhou, Jiuhong Xiao, Shizhao Ye +3Multi-View Image Synthesis Image Generation
New York University, New York, USA · Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94720, USA · U.S. Army Combat Capabilities Development Command, Army Research Laboratory, Adelphi, MD 20783, USA
LiFi: LiDAR Generation from Multi-View Images via Geometric and Semantic Collaborative Guidance
Oct 14, 2026Sizhuo Zhou, Xiaosong Jia, Fanrui Zhang +10Point Cloud Generation Diffusion Model Guidance
University of Science and Technology of China · Shanghai Innovation Institute · Fudan University +1

Jul 5, 2026J/K move · Enter open · S save

Neural LiDAR Bundle Adjustment

Chin Yung Anson Hon, Kaicheng Zhang, Sen Wang

Imperial College London, London, United Kingdom

3D Mapping Point Cloud Registration LiDAR Localization Neural Radiance Fields

Open paper AskPDF arXiv

Recent research has achieved remarkable novel view rendering and scene reconstruction results with Neural Radiance Field (NeRF), including extensions to the LiDAR modality. Few studies have, however, explored the key design differences between RGB NeRFs and LiDAR NeRFs, particularly considering their underlying working principles. In this work, we provide both theoretical and empirical evidence suggesting that the density of volume sampling plays a significant role in LiDAR NeRF. Based on this finding, we propose a novel Neural LiDAR Bundle Adjustment (NeLD-BA) algorithm, which is tailored using efficient volume sampling of LiDAR rays for joint optimization of LiDAR map and poses. Extensive experiments are performed using the Newer College and FusionPortable datasets to demonstrate the proposed NeLD-BA's state-of-the-art performance in multi-view point cloud registration and 3D mapping. We will open-source our code for the community.

Abstract

Neural Radiance Fields

Explore similar work

Neural LiDAR Bundle Adjustment

M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesis

LiFi: LiDAR Generation from Multi-View Images via Geometric and Semantic Collaborative Guidance