LiFi: LiDAR Generation from Multi-View Images via Geometric and Semantic Collaborative Guidance

Sizhuo Zhou; Xiaosong Jia; Fanrui Zhang; Qifeng Li; Junjie Li; Zirui Wang; Yukang Feng; Yu Hong

cs.CVOct 14, 2026

LiFi: LiDAR Generation from Multi-View Images via Geometric and Semantic Collaborative Guidance

Authors: Sizhuo Zhou, Xiaosong Jia, Fanrui Zhang, Qifeng Li, Junjie Li, Zirui Wang, Yukang Feng, Yu Hong,

Organizations: University of Science and Technology of China · Shanghai Innovation Institute · Fudan University · Shanghai Jiao Tong University

Abstract

Generating LiDAR point clouds from multi-view camera images is a valuable task with applications in controllable data synthesis, cross-modal simulation and various perception tasks. However, most existing methods are not specifically designed for this task, and the quality of their generated LiDAR remains limited. Moreover, these methods typically focus only on semantic cues in images or generate LiDAR from a monocular camera image. In this work, we introduce LiFi (LiDAR From images), a dedicated framework for generating visually and geometrically aligned LiDAR scenes from multi-view camera images. LiFi guides the latent diffusion process through two complementary branches: geometry and semantics. The geometry branch constructs scene features from images through a reverse sampling algorithm and an uncertainty-aware geometric encoder combined with DepthAnything3, whereas the semantic branch provides high-level semantic representations via a cross-view interaction module. We further introduce a dual-branch balanced classifier-free guidance (CFG) strategy, which enhances the model's conditional generation capability while preserving the independent learning and collaborative controllability of the two branches. Extensive experiments demonstrate that LiFi outperforms state-of-the-art methods in generating high-fidelity LiDAR scenes. The code will be publicly available at: https://github.com/szzhou88/LiFi.

Figures & tables

Figure 1: Overview of the LiFi framework. The framework takes real multi-view camera images as input and is capable of generating 3D LiDAR scenes with high fidelity that are consistent with the visual information. LiFi mainly comprises the following key designs: (i) geometry–semantic decoupled conditioning branches; (ii) a dual-branch balanced classifier-free guidance module; and (iii) a latent diffusion network integrated with Transformer blocks.

Figure 2: Geometry-Aware Branch extracts structured geometric features from raw images via a range-view-centered reverse sampling algorithm and an uncertainty-modulated geometric encoder, thereby guiding the latent diffusion process.

Figure 3: Schematic illustration of the reverse sampling mechanism.

Figure 4: Cross-View Semantic Branch first aggregates multi-view camera tokens augmented with position encodings, then extracts cross-view features through a semantic encoder with a self-attention mechanism, and finally injects them into the latent diffusion process via Transformer blocks.

Figure 5: Visualization results of LiFi and several baseline methods for LiDAR scene generation. It can be observed that our method generates details that are closest to the real LiDAR point cloud data.

Figure 6: Qualitative results of LiFi and several major baseline models on the KITTI-360 dataset.

	Method	Venue	MMD $\downarrow$	JSD $\downarrow$	FRD $\downarrow$	FPD $\downarrow$
Uncond.	LiDARGen [ 57 ]	ECCV’22	55.14	27.69	–	–
	RangeLDM [ 14 ]	ECCV’24	72.01	8.43	1172.06	441.16
	LiDM [ 40 ]	CVPR’24	4.57	10.61	2358.15	420.36
Cond.	UniScene [ 24 ]	CVPR’25	107.18	12.35	1636.60	230.89
	OpenDWM [ 34 ]	CVPR’25	10.75	7.69	2607.81	146.30
	LiDARCrafter [ 28 ]	AAAI’26	24.69	7.37	1183.11	36.17

Table 1: The main quantitative results of LiFi and several baseline methods on the nuScenes dataset. The MMD values are reported in

10^{-5}

and JSD in

10^{-2}

Figure 7: Qualitative results of LiDAR scene generation from camera images. The first row shows the full scenes, and the second row presents zoomed-in local details.

Method	Venue	MMD $\downarrow$	JSD $\downarrow$	FRD $\downarrow$	FPD $\downarrow$
LiDARGen [ 57 ]	ECCV’22	13.36	13.67	2415.23	102.80
RangeLDM [ 14 ]	ECCV’24	4.31	9.42	1362.25	68.22
LiDM [ 40 ]	CVPR’24	9.31	19.00	2894.65	253.91
OpenDWM [ 34 ]	CVPR’25	12.93	14.63	1966.36	436.17
LiFi	–	1.99	8.77	1107.21	21.40

Table 2: The main quantitative results of LiFi and several baseline methods on the KITTI-360 dataset. The MMD values are reported in

10^{-4}

and JSD in

10^{-2}

Method	Venue	MMD $\downarrow$	JSD $\downarrow$	CD $\downarrow$	EMD $\downarrow$	DAS $\downarrow$	Time
LiDM [ 40 ]	CVPR’24	8.02	9.97	4.57	99.71	2.17	0.34
X-Drive [ 51 ]	ICLR’25	48.58	10.54	4.73	179.62	1.52	1.80
LiFi	–	2.78	3.66	0.83	50.08	1.02	0.52

Table 3: Quantitative results on the nuScenes dataset. These baseline models are used for generating LiDAR scenes from camera images. The MMD values are reported in

10^{-5}

and JSD in

10^{-2}

. Inference time is evaluated in s/frame. Evaluation is conducted on a single NVIDIA H200 GPU.

Method	Venue	TTCE		CTC $\downarrow$
Method	Venue	3	4	1	2	3	4	10
UniScene [ 24 ]	CVPR’25	2.74	3.69	0.90	1.84	3.64	3.90	–
OpenDWM [ 34 ]	CVPR’25	2.68	3.65	1.02	2.02	3.37	5.05	–
OpenDWM-DiT [ 34 ]	CVPR’25	2.71	3.66	0.89	1.79	3.06	4.64	–
LiDARCrafter [ 28 ]	AAAI’26	2.65	3.56	1.12	2.38	3.02	4.81	–
LiFi	–	2.67	3.61	0.80	1.11	1.52	2.22	3.19

Table 4: Quantitative results of LiFi for continuous LiDAR scene generation. Here, TTCE and CTC are adopted to evaluate the temporal consistency of the generated results.

Method	mAP $\uparrow$	NDS $\uparrow$
BEVDet (Camera Only) [ 17 ]	0.278	0.341
BEVFusion (+Real LiDAR) [ 30 ]	0.529	0.533
BEVFusion (+Gen LiDAR) [ 30 ]	0.526	0.520

Table 5: Quantitative results of applying LiFi to the downstream 3D object detection task. We compare the performance of the camera-only model and the fusion-based model.

Table 13

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Denoising Step	MMD	JSD	CD	Time
$T_{\text{inference}}=10$	180.97	18.00	1.29	0.11
$T_{\text{inference}}=30$	0.98	4.29	1.00	0.31
$T_{\text{inference}}=50$	2.78	3.66	0.83	0.52
$T_{\text{inference}}=100$	2.73	3.84	0.83	1.04
$T_{\text{inference}}=200$	2.76	3.83	0.82	2.08

Appendix

Table 8: Quantitative evaluation of LiFi’s inference efficiency. The MMD values are reported in

10^{-5}

and JSD in

10^{-2}

. Inference time is evaluated in s/frame.

Num.	MMD $\downarrow$	JSD $\downarrow$	CD $\downarrow$	EMD $\downarrow$
1	14.98	4.83	1.93	112.54
2	7.46	4.35	1.53	99.48
4	5.85	4.17	1.19	76.34
6	2.78	3.66	0.83	50.08

Appendix

Table 9: Quantitative evaluation of the impact of the number of input images on LiFi’s generation performance. The MMD values are reported in

10^{-5}

and JSD in

10^{-2}

Range Image Size	MMD $\downarrow$	JSD $\downarrow$	Time $\downarrow$
$[512,32]$	740.77	31.04	0.29
$[1024,32]$	2.78	3.66	0.52
$[2048,32]$	3.32	4.03	0.97

Appendix

Table 10: Quantitative evaluation of LiFi’s generation performance and inference efficiency across different range image resolutions. The MMD values are reported in

10^{-5}

and JSD in

10^{-2}

Radial Distance	Proportion of Points	MMD $\downarrow$	JSD $\downarrow$	CD $\downarrow$	EMD $\downarrow$
$[0,10)$ m	60.87%	2.08	2.06	0.11	2.05
$[10,20)$ m	23.69%	1.40	3.80	0.94	34.62
$[20,30)$ m	9.25%	2.48	5.52	4.13	189.64
$[30,40)$ m	3.92%	5.18	6.61	12.06	538.58
$[40,+\infty)$ m	2.27%	7.03	9.23	32.82	1264.71

Appendix

Table 11: Quantitative evaluation of LiFi’s generation performance across different radial distance ranges. The MMD values are reported in

10^{-5}

and JSD in

10^{-2}

Figure 8: Additional qualitative generation examples of our method and several baseline approaches on the LiDAR generation task on the nuScenes dataset.

Figure 9: Qualitative comparisons of our method with several baseline approaches on the camera-to-LiDAR task across more representative scenarios on the nuScenes dataset.

Point Cloud Generation

Point cloud generation creates 3D point sets from noise, images, scans, or other conditions for reconstruction, simulation, and synthetic training data.

See all 24 papers

M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesis
Sep 24, 2026Yang Zhou, Jiuhong Xiao, Shizhao Ye +3Multi-View Image Synthesis Image Generation
New York University, New York, USA · Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94720, USA · U.S. Army Combat Capabilities Development Command, Army Research Laboratory, Adelphi, MD 20783, USA
OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation
May 13, 2026Youquan Liu, Weidong Yang, Ao Liang +9Synthetic Data Generation for Robotics 3D Scene Generation
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · School of Computing, Department of Computer Science, National University of Singapore, Singapore · College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China +5
T2LDM++: A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation
Jun 29, 2026Wentao Qu, Qi Zhang, Chenxu Wang +5Diffusion Model Guidance Text-to-3D Generation
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China. · Faculty of Data Science, City University of Macau, Macau, 999078, China. · School of Computer Science and Technology, Beijing Institute of Technology, Beijing,2026 100081, China. +3

Sep 24, 2026J/K move · Enter open · S save

M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesis

Yang Zhou, Jiuhong Xiao, Shizhao Ye, Long Quang+2

New York University, New York, USA · Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94720, USA · U.S. Army Combat Capabilities Development Command, Army Research Laboratory, Adelphi, MD 20783, USA

Multi-View Image Synthesis Image Generation Camera-LiDAR Fusion Novel View Synthesis

Open paper AskPDF arXiv

Robotic novel view synthesis (NVS) must recover both visual appearance and metric 3D structure, yet most generative NVS methods rely only on images, overlooking LiDAR, a complementary sensor common on robotic platforms. We present M3GD, a Camera--LiDAR multimodal representation for generative NVS that composes independently pretrained 2D image and 3D point-cloud foundation models without separately pretraining a cross-modal translator. We show that, after camera projection, frozen LiDAR and image features exhibit substantial shared spatial structure, providing a natural cross-modal representation. M3GD conditions generation on LiDAR through this structure: it combines explicit geometry statistics with learned point-cloud descriptors into view-aligned packets on the image-latent grid, injected through a lightweight residual adapter into a multi-view flow-matching generator whose latent space, decoders, and training objective remain intact. On the GrandTour dataset, M3GD improves target-view RGB and depth synthesis over an image-only version of the same backbone. Ablations show that the gains come from pixel-aligned LiDAR content and that target-view LiDAR acts as a geometric query linking the requested view to source observations. Deployment on a ground robot demonstrates practical real-world operation, with a configurable quality--cost trade-off controlled by the number of Euler integration steps.

Abstract

Point Cloud Generation

Explore similar work

M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesis

OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation

T2LDM++: A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation