SpaRV: Sparse Radiance Voxels for Scalable Geometrically Consistent LiDAR-Visual Mapping and Rendering

Jianheng Liu; Yunfei Wan; Fangming Cheng; Xiangyu Hong; Zuhao Zou; Haotian Li; Longji Yin; Chunran Zheng

cs.ROOct 19, 2026

SpaRV: Sparse Radiance Voxels for Scalable Geometrically Consistent LiDAR-Visual Mapping and Rendering

Authors: Jianheng Liu, Yunfei Wan, Fangming Cheng, Xiangyu Hong, Zuhao Zou, Haotian Li, Longji Yin, Chunran Zheng,

Organizations: Department of Mechanical Engineering, The University of Hong Kong.

Abstract

Robotic simulation, navigation, and inspection need photorealistic and geometrically accurate maps. Radiance fields achieve photorealism, but building them from robot data raises three challenges. A robot's trajectory serves its task, not reconstruction, so images from few viewpoints leave the geometry under-constrained. Residual errors in camera poses and intrinsics blur the map. The map's GPU memory grows with scene size, which limits its level of detail in large scenes. We present SpaRV, a Sparse Radiance Voxel map rendered by differentiable ray marching, which lets irregular LiDAR scans supervise it alongside images. LiDAR depth and occupancy anchor the under-constrained geometry, and normal and smoothness regularizers keep it smooth between LiDAR points. To correct camera errors, SpaRV alternates map training with photometric optimization of the camera poses and focal lengths. To bound GPU memory in training and rendering, SpaRV holds one submap at a time at full resolution and the rest at coarser octree levels. Experiments show that SpaRV keeps the geometry consistent away from the captured trajectory and sharpens textures by optimizing the camera parameters. On one 24,GB GPU, SpaRV reconstructs a university campus captured in 63{,}582 images and simulates other cameras and LiDARs. Our code will be released at https://github.com/hku-mars/SpaRV.

Figures & tables

Panel of Figure 1 — Fig. 2: Rendering results on the FAST-LIVO2 Street scene at an extrapolated view away from the captured trajectory, with zoom crops.

Fig. 3: Overview of SpaRV. From RGB images, LiDAR scans, and initial camera parameters (1) , SpaRV builds a sparse radiance voxel map for geometrically consistent rendering. The top panel shows one frame as its allocated voxels (left) and the view rendered from them (right). (2) A granularity-driven cut selects voxels from the octree, and a differentiable ray marcher renders them into color $\mathbf{C}$ and depth $D$ . (3) Training runs over the submaps coarse-to-fine and alternates map training with camera optimization. The trained map is applied to novel view synthesis, surface reconstruction, and LiDAR simulation.

Fig. 4: Sparse voxels on a two-object scene. (a) Voxels are allocated only around the surfaces. (b) Finer voxels (darker) concentrate where the detail is required. (c) Trilinear interpolation of the eight vertex densities.

Fig. 5: Contraction map

\operatorname{Contr}

in 2D. (a) World space: rays are straight, the identity box of half-extent

r_{i}

is undistorted, and the warped regions map to axis-aligned voxels in contracted space. (b) Contracted space: the same rays curve, and the voxels fall into the inner, mixed, outer, and outside-scene classes. Dots mark corresponding voxel-boundary intersections.

Fig. 6: Rasterization versus ray marching. (a) Rasterization batches a pinhole pixel lattice. (b) Ray marching evaluates each ray independently.

Fig. 7: Loss terms of Eq. ( 11 ) and the regions they constrain. (a)

\mathcal{L}_{\text{rgb}}

: a floater (dashed) and a surface-aligned density profile (solid) render the same pixel color. (b)

\mathcal{L}_{\text{depth}}

: the target

q_{r}(d)

(rose) pulls the rendered depth distribution

p_{r}(d)

(blue) onto the measured range

d_{r}^{*}

. (c)

\mathcal{L}_{\text{n}}

and

\mathcal{L}_{\text{smooth}}

: between the LiDAR anchors, a rippling iso-surface (blue, dashed) fits the images as well as a smooth one, and the regularizers smooth it (rose).

Fig. 8: Occupied voxel guidance on the sparse voxel map. Teal dots represent input LiDAR points. (a) LiDAR depth supervision leaves floaters off the measured rays unconstrained. (b) Guiding camera rays to terminate in the first occupied voxel suppresses floaters in front of it. (c) Voxel adaptation removes floater voxels whose blending weights fall below the pruning threshold.

Panel of Figure 8 — Fig. 9: Effect of the occupied voxel guidance on an Oxford Spires scene: the same view from a map trained (a) without and (b) with $\mathcal{L}_{\text{occ}}$ of Eq. ( 17 ), as color, median depth, and the normal map derived from that depth.

Fig. 10: Rendering results with the odometry and the optimized cameras (Sec. VI ) on training views of FAST-LIVO2 Drive (top) and Sculpture (bottom).

Fig. 11: Large-scale training and deployment of SpaRV. (1) Coarse training (Sec. VII-B1 ) subdivides the octree over the whole scene up to the memory budget and keeps each coarser level as frozen nodes. (2) Submap training (Sec. VII-B2 ) trains one submap $B_{k}$ at a time, with its relevant views, on the train cut $\mathcal{C}(\tau_{t},B_{k},\mathcal{O}_{0})$ : the interior (blue) is subdivided (dashed leaves), the exterior (violet) stays frozen, and grey nodes are not in the cut. (3) Deployment (Sec. VII-C ) renders from the deploy cut $\mathcal{C}(\tau_{d},B_{d},\mathcal{O}_{d})$ of the refined octree around the deploy box $B_{d}$ that contains the camera.

Panel of Figure 11 — Fig. 12: Levels of detail of one octree, seen from a single viewpoint. (a)–(d) Increasingly fine levels. (e)–(h) Cuts at increasing $\tau$ .

Fig. 13: Granularity-driven cut selection. (a) With the box $B$ collapsed to the camera’s focal point, $r(n,B)$ of Eq. ( 23 ) is the number of pixels node $n$ spans on the image plane. (b) The cut of Eq. ( 24 ) against the distance $d(n,B)$ : on each root-to-leaf path, it takes the coarsest node that projects to at most $\tau$ pixels. Here the parent $\pi(n)$ projects larger than $\tau$ and is rejected (dashed, crossed), and its child $n$ is taken. Inside $B$ , where $d(n,B)=0$ , the cut keeps the leaves.

Metrics	Methods	Extrapolated (E)									Interp. (I)
Metrics	Methods	Office-0	Office-1	Office-2	Office-3	Office-4	Room-0	Room-1	Room-2	Avg.	Avg.
PSNR $\uparrow$	InstantNGP	39.863	40.473	35.799	35.928	34.217	32.019	31.761	35.080	35.643	33.713
	3DGS	36.047	36.315	25.438	23.639	28.532	25.944	27.098	26.703	28.714	40.439
	SVRaster	37.914	36.460	31.267	30.544	32.781	31.881	33.291	32.636	33.347	39.600
	LetsGo	30.263	30.808	23.130	21.762	26.225	21.949	24.512	21.756	25.051	38.892
	GS-SDF	40.523	39.488	33.214	33.719	37.017	33.833	35.015	35.615	36.053	40.548

TABLE I: Quantitative rendering results on the Replica dataset. Per-scene values are at extrapolated viewpoints (E); the last two columns average the extrapolated and the interpolated (I) viewpoints over the 8 scenes.

Panel of Figure 14 — Fig. 14: Rendering results on Replica Room-0. Top two rows: an interpolated training view, where every method is close to the reference. Bottom two rows: an extrapolated view of the same maps.

Fig. 15: The six FAST-LIVO2 scenes: LiDAR map, trajectory, and a SpaRV render per scene.

Fig. 16: Rendering results on three Oxford Spires sequences: Keble-05, Obs.-01, and Bodleian-02.

Method	Landmark	CBD	Culture	Drive	Sculpture	Street	Avg.
odometry cameras
InstantNGP	31.705	25.312	23.131	18.379	25.276	23.429	24.539
3DGS	32.260	22.033	21.571	25.570	24.294	26.836	25.427
SVRaster	31.616	26.419	24.749	26.035	24.636	26.025	26.580
SpaRV	31.223	25.923	24.681	26.871	24.483	26.130	26.552
SpaRV (cam. opt.)	36.190	28.114	26.512	29.817	28.548	30.007	29.865

TABLE II: Training-view PSNR on FAST-LIVO2.

Panel of Figure 18 — Fig. 17: Rendering results on the FAST-LIVO2 Street scene at an interpolated view on the captured trajectory.

Metrics	Methods	Landmark	CBD	Culture	Drive	Sculpture	Street	Avg.
PSNR $\uparrow$	InstantNGP	30.983	25.080	23.339	24.111	25.080	23.506	25.350
	3DGS	34.567	22.659	21.112	27.234	27.496	30.123	27.198
	SVRaster	34.172	27.042	24.720	26.763	27.311	28.242	28.042
	LetsGo	33.580	27.849	25.171	29.386	24.588	29.798	28.395
	GS-SDF	34.057	27.157	25.158	28.555	27.794	30.093	28.802
	SpaRV	33.971	26.605	24.720	29.227	27.887	29.643	28.676

TABLE III: Quantitative results on the FAST-LIVO2 test split, under the optimized cameras.

Metric	Method	Keble-02	Keble-03	Keble-04	Keble-05	Obs.-01	Obs.-02	Blenheim-01	Blenheim-02	Blenheim-05	Christ-02	Christ-03	Bodleian-02	Avg.
PSNR $\uparrow$	InstantNGP	22.182	22.265	22.627	23.342	24.082	24.280	24.845	25.788	25.189	23.703	27.079	23.847	24.102
	3DGS	28.910	29.146	24.652	25.082	25.727	26.027	24.631	25.992	25.503	19.971	28.752	24.716	25.759
	SVRaster	26.927	27.544	24.562	24.816	25.240	25.431	23.367	24.011	23.938	22.491	28.819	26.286	25.286
	LetsGo	27.801	27.819	24.612	26.202	23.468	23.556	25.135	24.837	24.090	24.966	29.070	27.009	25.714
	GS-SDF	28.063	28.805	25.270	25.728	25.667	25.765	24.624	25.541	25.054	23.834	26.676	25.536	25.880
	SpaRV	28.452	29.159	26.171	26.705	26.937	27.217	25.033	25.850	25.936	24.357	28.485	27.740	26.837

TABLE IV: Oxford Spires dataset reconstruction benchmark results.

Fig. 18: Rendering results on Garden (top) and Campus (bottom): H3DGS, SpaRV (coarse), and SpaRV (deploy).

Scene	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Garden	H3DGS	18.02	0.620	0.282
	SpaRV (coarse)	20.89	0.712	0.346
	SpaRV (deploy)	21.12	0.713	0.295
Campus	H3DGS	20.01	0.660	0.295
	SpaRV (coarse)	20.62	0.652	0.451
	SpaRV (deploy)	23.16	0.760	0.283

TABLE V: Rendering quality on the two self-collected scenes, evaluated on the training views.

Method	Train [h] $\downarrow$	FPS $\uparrow$	Prim. [M]	Peak VRAM [GiB] $\downarrow$
Method	Train [h] $\downarrow$	FPS $\uparrow$	Prim. [M]	Train	Render
H3DGS	7.3	29.6	15.5	8.3	22.5
SpaRV (coarse)	0.8	248.4	9.9	12.5	4.3
SpaRV (deploy)	8.7	68.8	14.3	11.7	15.6

TABLE VI: Training and rendering cost on Campus, over one

3960

-frame fly-through at

1280{\times}960

. Prim. is the largest number of primitives rendered in a frame.

Method	OmniBlender (11)			Ricoh360 (11)
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
EgoNeRF [ 55 ]	30.93	0.892	0.207	24.89	0.752	0.353
OmniGS [ 56 ]	33.09	0.918	0.174	25.99	0.825	0.262
SpaRV	33.43	0.926	0.129	26.14	0.838	0.233

TABLE VII: Novel-view synthesis results on the test frames of the EgoNeRF dataset.

Fig. 19: Rendering results on two Ricoh360 scenes of the EgoNeRF dataset.

Panel of Figure 26 — Fig. 20: Sensor simulation from one trained map: three camera models (top) and three LiDAR scan patterns (bottom).

Panel of Figure 27 — Fig. 21: Meshes extracted from the six FAST-LIVO2 maps of Sec. VIII-C by marching cubes.

3D Mapping

See all 32 papers

Oct 8, 2026

LVS: Local View Synthesis from Relative Camera Pose by Reusing Previous Views

Interactive scene exploration requires frequent view updates, although small camera motions preserve much of the visible content. Conventional 3D Gaussian Splatting nevertheless renders each target view, leaving this image overlap unexploited. Reusing rendered images offers an alternative. Geometric warping alone cannot recover newly exposed content and remains sensitive to depth errors. We propose a per-scene framework that replaces repeated scene rendering for nearby views with relative-pose-guided RGB-D image reuse. Geometric warping uses depth and relative pose to transport source content, while a lightweight multiscale network predicts RGB residuals to correct artifacts and infer missing appearance. Cached source features further reduce repeated computation. On GS-render, residual refinement improves PSNR by 0.72~dB over pure warping; evaluations on captured and rendered scenes demonstrate low query latency. This separation of scene rendering from local view updates supports responsive scene exploration, with potential applications in augmented and virtual reality.

Qizhou Huo, Xuan Sun, Yongfei Guo +2

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China · Chinese Academy of Sciences, Beijing, China

Novel View Synthesis

Ask PDF

Sep 17, 2026

Uncertainty-Aware RL-Controlled Adaptive 3D Mapping

Voxel-based volumetric mapping is fundamental to 3D reconstruction, yet fixed-resolution grids remain inherently inefficient - wasting memory in uniform regions and losing detail in complex ones. Existing adaptive methods, such as MAP-ADAPT, partially address this by varying resolution based on geometry and user-defined semantic class lists, but these heuristics require expert tuning, lack generalization to unseen objects, and provide no explicit mechanism to control memory usage. We propose an adaptive framework that refines voxels based on semantic entropy, which captures label uncertainty, together with geometric curvature and texture richness as scene complexity cues, yielding principled resolution allocation without reliance on semantic taxonomies. To make the accuracy-memory trade-off explicit and user-controlled, we further introduce a reinforcement learning agent that learns voxel subdivision policies under a user-specified target memory budget, replacing hand-tuned thresholds with a single intuitive control parameter. The resulting multi-resolution TSDF achieves higher geometric accuracy, better semantic consistency, and improved memory-accuracy trade-offs compared to MAP-ADAPT and fixed-resolution baselines on both synthetic and real-world datasets. Our code and models are available at https://github.com/alpayozkan/UnRL.

Alpay Ozkan, Tunc Ozan Aydin, Marc Pollefeys +2

ETH Zurich Zurich, Switzerland · Microsoft Zurich, Switzerland · ETH AI Center Zurich, Switzerland

3D Mapping 3D Reconstruction

Ask PDF

Mar 28, 2026

DiffSoup: Direct Differentiable Rasterization of Triangle Soup for Extreme Radiance Field Simplification

Radiance field reconstruction aims to recover high-quality 3D representations from multi-view RGB images. Recent advances, such as 3D Gaussian splatting, enable real-time rendering with high visual fidelity on sufficiently powerful graphics hardware. However, efficient online transmission and rendering across diverse platforms requires drastic model simplification, reducing the number of primitives by several orders of magnitude. We introduce DiffSoup, a radiance field representation that employs a soup (i.e., a highly unstructured set) of a small number of triangles with neural textures and binary opacity. We show that this binary opacity representation is directly differentiable via stochastic opacity masking, enabling stable training without a mollifier (i.e., smooth rasterization). DiffSoup can be rasterized using standard depth testing, enabling seamless integration into traditional graphics pipelines and interactive rendering on consumer-grade laptops and mobile devices. Code is available at https://github.com/kenji-tojo/diffsoup.

Kenji Tojo, Bernd Bickel, Nobuyuki Umetani

The University of Tokyo · ETH Zürich

Neural Radiance Fields Differentiable Rendering

Ask PDF

Abstract

3D Mapping

Explore similar work

LVS: Local View Synthesis from Relative Camera Pose by Reusing Previous Views

Uncertainty-Aware RL-Controlled Adaptive 3D Mapping

DiffSoup: Direct Differentiable Rasterization of Triangle Soup for Extreme Radiance Field Simplification