SGS-Intrinsic: Semantic-Invariant Gaussian Splatting for Sparse-View Indoor Inverse Rendering

Jiahao Niu; Rongjia Zheng; Wenju Xu; Wei-Shi Zheng; Qing Zhang

cs.CVMar 29, 2026

SGS-Intrinsic: Semantic-Invariant Gaussian Splatting for Sparse-View Indoor Inverse Rendering

Authors: Jiahao Niu, Rongjia Zheng, Wenju Xu, Wei-Shi Zheng, Qing Zhang

Organizations: Sun Yat-sen University, China · Amazon, USA · Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China

Abstract

We present SGS-Intrinsic, an indoor inverse rendering framework that works well for sparse-view images. Unlike existing 3D Gaussian Splatting (3DGS) based methods that focus on object-centric reconstruction and fail to work under sparse view settings, our method allows to achieve high-quality geometry reconstruction and accurate disentanglement of material and illumination. The core idea is to construct a dense and geometry-consistent Gaussian semantic field guided by semantic and geometric priors, providing a reliable foundation for subsequent inverse rendering. Building upon this, we perform material-illumination disentanglement by combining a hybrid illumination model and material prior to effectively capture illumination-material interactions. To mitigate the impact of cast shadows and enhance the robustness of material recovery, we introduce illumination-invariant material constraint together with a deshadowing model. Extensive experiments on benchmark datasets show that our method consistently improves both reconstruction fidelity and inverse rendering quality over existing 3DGS-based inverse rendering approaches. Our code is available at https://github.com/GrumpySloths/SGS_Intrinsic.github.io.

Figures & tables

Figure 2: Overview of our method. There are two-stage training in our framework. In stage I, we leverage a pretrained VGGT model to get a dense scene layout point cloud. Next, the geometry of the 3D Gaussians is supervised by normal and semantic priors distilled from a pretrained model. In state II, we perform inverse rendering based on the Gaussians obtained from the first stage. A mixture light model together with a deshadowing module are employed to model illumination and occlusion relationships, which are then integrated with the Gaussian-rendered G-buffer for physically based (PBR) novel view rendering. Moreover, to ensure consistent material representations during training, we incorporate an additional material consistency constraint.

Method	Roughness MSE $\downarrow$	Normal MAE $\downarrow$	Albedo			Novel View Synthesis			Novel View Synthesis for PBR			FPS
Method	Roughness MSE $\downarrow$	Normal MAE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FPS
GSIR [ 24 ]	28.1	26.2	16.4	0.56	0.41	18.5	0.62	0.35	17.8	0.60	0.43	40.01
GIGS [ 7 ]	30.2	25.9	12.1	0.49	0.42	18.6	0.60	0.59	17.2	0.58	0.44	25.26
R3DG [ 13 ]	25.6	24.3	15.1	0.68	0.36	19.8	0.75	0.30	19.13	0.68	0.43	9.5
IRGS [ 14 ]	26.2	23.2	14.4	0.61	0.35	18.9	0.68	0.34	19.3	0.71	0.32	0.20
SVGIR [ 37 ]	22.3	20.1	16.6	0.70	0.33	19.0	0.76	0.29	18.73	0.74	0.31	11.83

Table 1: Quantitative evaluation on the Interiorverse dataset.

Figure 3: Comparison of scene radiance and local-light rendering for novel views. As shown, both R3DG and SVGIR fail to accurately model local illumination and occlusion for sparse-view indoor scenes, while our method produces physically plausible rendering with smooth local light effects.

Figure 4: Qualitative comparison of inverse rendering on the Interiorverse dataset [ 56 ] .

Figure 5: Qualitative comparison of inverse rendering on the MipNeRF dataset [ 4 ] .

Figure 6: Qualitative ablation study on the Interiorverse dataset. Note, “Mat-inv losses” refer to $\mathcal{L}_{\text{relight}}$ and $\mathcal{L}_{\text{cross\_semantic}}$ .

Method	DL3DV			FIPT			MipNeRF
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
GSIR [ 24 ]	18.65	0.38	0.48	19.12	0.42	0.42	18.95	0.40	0.45
R3DG [ 13 ]	18.22	0.35	0.52	18.85	0.39	0.47	18.61	0.37	0.49
SVGIR [ 37 ]	17.96	0.31	0.59	18.53	0.36	0.54	18.42	0.35	0.56
GeoSplat [ 50 ]	18.98	0.40	0.44	19.75	0.45	0.38	19.44	0.43	0.41
IRGS [ 14 ]	18.34	0.36	0.51	18.96	0.40	0.45	18.72	0.38	0.48

Table 2: Quantitative comparison with previous methods in terms of novel view synthesis on real-world datasets.

Panel of Figure 7 — Figure 7: Expressiveness of our SGM-based interior lighting model for complex emitters.

Figure 8: Application to lighting control.

Method	Albedo			NVS For PBR
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
w/ Geom noise	17.5	0.73	0.31	21.6	0.75	0.45
w/ normal noise	17.5	0.73	0.31	21.4	0.74	0.46
w/ semantic noise	17.6	0.75	0.30	21.7	0.75	0.45
w/ GT Cameras	17.7	0.75	0.28	21.7	0.76	0.42
Ours	17.7	0.76	0.29	21.7	0.76	0.43

Table 3: Robustness analysis on Interiorverse dataset. “w/ Geom noise” denotes introducing random positional perturbations to the Gaussian geometric field, while “w/ normal noise” and “w/ semantic noise” mean adding random noise to normal priors and semantic labels. “GT Cameras” refers to camera poses reconstructed from 200 views via VGGT.

Method	Albedo			NVS For PBR
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
w/o $\mathcal{L}_{normal}$	16.90	0.73	0.32	20.35	0.70	0.32
w/o $\mathcal{L}_{corresponding}$	16.80	0.71	0.32	19.78	0.70	0.30
w/o $L_{i}^{\text{SGM}}$	14.78	0.65	0.40	20.03	0.69	0.37
w/o Deshadowing	16.22	0.68	0.35	20.71	0.72	0.35
w/o $\mathcal{L}_{relight}$	17.20	0.73	0.34	20.30	0.73	0.34

Table 4: Ablation study on the Interiorverse dataset. “Deshadowing” refers to the deshadowing module.

Figure 9: Qualitative evaluation on the TensoIR dataset.

Module	Parameter	Value
HashEnc	Number of levels	16
	Max. entries per level	$2^{19}$
	feature dimensions	2
	Coarsest resolution	32
	Finest resolution	4096
MLP	MLP layers	$36\times 32\times 32\times 1$

Table 5: Architecture of Deshadowing Module

Method	Albedo			NVS For PBR
Method	PSNR $\uparrow$	SSIM $\uparrow$	LIPPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LIPPS $\downarrow$
GSIR [ 24 ]	23.67	0.90	0.11	23.98	0.82	0.14
R3DG [ 13 ]	22.46	0.89	0.12	25.11	0.85	0.12
SVGIR [ 37 ]	24.98	0.90	0.13	21.05	0.68	0.19
GeoSplat [ 50 ]	24.31	0.89	0.13	25.45	0.87	0.12
IRGS [ 14 ]	22.57	0.86	0.14	25.32	0.85	0.13

Table 6: Quantitative comparison on the TensoIR dataset.

	VGGT	LSeg	StableNorm	RGBX	Training
Time	$\sim$ 20s	$\sim$ 2min	$\sim$ 10s	$\sim$ 30s	$\sim$ 40min

Table 7: Training time breakdown

Figure 10: Effect of normal and semantic priors.

Figure 11: Multi-resolution feature visualization and comparison with GT view

Figure 12: More qualitative comparison of albedo estimation on the synthetic InteriorVerse dataset.

Figure 13: More qualitative comparison of PBR Rendered Novel View on the synthetic InteriorVerse dataset.

Figure 14: More qualitative comparison of albedo estimation on the synthetic InteriorVerse dataset.

Figure 15: More illumination decomposition results on the synthetic InteriorVerse dataset.

Figure 16: More results of object insertion, material and light editing

Inverse Rendering

Inverse rendering estimates scene geometry, materials, lighting, and other causes of images from observed visual or sensor data.

See all 68 papers

Dec 10, 2025

GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures

Recent advances in Gaussian Splatting-based inverse rendering extend Gaussian primitives with shading parameters and physically grounded light transport, enabling high-quality material recovery from dense multi-view captures. However, the accuracy of these methods degrades under sparse-view settings, where limited observations lead to severe ambiguity between geometry, reflectance, and lighting. We introduce GAINS (Gaussian-based Inverse rendering from Sparse multi-view captures), a two-stage inverse rendering framework that leverages foundation models as priors to stabilize geometry and material estimation. The core technical contribution of this paper is an inverse rendering framework that unifies foundation model priors with physically-based representations in an optimization scheme. GAINS first refines geometry using monocular depth, normal, and diffusion priors, and then employs segmentation, intrinsic image decomposition (IID), and diffusion priors to regularize material recovery. Extensive experiments on synthetic and real-world datasets show that GAINS significantly improves material parameter accuracy, relighting quality, and novel-view synthesis compared to state-of-the-art Gaussian-based inverse rendering methods. While GAINS outperforms and remains competitive across a wide range of objects captured with 4 to 32 cameras, the improvement is particularly pronounced under sparse-view settings, where ambiguity is high and learning-based priors become especially beneficial. Project page: https://patrickbail.github.io/gains/

Patrick Noras, Jun Myeong Choi, Didier Stricker +2

University of Kaiserslautern-Landau, Kaiserslautern, Germany · German Research Center for Artificial Intelligence, Kaiserslautern, Germany · University of North Carolina at Chapel Hill, NC, USA +1

Inverse Rendering 3D Gaussian Splatting Sparse-View 3D Reconstruction Physically Based Rendering

Ask PDF

Jul 2, 2026

InvSplat: Inverse Feed-Forward Scene Splatting

Inverse rendering aims to recover both 3D geometry and physically meaningful material properties from images, enabling applications such as relighting and novel view synthesis. Optimization-based methods achieve high fidelity but require costly per-scene fitting, while image-space learning-based approaches often suffer from multi-view inconsistencies and lack an explicit 3D representation for stable novel view rendering. We present a feed-forward multi-view reconstruction framework for inverse rendering that directly predicts a structured 3D Gaussian representation with intrinsic material attributes. Each Gaussian primitive is parameterized by mean, normal, opacity, rotation, scale, albedo, metallic, and roughness, enabling a disentangled and physically grounded scene representation. Our model integrates priors from a material estimation network with a multi-view 3D reconstruction backbone, allowing joint prediction of geometry and reflectance parameters in a single forward pass. Experiments on synthetic and real-world datasets demonstrate improved multi-view consistency compared to 2D baselines, accurate material recovery, and stable novel view rendering. Our representation further supports physically-based relighting and more faithful modeling of view-dependent effects compared to existing RGB-based feed-forward reconstruction methods. Our project webpage is: https://poliik.github.io/invsplat/.

Polina Karpikova, Wenjing Bian, Haofei Xu +2

University of Tübingen, Tübingen AI Center · ETH Zurich

Inverse Rendering 3D Gaussian Splatting 3D Reconstruction Multi-View 3D Reconstruction Feed-Forward 3D Reconstruction Physically Based Rendering

Ask PDF

Oct 14, 2026

VisibilitySplat: Ray-Traced Occlusion for Relighting with 3D Gaussian Splatting

We address inverse rendering with 3D Gaussian Splatting (3DGS) by introducing explicit visibility modeling for relighting as well as material and light decomposition. Our approach augments the standard rasterized 3DGS pipeline with a sparse visibility oracle that estimates transmittance-weighted occlusion for Gaussian primitives from sampled lighting directions. This oracle is implemented using a point-based ray tracer accelerated by a bounding volume hierarchy (BVH) and queried sparsely within the inverse-rendering objectives, while the original 3DGS loop is retained for training efficiency, interactive visualization, and tooling compatibility. Shading follows a physically based rendering (PBR) formulation with a principled microfacet bidirectional reflectance distribution function (BRDF) and low-resolution high-dynamic-range (HDR) environment maps. We evaluate our method on the TensoIR synthetic dataset against baselines based on neural radiance fields (NeRFs), tensor representations, and 3DGS. The results show competitive performance and improvements over several earlier baselines for novel-view synthesis (NVS), albedo reconstruction, and relighting. Ablations indicate that explicit visibility modeling is an important contributor to relighting and material-illumination disentanglement, while pure PBR supervision can favor appearance fitting in NVS. Overall, the findings suggest that sparse, point-based visibility queries effectively complement rasterized splatting for inverse rendering while preserving the practical advantages of the 3DGS ecosystem.

Sven Kluge, Oliver Staadt

University of Rostock, Albert-Einstein-Straße 22, 18059 Rostock, Germany

Inverse Rendering 3D Gaussian Splatting

Ask PDF

Abstract

Inverse Rendering

Explore similar work

GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures

InvSplat: Inverse Feed-Forward Scene Splatting

VisibilitySplat: Ray-Traced Occlusion for Relighting with 3D Gaussian Splatting