We address inverse rendering with 3D Gaussian Splatting (3DGS) by introducing explicit visibility modeling for relighting as well as material and light decomposition. Our approach augments the standard rasterized 3DGS pipeline with a sparse visibility oracle that estimates transmittance-weighted occlusion for Gaussian primitives from sampled lighting directions. This oracle is implemented using a point-based ray tracer accelerated by a bounding volume hierarchy (BVH) and queried sparsely within the inverse-rendering objectives, while the original 3DGS loop is retained for training efficiency, interactive visualization, and tooling compatibility. Shading follows a physically based rendering (PBR) formulation with a principled microfacet bidirectional reflectance distribution function (BRDF) and low-resolution high-dynamic-range (HDR) environment maps. We evaluate our method on the TensoIR synthetic dataset against baselines based on neural radiance fields (NeRFs), tensor representations, and 3DGS. The results show competitive performance and improvements over several earlier baselines for novel-view synthesis (NVS), albedo reconstruction, and relighting. Ablations indicate that explicit visibility modeling is an important contributor to relighting and material-illumination disentanglement, while pure PBR supervision can favor appearance fitting in NVS. Overall, the findings suggest that sparse, point-based visibility queries effectively complement rasterized splatting for inverse rendering while preserving the practical advantages of the 3DGS ecosystem.
Figures & tables
Figure 1
Figure 1: VisibilitySplat. Given a Gaussian point cloud optimized from multi-view images under unknown illumination (left), VisibilitySplat recovers physical properties (normals, materials, and illumination) using a visibility ray tracer (center), enabling relighting (right).
Figure 2: Method Overview. VisibilitySplat is a two-stage, 3DGS-based IR pipeline that reconstructs scene geometry, materials, and environment illumination from multi-view images. Stage 1 pretrains a 3D Gaussian point cloud with the standard 3DGS attributes and per-splat normals derived from depth. Stage 2 adds a sparse visibility oracle, implemented with BVH-accelerated point-based ray tracing, to compute transmittance-weighted visibility over the Gaussians. Together with physically based rendering (PBR) using a Disney “principled” BRDF [ 6 ] , this enables joint decomposition into material parameters and estimation of a compact high-dynamic-range (HDR) environment map for relighting.
Figure 3: Qualitative comparison of albedo, normal and environmental lighting estimation on the TensoIR synthetic dataset [ 18 ] .
Figure 4: (a) mean transmittance from all directions; (b) transmittance from a single direction.
Figure 5: Visual results on the TensoIR synthetic dataset [ 18 ] . Each model is displayed in order from left to right and from top to bottom: ground truth, novel-view synthesis (NVS), normal map, and mean visibility.
Table 2: Quantitative comparison on the TensoIR Synthetic dataset [ 18 ] . All reported values are aggregated means over the evaluated objects. Most results are taken from the respective papers. For IRGS [ 15 ] , we ran their evaluation ourselves because the publication did not include the full results.
Method
NVS
Albedo
Relighting
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PBR-only
35.697
0.971
0.046
20.848
0.922
0.105
28.848
0.932
0.077
w/o visibility
34.789
0.966
0.053
30.313
0.933
0.101
29.091
0.933
0.077
w/o opacity loss
34.996
0.967
0.052
29.834
0.932
0.103
29.939
0.938
0.077
Full
34.996
0.976
0.053
30.321
0.933
0.101
30.594
0.942
0.076
Table 3: Ablation study of the different components of VisibilitySplat on the Lego model from the TensoIR synthetic dataset [ 18 ] .
Faithful inverse rendering requires visibility and indirect radiance to explain secondary illumination and inter-reflection, yet rasterization-oriented Gaussian representations do not naturally support the secondary-ray queries needed to recover them. We present IRGS++ (Inter-Reflective Gaussian Splatting), a unified robust and efficient Gaussian inverse rendering framework. During transport-aware optimization, IRGS++ employs differentiable 2D Gaussian ray tracing on surface-oriented Gaussian primitives to query visibility and indirect radiance on the fly and evaluate the full rendering equation for inter-reflective transport. This physical core makes Gaussian inverse rendering physically grounded beyond rasterized appearance modeling. To make this backbone useful beyond low-gloss dielectric scenes, the framework incorporates metallic-aware material modeling and robust reflective initialization for glossy, specular, and metallic materials. To make it practical, multiple importance sampling and denoising stabilize finite-sample rendering, while mesh-based secondary-attribute queries reduce the cost of relighting under novel illumination. Quantitative evaluations on low-gloss and glossy benchmarks show improved decomposition and relighting quality together with favorable quality--speed trade-offs under the reported configurations, while real-world studies illustrate plausible relighting under novel illumination.
Chun Gu, Xiaofei Wei, Zixuan Zeng +2
School of Data Science, Fudan University, and Shanghai Innovation Institute · Shanghai Innovation Institute · School of Data Science, Fudan University
We present SGS-Intrinsic, an indoor inverse rendering framework that works well for sparse-view images. Unlike existing 3D Gaussian Splatting (3DGS) based methods that focus on object-centric reconstruction and fail to work under sparse view settings, our method allows to achieve high-quality geometry reconstruction and accurate disentanglement of material and illumination. The core idea is to construct a dense and geometry-consistent Gaussian semantic field guided by semantic and geometric priors, providing a reliable foundation for subsequent inverse rendering. Building upon this, we perform material-illumination disentanglement by combining a hybrid illumination model and material prior to effectively capture illumination-material interactions. To mitigate the impact of cast shadows and enhance the robustness of material recovery, we introduce illumination-invariant material constraint together with a deshadowing model. Extensive experiments on benchmark datasets show that our method consistently improves both reconstruction fidelity and inverse rendering quality over existing 3DGS-based inverse rendering approaches. Our code is available at https://github.com/GrumpySloths/SGS_Intrinsic.github.io.
Jiahao Niu, Rongjia Zheng, Wenju Xu +2
Sun Yat-sen University, China · Amazon, USA · Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China
Inverse rendering aims to recover both 3D geometry and physically meaningful material properties from images, enabling applications such as relighting and novel view synthesis. Optimization-based methods achieve high fidelity but require costly per-scene fitting, while image-space learning-based approaches often suffer from multi-view inconsistencies and lack an explicit 3D representation for stable novel view rendering. We present a feed-forward multi-view reconstruction framework for inverse rendering that directly predicts a structured 3D Gaussian representation with intrinsic material attributes. Each Gaussian primitive is parameterized by mean, normal, opacity, rotation, scale, albedo, metallic, and roughness, enabling a disentangled and physically grounded scene representation. Our model integrates priors from a material estimation network with a multi-view 3D reconstruction backbone, allowing joint prediction of geometry and reflectance parameters in a single forward pass. Experiments on synthetic and real-world datasets demonstrate improved multi-view consistency compared to 2D baselines, accurate material recovery, and stable novel view rendering. Our representation further supports physically-based relighting and more faithful modeling of view-dependent effects compared to existing RGB-based feed-forward reconstruction methods. Our project webpage is: https://poliik.github.io/invsplat/.
Polina Karpikova, Wenjing Bian, Haofei Xu +2
University of Tübingen, Tübingen AI Center · ETH Zurich