Reconstructing translucent objects from multi-view images is a difficult problem. Previously, researchers have used differentiable path tracing and the neural implicit field, which require relatively large computational costs. Recently, many works have achieved good reconstruction results for opaque objects based on a 3DGS pipeline with much higher efficiency. However, such methods have difficulty dealing with translucent objects, because they do not consider the optical properties of translucent objects. In this paper, we propose a novel 3DGS-based pipeline (GTSR) to reconstruct the surface geometry of translucent objects. GTSR combines two sets of Gaussians, surface and interior Gaussians, which are used to model the surface and scattering color when lights pass translucent objects. To render the appearance of translucent objects, we introduce a method that uses the Fresnel term to blend two sets of Gaussians. Furthermore, to improve the reconstructed details of non-contour areas, we introduce the Disney BSDF model with deferred rendering to enhance constraints of the normal and depth. Experimental results demonstrate that our method outperforms baseline reconstruction methods on the NeuralTO Syn dataset while showing great real-time rendering performance. We also extend the dataset with new translucent objects of varying material properties and demonstrate our method can adapt to different translucent materials.
Figures & tables
Figure 1. Our proposed pipeline takes the multi-view images of translucent objects as input, and aims to efficiently reconstruct the meshes by decomposing the geometry and material properties during optimization. Besides, our pipeline can also produce high-quality rendering results for novel views. Compared to NeRF-based and 3DGS-based surface reconstruction methods, our method can achieve the best surface reconstruction performance with efficient training process.
Figure 2. The overview of the surface reconstruction pipeline. We combine the surface and interior Gaussians by introducing a Fresnel term and get the first rendering result CSH . Then, we use surface Gaussians to brake several texture maps (depth, normal, base color and roughness), and leverage interior Gaussians to render the scattering map. After that, we obtain the second rendering result CPBR by blending these maps based on Disney BSDF model in screen space. Finally, we use ground truth to supervise the two rendering results simultaneously.
Figure 3. The visualization of optimization results for different methods. The first row shows the opacity of Gaussian kernels. The second row shows the reconstructed meshes for different methods. The color banding at the top shows the opacity ranging from 0 to 1. Compared to other methods, our method can produce much denser and opaque surface Gaussians. The area of 3DGS’s result pointed out by the arrows shows that many Gaussian kernels with high opacity are located inside the object.
Figure 4. Different ways to blend the Gaussian kernels the light pass through. The notes beside the ellipses represent the opacity. (a): The method used in 3DGS. (b): Using surface and interior Gaussians to render C1 and C2 , then blending C1 and C2 with a weight γ . NeuralTO adopts this method. (c): Embedding weights Fr into 3DGS pipeline.
Figure 5. The different effects of optimizing CSH and CPBR on surface geometry. Different type of lines indicate the surface. (a): CSH renders the appearance of the object via view-dependent color representation, spherical harmonics. The color of spherical harmonics is only related to the view direction, so the optimization results of rendering may be great while the surface geometry could be incorrect. (b): CPBR needs correct normal to get the correct rendering results. Therefore, by supervising the rendering results, the normal can be optimized in the correct direction.
Figure 6. Surface reconstruction results on NeuralTO dataset. We compare several surface reconstruction method with our method, including NeuS-based (NeuralTO ( Cai et al., 2024 ) ) and 3DGS-based (PGSR ( Chen et al., 2024 ) ).
Yuanbao
C. Drag.
A. Drag.
Bear
Juice
Nail
NeuS ( Wang et al., 2021 )
8.5
64.6
N/A
4.7
15.0
137.1
HFNeuS ( Wang et al., 2022 )
9.3
7.0
5.9
2.4
17.0
38.0
RefNeuS ( Ge et al., 2023 )
8.9
4.5
3.7
2.3
21.6
32.7
NeuralTO ( Cai et al., 2024 )
1.2
3.1
1.0
1.1
13.2
2.2
2DGS ( Huang et al., 2024 )
30.4
4.9
4.7
2.3
25.3
4.6
PGSR ( Chen et al., 2024 )
19.4
6.7
17.4
9.9
23.4
39.5
Table 1. Quantitative results of chamfer distance(mm, ×10−3 ) ↓ on NeuralTO Syn dataset ( Cai et al., 2024 ) .
PSNR
SSIM
LPIPS
NeRF ( Mildenhall et al., 2021 )
32.90
0.9605
0.0667
IRON ( Zhang et al., 2022 )
26.83
0.0947
0.7740
Ref-NeuS ( Ge et al., 2023 )
28.70
0.9562
0.0703
NeuralTO ( Cai et al., 2024 )
41.34
0.9853
0.0232
3DGS ( Kerbl et al., 2023 )
43.76
0.9946
0.0137
SSS ( Zhu et al., 2025 )
41.60
0.9917
0.0143
Table 2. Quantitative results of rendering quality for novel view synthesis on NeuralTO Syn dataset.
CD
PSNR
SSIM
LPIPS
w/o PBR
0.0062
38.81
0.9929
0.0203
w/o Fresnel
0.0075
39.91
0.9941
0.0164
w/o Gin
0.0086
40.90
0.9938
0.0179
full
0.0020
45.28
0.9961
0.0126
Table 3. Ablation study on NeuralTO Syn dataset.
Figure 7. Comparison of different methods in the ablation study in some scenes of NeuralTO Syn dataset. (a) uses point wised error map to visualize the difference between wo PBR and full. (b) and (c) show the reconstructed meshes using different methods.
Figure 8. Surface reconstruction results on real-world translucent object datasets. The input images are captured by a camera with a flash in a dark environment.
Transmissive scenes are ubiquitous in daily life, yet reconstructing and rendering them remains highly challenging due to the inherent entanglement between near-field reflections from the surrounding environment on the transmissive surface, and the transmitted content of the scene behind it. This coupling gives rise to dual surface geometries and dual radiance components within each observation, posing ambiguities for standard methods. We present TransmissiveGS, a novel framework for disentangled reconstruction and rendering of transmissive scenes. Specifically, we model the scene with a dual-Gaussian representation and introduce a deferred shading function to jointly render the two Gaussian components. To separate reflection and transmission, we exploit the inherent multi-view inconsistency of reflections and leverage the residuals from reconstructing multi-view consistent content as cues for disentangled geometry and appearance modeling. We further propose a reflection light field that enables high-fidelity estimation of near-field reflections. During training, we introduce a high-frequency regularization to preserve fine details. We also contribute a new synthetic dataset for evaluating transmissive surface reconstruction. Experiments on both synthetic and real-world scenes demonstrate that TransmissiveGS consistently outperforms prior Gaussian Splatting-based methods in both reconstruction and rendering quality for transmissive scenes.
Gaussian Splatting is a powerful tool for reconstructing diffuse scenes, but it struggles to simultaneously model specular reflections and the appearance of objects behind semi-transparent surfaces. These specular reflections and transmittance are essential for realistic novel view synthesis, and existing methods do not properly incorporate the underlying physical processes to simulate them. To address this issue, we propose RT-GS, a unified framework that integrates a microfacet material model and ray tracing to jointly model specular reflection and transmittance in Gaussian Splatting. We accomplish this by using separate Gaussian primitives for reflections and transmittance, which allow modeling distant reflections and reconstructing objects behind transparent surfaces concurrently. We utilize a differentiable ray tracing framework to obtain the specular reflection and transmittance appearance. Our experiments demonstrate that our method successfully produces reflections and recovers objects behind transparent surfaces in complex environments, achieving significant qualitative improvements over prior methods where these specular light interactions are prominent.
Kunnong Zeng, Chensheng Peng, Yichen Xie +2
University of Utah · University of California, Berkeley
While 3D Gaussian splatting has emerged as a powerful paradigm, it fundamentally fails to model transparency such as glass panels. The core challenge lies in decoupling the intertwined radiance contributions from transparent interfaces and the transmitted geometry observed through the glass. We present GLINT, a framework that models scene-scale transparency through explicit decomposed Gaussian representation. GLINT reconstructs the primary interface and models reflected and transmitted radiance separately, enabling consistent radiance transport. During optimization, GLINT bootstraps transparency localization from geometry-separation cues induced by the decomposition, together with geometry and material priors from a pre-trained video relighting model. Extensive experiments demonstrate consistent improvements over prior methods for reconstructing complex transparent scenes.