cs.GRApr 1, 2026

RT-GS: Gaussian Splatting with Reflection and Transmittance Primitives

Authors: Kunnong Zeng, Chensheng Peng, Yichen Xie, Masayoshi Tomizuka, Cem Yuksel

Organizations: University of Utah · University of California, Berkeley

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Gaussian Splatting is a powerful tool for reconstructing diffuse scenes, but it struggles to simultaneously model specular reflections and the appearance of objects behind semi-transparent surfaces. These specular reflections and transmittance are essential for realistic novel view synthesis, and existing methods do not properly incorporate the underlying physical processes to simulate them. To address this issue, we propose RT-GS, a unified framework that integrates a microfacet material model and ray tracing to jointly model specular reflection and transmittance in Gaussian Splatting. We accomplish this by using separate Gaussian primitives for reflections and transmittance, which allow modeling distant reflections and reconstructing objects behind transparent surfaces concurrently. We utilize a differentiable ray tracing framework to obtain the specular reflection and transmittance appearance. Our experiments demonstrate that our method successfully produces reflections and recovers objects behind transparent surfaces in complex environments, achieving significant qualitative improvements over prior methods where these specular light interactions are prominent.

Figures & tables

Figure 1: We present a method that optimizes the quality of specular reflection and transmittance concurrently in GS-based method. For the rasterization-based gaussian splatting method [ 8 ] in the left image, the specular reflection details are blurred on the surface of objects. For the reflection rays integrated method [ 23 ] in the middle image, the object behind the semi-transparent surface is blurred. We solve both the complex reflection and transmittance ray effects at the same time by integrating reflection and transmittance rays into gaussian splatting. Then, we can see the transmittance appearance and specular reflection details, as shown in the right image.
Figure 2: Overview of our method. In the first stage, we start with the rasterization process using diffuse Gaussians to obtain per-pixel material maps and diffuse color. In the second stage, we generate specular reflection rays from the surface of diffuse Gaussians and utilize ray tracing on reflection Gaussians. In the third stage, we trace first-bounce transmittance rays from the surface into the transparent objects on transmittance Gaussians to obtain the inside color and second-bounce transmittance rays from the back surface of the mesh on diffuse Gaussian again to obtain the outside color. Finally, we combine the diffuse, reflection and transmittance colors for the final image. We jointly optimize these three types of Gaussians using ground truth images for supervision.
Figure 3: Mesh-guided ray tracing for transparent objects. The surface mesh is extracted from the diffuse Gaussian. The first bounces of transmittance rays are traced from the depth positions of the diffuse Gaussian on the transmittance Gaussian. The second bounces of transmittance rays are traced from the back surface positions of the extracted mesh on the diffuse Gaussian. To obtain the back surface positions, we trace rays on the mesh to get the second nearest depth.
Figure 4: Illustration of the regularization to the depth. When reconstructing the internal object, the depth of first-bounce transmittance rays of the part of the object that exists is reconstructed well, as demonstrated by the solid red curve. But in the parts where no object is existing, the depth is trained to the outside the transparent object, as shown by the red dashes curve, because it tries to reconstruct the outside color, which should be rendered by second-bounce transmittance rays. Therefore, we add regularization to the depth to push it back into the transparent object.
Figure 5: Qualitative comparison on Ref-Real [ 21 ] Scenes.
Figure 6: Qualitative comparison on NU-NeRF [ 19 ] Scenes.
Figure 7: Ablation study on the BallStatue scene.
Figure 8: Ablation study on the Sedan scene. (a) Approximating the BSDF with separate components [ 24 , 13 ] leads to substantial errors in the reconstructions of reflections. (b) Our solution of using the full microfacet model produces better reflections.
 PSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrow
w/o transmittance Gaussians24.8440.8660.196
w/o specular loss25.6730.8820.174
w/o depth loss25.9410.8810.172
w/o mesh guided ray tracing25.8890.8800.174
Full Model26.0530.8850.174
Table 3: Ablation study on the BallStatue scene.
Figure 9: Qualitative comparison on Ref-Real [ 21 ] Scenes.
Figure 10: Qualitative comparison on NU-NeRF [ 19 ] Scenes.
More in this topic

Differentiable Ray Tracing

See all 12 papers

Explore similar work

CardsList

  1. RT-Splatting: Joint Reflection-Transmission Modeling with Gaussian Splatting

    May 18, 2026Ji Shi, Xianghua Ying, Bowei Xing +23D Gaussian SplattingNovel View Synthesis

    State Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology Peking University

  2. TransmissiveGS: Residual-Guided Disentangled Gaussian Splatting for Transmissive Scene Reconstruction and Rendering

    May 11, 2026Zhenyu Liang, Xiao Zhang, Tianchao Li +23D Gaussian Splatting3D Reconstruction

    HKUST

  3. Ref-DGS: Reflective Dual Gaussian Splatting

    Mar 8, 2026Ningjing Fan, Yiqun Wang, Dong-Ming Yan +13D Gaussian Splatting3D Reconstruction

    Chongqing University, China · MAIS, Institute of Automation, Chinese Academy of Sciences and UCAS, China · King Abdullah University of Science and Technology (KAUST), Saudi Arabia