Gaussian Splatting with Reflections Guided by What is Seen

Yiming Liang; Tianyu Xiao; Hiroshi Ishikawa

cs.CVOct 9, 2026

Gaussian Splatting with Reflections Guided by What is Seen

Authors: Yiming Liang, Tianyu Xiao, Hiroshi Ishikawa

Organizations: Department of Computer Science and Communications Engineering, Waseda University, Japan

Abstract

While novel view synthesis methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have shown impressive performance on common scenes, they often struggle with capturing view-dependent specular effects, particularly near-field reflections. Existing methods utilize environment maps, which only represent lights from infinity, or environment Gaussians, which rely solely on indirect supervision from reflections, are insufficient for accurately modeling such effects. Here, we propose a framework based on 2D Gaussian Splatting (2DGS) that leverages directly observable environment information in input images. Our approach also integrates a hybrid reflection appearance representation and view-dependent specular modeling. Experiments on public benchmarks and a newly curated near-field reflection dataset demonstrate that our method achieves superior rendering quality and geometric accuracy.

Figures & tables

Figure 1: Overview of our method. We use two Gaussian sets: primary for the base scene and secondary for near-field environment light. Reflections are rendered by ray-tracing through both sets, with colors decoded from a hybrid triplane and SH coefficients. Final pixel color is obtained by blending diffuse and reflection components using a view-dependent specular weight.

	NeRF-Casting [ 17 ]			EnvGS [ 6 ]			Ref-Real [ 14 ]			Average
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
3DGS [ 2 ]	29.7161	0.8930	0.1442	26.6340	0.8831	0.1525	24.0680	0.6600	0.2532	27.1485	0.8131	0.1824
2DGS [ 7 ]	28.7545	0.8728	0.1825	26.2465	0.8752	0.1738	23.8770	0.6699	0.2749	26.5714	0.8057	0.2114
GShader [ 3 ]	27.1522	0.8453	0.2243	25.4491	0.8553	0.1982	23.1039	0.6370	0.3098	25.4247	0.7781	0.2470
3DGS-DR [ 4 ]	28.8993	0.8805	0.1956	25.8913	0.8594	0.1908	22.8938	0.6303	0.2913	26.2290	0.7924	0.2264
RefGS [ 5 ]	24.2534	0.8122	0.2408	25.6502	0.8536	0.2014	23.8390	0.6493	0.2898	24.4257	0.7671	0.2484

Table 1: Quantitative NVS results on existing datasets [ 17 , 6 , 14 ] . Best , second best , and third best results are highlighted.

	Black Monitor			White Monitor			Coffee Machine			Cleaner			Average
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
3DGS [ 2 ]	28.4841	0.9140	0.1071	35.0724	0.9698	0.0938	29.5640	0.9414	0.0983	33.1301	0.9598	0.0882	31.5626	0.9463	0.0969
2DGS [ 7 ]	27.5291	0.9035	0.1181	33.5976	0.9647	0.1153	28.5130	0.9287	0.1270	31.8991	0.9536	0.1070	30.3847	0.9376	0.1168
GShader [ 3 ]	24.1317	0.8507	0.1817	26.9088	0.9143	0.1956	24.6470	0.8701	0.2018	28.3498	0.9292	0.1537	26.0093	0.8911	0.1832
3DGS-DR [ 4 ]	27.1847	0.8992	0.1254	33.2582	0.9623	0.1194	28.8980	0.9315	0.1216	28.2506	0.9313	0.1358	29.3979	0.9310	0.1256
RefGS [ 5 ]	27.8328	0.8970	0.1382	32.8170	0.9572	0.1271	28.7207	0.9186	0.1479	25.5781	0.9129	0.1702	28.7372	0.9214	0.1458

Table 2: Quantitative NVS results on our custom NearRef dataset. Best , second best , and third best results are highlighted.

Panel of Figure 2 — Figure 2: Qualitative NVS comparison on the existing datasets [ 17 , 6 ] .

Panel of Figure 3 — Figure 3: Qualitative NVS comparison on our custom NearRef dataset.

Panel of Figure 4 — Figure 4: Qualitative comparison of surface normal estimation.

Panel of Figure 5 — Figure 5: Disentanglement of diffuse and reflection components.

Panel of Figure 6 — Figure 6: Reconstructed environment light.

Methods	$\mathcal{G}_{\text{prim}}\cup\mathcal{G}_{\text{sec}}$	$\bm{c}_{\text{ref}}\neq\bm{c}_{\text{prim}}$	$\bm{c}_{\text{sec}}$	$\bm{c}_{\text{tri}}$	$\bm{k}$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
A						27.4097	0.8437	0.1827
B	✓					27.4368	0.8443	0.1799
C	✓	✓	✓			27.5775	0.8462	0.1766
D	✓	✓		✓		27.6532	0.8470	0.1791
E	✓	✓	✓	✓		27.6678	0.8479	0.1772
F (Full)	✓	✓	✓	✓	✓	27.7905	0.8497	0.1752

Table 3: Ablation studies. “✓” denotes included components for variant A–F. Best results in bold .

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Point cloud and camera trajectory of each scene in custom NearRef dataset.

	NeRF-Casting [ 17 ]												EnvGS [ 6 ]
	Compact			Grinder			Hatchback			Toaster			Audi
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
3DGS [ 2 ]	30.1704	0.9090	0.1371	31.6084	0.8598	0.1696	27.1412	0.8703	0.1567	29.9444	0.9328	0.1133	27.9984	0.8849	0.1296
2DGS [ 7 ]	29.6057	0.8880	0.1758	29.9115	0.8374	0.2165	26.5643	0.8472	0.1983	28.9365	0.9186	0.1395	27.5342	0.8750	0.1628
GShader [ 3 ]	28.2558	0.8692	0.2005	26.9084	0.7899	0.3108	25.7765	0.8248	0.2222	27.6720	0.8972	0.1638	26.5692	0.8503	0.1942
3DGS-DR [ 4 ]	29.7766	0.8848	0.1851	29.8010	0.8766	0.2198	26.5551	0.8358	0.2222	29.4645	0.9248	0.1554	27.8997	0.8735	0.1644

Appendix

Table 4: Per-scene quantitative results of NVS on existing datasets [ 17 , 6 , 14 ] . Best , second best , and third best results are highlighted.

Panel of Figure 8 — Figure 8: Additional qualitative comparison of novel view synthesis on the existing dataset.

Panel of Figure 9 — Figure 9: Additional qualitative comparison of NVS on custom NearRef dataset.

Neural Rendering

Neural rendering uses learned 3D scene and light representations to synthesize, edit, and relight images from new viewpoints.

See all 118 papers

Ref-DGS: Reflective Dual Gaussian Splatting
Mar 8, 2026Ningjing Fan, Yiqun Wang, Dong-Ming Yan +13D Gaussian Splatting 3D Reconstruction
Chongqing University, China · MAIS, Institute of Automation, Chinese Academy of Sciences and UCAS, China · King Abdullah University of Science and Technology (KAUST), Saudi Arabia
RGS: Reflection-aware Gaussian Splatting via Learning Geometry Continuity for Reflective Objects
Sep 16, 2026Xiaobiao Du, Yida Wang, Cheng Bi +23D Gaussian Splatting 3D Reconstruction
University of Technology Sydney · Li Auto Inc. · Adelaide University
NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting
Sep 29, 2026Pratik Singh Bisht, Andreas Kolb3DGS Compression 3D Gaussian Splatting
Computer Graphics Group University of Siegen

Mar 8, 2026J/K move · Enter open · S save

Ref-DGS: Reflective Dual Gaussian Splatting

Ningjing Fan, Yiqun Wang, Dong-Ming Yan, Peter Wonka

Chongqing University, China · MAIS, Institute of Automation, Chinese Academy of Sciences and UCAS, China · King Abdullah University of Science and Technology (KAUST), Saudi Arabia

3D Gaussian Splatting 3D Reconstruction Novel View Synthesis

Open paper AskPDF arXiv

The reflective appearance, especially strong and typically near-field specular reflections, poses a fundamental challenge for accurate surface reconstruction and novel view synthesis. Existing Gaussian splatting methods either fail to model near-field specular reflections or rely on explicit ray tracing at substantial computational cost. We present Ref-DGS, a reflective dual Gaussian splatting framework that addresses this trade-off by decoupling surface reconstruction from specular reflection within an efficient rasterization-based pipeline. Ref-DGS introduces a dual Gaussian scene representation consisting of geometry Gaussians and complementary local reflection Gaussians that capture near-field specular interactions without explicit ray tracing, along with a global environment reflection field for modeling far-field specular reflections. To predict specular radiance, we further propose a lightweight, physically-aware specular adaptive mixing shader that fuses global and local specular features. Experiments demonstrate that Ref-DGS achieves state-of-the-art performance on reflective scenes while training substantially faster than ray-based Gaussian methods.

Abstract

Neural Rendering

Explore similar work

Ref-DGS: Reflective Dual Gaussian Splatting

RGS: Reflection-aware Gaussian Splatting via Learning Geometry Continuity for Reflective Objects

NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting