RefracGS: Novel View Synthesis Through Refractive Water Surfaces with 3D Gaussian Ray Tracing

Yiming Shao; Qiyu Dai; Chong Gao; Guanbin Li; Yequan Wang; He Sun; Qiong Zeng; Baoquan Chen

cs.CVMar 23, 2026

RefracGS: Novel View Synthesis Through Refractive Water Surfaces with 3D Gaussian Ray Tracing

Authors: Yiming Shao, Qiyu Dai, Chong Gao, Guanbin Li, Yequan Wang, He Sun, Qiong Zeng, Baoquan Chen,

Organizations: Shandong University, China · Peking University, Beijing, China · Sun Yat-sen University, China · Shenzhen Loop Area Institute, China ·

Abstract

Novel view synthesis (NVS) through non-planar refractive surfaces presents fundamental challenges due to severe, spatially varying optical distortions. While recent representations like NeRF and 3D Gaussian Splatting (3DGS) excel at NVS, their assumption of straight-line ray propagation fails under these conditions, leading to significant artifacts. To overcome this limitation, we introduce RefracGS, a framework that jointly reconstructs the refractive water surface and the scene beneath the interface. Our key insight is to explicitly decouple the refractive boundary from the target objects: the refractive surface is modeled via a neural height field, capturing wave geometry, while the underlying scene is represented as a 3D Gaussian field. We formulate a refraction-aware Gaussian ray tracing approach that accurately computes non-linear ray trajectories using Snell's law and efficiently renders the underlying Gaussian field while backpropagating the loss gradients to the parameterized refractive surface. Through end-to-end joint optimization of both representations, our method ensures high-fidelity NVS and view-consistent surface recovery. Experiments on both synthetic and real-world scenes with complex waves demonstrate that RefracGS outperforms prior refractive methods in visual quality, while achieving 15x faster training and real-time rendering at 200 FPS. The project page for RefracGS is available at https://yimgshao.github.io/refracgs/.

Figures & tables

Figure 2: Overview of the RefracGS pipeline . A proxy refractive - surface mesh is first extracted from the neural height field, after which camera rays intersect the mesh using a hierarchical global - to - local strategy to obtain intersection points and normals. The refracted directions are then computed with Snell’s law, and the resulting rays traverse the underlying scene before being rendered with Gaussian ray tracing.

Dataset	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Train $\downarrow$	FPS $\uparrow$	RMSE(cm) $\downarrow$
NeRFrac Real	Mip-NeRF	10.385	0.221	0.916	$\sim$ 15 h	$<$ 1	-
	TensoRF	22.973	0.809	0.139	11 min	$<$ 1	-
	Plenoxels	14.867	0.389	0.631	9 min	61	-
	3DGS	27.711	0.864	0.130	3 min	857	-
	3DGRT	27.732	0.842	0.164	16 min	157	-
	WSGS †	22.449	0.679	0.279	106 min	129	-

Table 1: Quantitative experimental results. Our method achieves state-of-the-art novel view synthesis quality and water surface reconstruction accuracy, while maintaining fast training speed and real-time rendering frame rates. (

\dagger

indicates methods specifically designed for refractive scenes.)

Panel of Figure 5 — Figure 5: Qualitative comparison with existing methods. On the NeRFrac dataset, our method preserves fine textures and high-frequency details better than NeRFrac. On the more challenging RefracGS dataset—featuring wider viewpoint coverage and stronger refractive effects—our method consistently produces high-fidelity renderings, whereas existing methods fail to learn accurate scene representations under such conditions, leading to noticeable artifacts and incorrect novel-view synthesis.

Method\|Metric	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Train $\downarrow$	FPS $\uparrow$
(a) w/o Water Height Map	27.782	0.869	0.168	22 min	57
(b) w/o Normal Smoothing	32.643	0.928	0.133	11 min	232
(c) w/o Recursive Intersection	32.966	0.929	0.132	20 min	218
(d) w/o Opacity Loss	31.927	0.920	0.125	12 min	224
Full Model	32.876	0.929	0.132	11 min	223

Table 2: Ablation study results. The ablations show that the components of our method collectively provide a favorable balance between reconstruction quality and computational efficiency.

Method\|Metric	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
NeRFrac	14.950	0.380	0.563
Ours	23.151	0.869	0.110

Table 3: Metrics of water removal quality. Our method yields more reliable results compared with NeRFrac.

Figure 6: Water removal and surface editting Benefiting from our physically-based modeling approach, our method naturally decouples the water surface geometry from underwater objects. This capability enables the removal of surface refraction, as well as water surface editing using arbitrary meshes. Crucially, our precise refraction modeling allows these refraction-removed views to serve directly as inputs for methods like PGSR to achieve complete geometric reconstruction of the underlying scene.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure A3 : Boundary cases of single-refraction. (a) When cameras are fully above the water, the rays are downward or slightly oblique, single refraction holds. (b) When cameras are near water, extreme grazing angles cause multiple refractions. (c) Submerged cameras are more complicated and are beyond models of our method.

Scene	Method	PSNR↑	SSIM↑	LPIPS↓	Train↓	FPS↑
Plant	Mip-NeRF	10.350	0.322	0.945	$\sim$ 15 h	$<$ 1
	TensoRF	21.857	0.798	0.138	$\sim$ 11 min	$<$ 1
	Plenoxels	14.491	0.378	0.655	$\sim$ 9 min	$\sim$ 60
	3DGS	28.520	0.801	0.198	3 min	1019
	3DGRT	27.579	0.845	0.155	13 min	143
	3DGS*	27.007	0.870	0.136	2 min	932

Appendix

Table A1 : Quantitative results on the Nerfrac real dataset. (* indicates models trained for 15,000 iterations;

\sim

indicates that the runtime was not precisely recorded for each scene, but rather estimated.

\dagger

indicates methods specifically designed for refractive scenes.)

Scene	Method	PSNR↑	SSIM↑	LPIPS↓	Train↓	FPS↑
Primary Sine	Mip-NeRF	12.482	0.402	0.852	$\sim$ 15 h	$<$ 1
	TensoRF	16.301	0.527	0.614	$\sim$ 11 min	$<$ 1
	Plenoxels	13.390	0.403	0.810	$\sim$ 12 min	$\sim$ 70
	3DGS	21.279	0.705	0.311	3 min	693
	3DGRT	21.135	0.663	0.390	29 min	80
	3DGS*	21.459	0.726	0.346	2 min	686

Appendix

Table A2 : Quantitative results on the Nerfrac synthetic dataset. (* indicates models trained for 15,000 iterations;

\sim

indicates that the runtime was not precisely recorded for each scene, but rather estimated.

\dagger

indicates methods specifically designed for refractive scenes.)

Scene	Method	PSNR↑	SSIM↑	LPIPS↓	Train↓	FPS↑	RMSE(cm)↓
Toys	Mip-NeRF	12.122	0.363	0.855	$\sim$ 15 h	$<$ 1	-
	TensoRF	12.353	0.255	0.621	$\sim$ 13 min	$<$ 1	-
	Plenoxels	12.616	0.286	0.717	$\sim$ 12 min	$\sim$ 20	-
	3DGS	15.230	0.297	0.525	5 min	526	-
	3DGRT	16.027	0.304	0.566	16 min	171	-
	3DGS*	15.085	0.303	0.528	2 min	425	-

Appendix

Table A3 : Quantitative results on the RefracGS dataset. (* indicates models trained for 15,000 iterations;

\sim

indicates that the runtime was not precisely recorded for each scene, but rather estimated.

\dagger

indicates methods specifically designed for refractive scenes.)

Scene	Method	PSNR↑	SSIM↑	LPIPS↓	Train↓	FPS↑	RMSE(cm)↓
Toys	Mip-NeRF	12.637	0.290	0.833	$\sim$ 15 h	$<$ 1	-
	TensoRF	13.657	0.287	0.519	$\sim$ 13 min	$<$ 1	-
	Plenoxels	12.261	0.271	0.701	$\sim$ 12 min	$\sim$ 20	-
	3DGS	18.840	0.536	0.352	3 min	625	-
	3DGRT	19.646	0.562	0.394	23 min	114	-
	3DGS*	19.398	0.559	0.333	2 min	621	-

Appendix

Table A4 : Quantitative results of different methods on RefracGS dataset scenes under the additional nine-view setting as discussed in Sec. 0.C.3 . (* indicates models trained for 15,000 iterations;

\sim

indicates that the runtime was not precisely recorded for each scene, but rather estimated.

\dagger

indicates methods specifically designed for refractive scenes.)

Panel of Figure 14 — Figure A4 : Qualitative results on the Nerfrac real dataset.

Panel of Figure 15 — Figure A5 : Qualitative results on the Nerfrac synthetic dataset.

Panel of Figure 16 — Figure A6 : Qualitative results on the RefracGS dataset.

Panel of Figure 17 — Figure A7 : Qualitative results of different methods on RefracGS dataset scenes under the additional nine-view setting as discussed in Sec. 0.C.3 .

Novel View Synthesis

Novel View Synthesis creates images of a scene from camera viewpoints not present in the input, using learned geometry and appearance models.

See all 289 papers

Gaussian Splatting with Reflections Guided by What is Seen
Oct 9, 2026Yiming Liang, Tianyu Xiao, Hiroshi IshikawaNeural Rendering Novel View Synthesis
Department of Computer Science and Communications Engineering, Waseda University, Japan
Ref-DGS: Reflective Dual Gaussian Splatting
Mar 8, 2026Ningjing Fan, Yiqun Wang, Dong-Ming Yan +13D Gaussian Splatting 3D Reconstruction
Chongqing University, China · MAIS, Institute of Automation, Chinese Academy of Sciences and UCAS, China · King Abdullah University of Science and Technology (KAUST), Saudi Arabia
RGS: Reflection-aware Gaussian Splatting via Learning Geometry Continuity for Reflective Objects
Sep 16, 2026Xiaobiao Du, Yida Wang, Cheng Bi +23D Gaussian Splatting 3D Reconstruction
University of Technology Sydney · Li Auto Inc. · Adelaide University

Oct 9, 2026J/K move · Enter open · S save

Gaussian Splatting with Reflections Guided by What is Seen

Yiming Liang, Tianyu Xiao, Hiroshi Ishikawa

Department of Computer Science and Communications Engineering, Waseda University, Japan

Neural Rendering Novel View Synthesis 3D Gaussian Splatting Physically Based Rendering

Open paper AskPDF arXiv

While novel view synthesis methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have shown impressive performance on common scenes, they often struggle with capturing view-dependent specular effects, particularly near-field reflections. Existing methods utilize environment maps, which only represent lights from infinity, or environment Gaussians, which rely solely on indirect supervision from reflections, are insufficient for accurately modeling such effects. Here, we propose a framework based on 2D Gaussian Splatting (2DGS) that leverages directly observable environment information in input images. Our approach also integrates a hybrid reflection appearance representation and view-dependent specular modeling. Experiments on public benchmarks and a newly curated near-field reflection dataset demonstrate that our method achieves superior rendering quality and geometric accuracy.

Abstract

Novel View Synthesis

Explore similar work

Gaussian Splatting with Reflections Guided by What is Seen

Ref-DGS: Reflective Dual Gaussian Splatting

RGS: Reflection-aware Gaussian Splatting via Learning Geometry Continuity for Reflective Objects