PhyGaP: Physically-Grounded Gaussians with Polarization Cues

Jiale Wu; Xiaoyang Bai; Zongqi He; Weiwei Xu; Yifan Peng

cs.CVMar 14, 2026

PhyGaP: Physically-Grounded Gaussians with Polarization Cues

Authors: Jiale Wu, Xiaoyang Bai, Zongqi He, Weiwei Xu, Yifan Peng

Organizations: The University of Hong Kong · Zhejiang University

Abstract

Recent advances in 3D Gaussian Splatting (3DGS) have demonstrated great success in modeling reflective 3D objects and their interaction with the environment via deferred rendering (DR). However, existing methods often struggle with correctly reconstructing physical attributes such as albedo and reflectance, and therefore they do not support high-fidelity relighting. Observing that this limitation stems from the lack of shape and material information in RGB images, we present PhyGaP, a physically-grounded 3DGS method that leverages polarization cues to facilitate precise reflection decomposition and visually consistent relighting of reconstructed objects. Specifically, we design a polarimetric deferred rendering (PolarDR) process to model polarization by reflection, and a self-occlusion-aware environment map building technique (GridMap) to resolve indirect lighting of non-convex objects. We validate on multiple synthetic and real-world scenes, including those featuring only partial polarization cues, that PhyGaP not only excels in reconstructing the appearance and surface normal of reflective 3D objects (~2 dB in PSNR and 45.7% in Cosine Distance better than existing RGB-based methods on average), but also achieves state-of-the-art inverse rendering and relighting capability. Our code will be released soon.

Figures & tables

Figure 2: Overview of the PhyGaP pipeline. We represent physically-grounded attributes such as roughness, albedo, IoR and surface normal with 2DGS, and render them into Stokes values via the PolarDR process. Furthermore, we design the GridMap technique to tackle self-occlusion of nonconvex objects. By utilizing polarization cues, we achieve accurate , explicit and disentangled representation of object albedo, diffuse reflection and specular reflection in PhyGaP.

Figure 3: Overview of GridMap. (a) We sample a set of anchor cameras on the object bounding box. (b) For each anchor camera, we ray trace in all directions to blend object color with the global environment map. (c) The resulting local environment map enables more accurate diffuse irradiance computation.

	PSNR (dB) $\uparrow$									Cosine Distance (CD) $\downarrow$
	PANDORA		RMVP		SMVP			Mitsuba3		RMVP		SMVP			Mitsuba3
	owl	vase	frog	dog	squirrel	snail	david	matpre.	teapot	frog	dog	squirrel	snail	david	matpre.	teapot
R3DG	24.49	25.35	26.56	17.77	19.71	26.97	22.93	25.41	28.30	0.0679	0.1376	0.0916	0.0473	0.1054	0.0795	0.0326
GS-IR	23.76	24.40	31.07	33.33	18.59	27.92	20.47	16.43	21.77	0.0954	0.1523	0.0931	0.0909	0.1005	0.1130	0.0847
GIR	23.81	23.37	28.26	41.10	20.88	23.67	22.78	24.41	27.10	0.0603	0.1674	0.0536	0.0491	0.1307	0.0616	0.0213
3DGS-DR	24.20	26.94	34.68	39.59	20.63	28.46	24.72	24.33	29.07	0.0933	0.1420	0.0484	0.0462	0.1353	0.1069	0.0325

Table 1: Quantitative results of different methods on novel view synthesis ( left ) and surface normal reconstruction ( right ). Best results are highlighted as 1st , 2nd , and 3rd . This visualization applies to all following quantitative assessments.

Figure 4: Visualization of reconstructed surface normal for synthetic ( snail ) and real-world ( frog ) objects.

Figure 5: Qualitative comparison on estimated environment maps.

Figure 6: Comparison on reflection decomposition. Note that PolGS and Ref-Gaussian do not explicitly model object albedo.

Method	Envmap (PSNR $\uparrow$ )			Relighting
Method	Teapot	Matpre.	David	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
R3DG	-	-	-	11.42	0.916	0.0754
GS-IR	6.70	6.70	6.72	16.90	0.957	0.0359
GIR	10.30	10.73	10.01	18.02	0.960	0.0327
3DGS-DR	6.71	8.26	6.71	-	-	-
Ref-Gaussian	7.51	9.86	8.28	-	-	-

Table 2: Quantitative evaluation of environment map reconstruction and relighting. Left : PSNR of optimized envmaps for different methods on the Mitsuba-rendered dataset. Right : Polarimetric evaluation of david relighted with the sunset envmap. Ablation results are also included. “No PDR” is realized by setting

\lambda_{1}=0

Figure 7: Relighting results from different methods. The environment maps are presented at the top. Note that R3DG suffers severe overexposure in the rightmost Brown Photostudio environment.

Figure 8: Examples of our real-world captured data and ( from top-right, clockwise ) the reconstructed surface normal, specular reflection, diffuse reflection, and final rendered results.

Figure 9: Ablation results with and without PolarDR (PDR) and/or GridMap (GM). We observe visible differences in color correctness, albedo quality, and environment map smoothness.

Figure 10: Real-word acquisition samples and reconstruction results. Note that reflection patterns captured by two cameras differ, due to the angular disparity between two LPs. From top to bottom: buddha-room , buddha-corridor , bull-corridor , and figurine-garden .

Methods	PANDORA		RMVP		SMVP			Mitsuba3
Methods	owl	vase	frog	dog	squirrel	snail	david	matpre.	teapot
R3DG	0.935 / 0.059	0.962 / 0.052	0.983 / 0.032	0.955 / 0.029	0.915 / 0.071	0.932 / 0.098	0.939 / 0.062	0.968 / 0.047	0.945 / 0.147
GS-IR	0.915 / 0.076	0.929 / 0.091	0.983 / 0.028	0.996 / 0.011	0.898 / 0.088	0.939 / 0.096	0.921 / 0.085	0.903 / 0.085	0.882 / 0.179
GIR	0.931 / 0.058	0.963 / 0.053	0.987 / 0.023	0.999 / 0.003	0.936 / 0.058	0.940 / 0.090	0.949 / 0.051	0.966 / 0.051	0.941 / 0.149
3DGS-DR	0.938 / 0.050	0.971 / 0.040	0.991 / 0.018	0.999 / 0.006	0.933 / 0.057	0.947 / 0.075	0.952 / 0.043	0.968 / 0.046	0.947 / 0.135
Ref-Gaussian	0.924 / 0.064	0.972 / 0.039	0.988 / 0.028	0.999 / 0.008	0.897 / 0.084	0.941 / 0.079	0.950 / 0.049	0.971 / 0.043	0.949 / 0.135

Table 3: SSIM

\uparrow

( left ) and LPIPS

\downarrow

( right ) of different methods on novel view synthesis. Best and second best results are indicated by bold and underline fonts respectively.

Figure 11: More qualitative comparison on estimated environment maps. Ground truths for these two scenes ( frog and owl ) are not available.

Methods	RMVP		SMVP			Mitsuba3
Methods	frog	dog	squirrel	snail	david	matpre.	teapot
R3DG	17.46	24.86	19.90	13.22	21.12	18.22	11.62
GS-IR	21.35	27.31	21.13	20.86	21.62	22.45	19.26
GIR	15.98	29.74	13.91	14.04	24.18	13.61	7.90
3DGS-DR	19.89	24.96	12.01	13.03	24.06	20.64	9.21
Ref-Gaussian	14.01	27.43	17.12	8.35	21.23	9.01	4.97

Table 4: MAE

\downarrow

of different methods on surface normal reconstruction. Best and second best results are indicated by bold and underline fonts respectively.

Figure 12: Meshes reconstructed by PhyGaP and PolGS. Results for GIR and GS-IR are not presented since their meshes exhibit significantly worse quality due to floating Gaussian points.

Figure 13: Relighting results using SH-based indrect light modeling and our GridMap.

Scene	Chamfer Distance $\downarrow$ (mm)
Scene	Ours	PolGS	GIR	GS-IR
david	6.535	6.537	62.075	77.032
squirrel	9.527	6.604	13.118	28.148
snail	8.627	9.652	21.847	21.288

Table 5: Chamfer Distance on the SMVP dataset. Optimal results are indicated by bold font.

Figure 14: Cosine distance (CD $\downarrow$ , left axis) and PSNR $\uparrow$ (right axis) across $\lambda$ choices. The reported values are indicated by triangles.

Figure 15: Left ground-truth colors for object body and stand . Mid albedo reconstructed using GridMap. Object concavity and self-occlusion result in reddish artifacts. Right albedo reconstructed using near-surface anchor placement. More severe color shifts are observed due to anchors being placed inside the object.

Figure 16: Additional reconstruction results with our model. From top to bottom: david , snail , matpre. , and teapot .

3D Scene Relighting

See all 21 papers

PolGS++: Physically-Guided Polarimetric Gaussian Splatting for Fast Reflective Surface Reconstruction
Mar 11, 2026Yufei Han, Chu Zhou, Youwei Lyu +63D Reconstruction 3D Gaussian Splatting
Beijing University of Posts and Telecommunications, China · National Institute of Informatics, Japan · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., Shanghai, China +2
R3GW: Relightable 3D Gaussians for Outdoor Scenes in the Wild
Mar 3, 2026Margherita Lea Corona, Wieland Morgenstern, Peter Eisert +13D Scene Relighting 3D Gaussian Splatting
Fraunhofer Heinrich Hertz Institute, Berlin, Germany · Humboldt Universität zu Berlin, Berlin, Germany
SSD-GS: Scattering and Shadow Decomposition for Relightable 3D Gaussian Splatting
Apr 14, 2026Iris Zheng, Guojun Tang, Alexander Doronin +2Image Relighting Physically Based Rendering
Victoria University of Wellington

Mar 11, 2026J/K move · Enter open · S save

PolGS++: Physically-Guided Polarimetric Gaussian Splatting for Fast Reflective Surface Reconstruction

Yufei Han, Chu Zhou, Youwei Lyu, Qi Chen+5

Beijing University of Posts and Telecommunications, China · National Institute of Informatics, Japan · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., Shanghai, China+2

3D Reconstruction 3D Gaussian Splatting Physically Based Rendering Surface Normal Estimation

Open paper AskPDF arXiv

Accurate reconstruction of reflective surfaces remains a fundamental challenge in computer vision, with broad applications in real-time virtual reality and digital content creation. Although 3D Gaussian Splatting (3DGS) enables efficient novel-view rendering with explicit representations, its performance on reflective surfaces still lags behind implicit neural methods, especially in recovering fine geometry and surface normals. To address this gap, we propose PolGS++, a physically-guided polarimetric Gaussian Splatting framework for fast reflective surface reconstruction. Specifically, we integrate a polarized BRDF (pBRDF) model into 3DGS to explicitly decouple diffuse and specular components, providing physically grounded reflectance modeling and stronger geometric cues for reflective surface recovery. Furthermore, we introduce a depth-guided visibility mask acquisition mechanism that enables angle-of-polarization (AoP)-based tangent-space consistency constraints in Gaussian Splatting without costly ray-tracing intersections. This physically guided design improves reconstruction quality and efficiency, requiring only about 10 minutes of training. Extensive experiments on both synthetic and real-world datasets validate the effectiveness of our method.

Abstract

3D Scene Relighting

Explore similar work

PolGS++: Physically-Guided Polarimetric Gaussian Splatting for Fast Reflective Surface Reconstruction

R3GW: Relightable 3D Gaussians for Outdoor Scenes in the Wild

SSD-GS: Scattering and Shadow Decomposition for Relightable 3D Gaussian Splatting