3D imaging of specular surfaces remains challenging in real-world scenarios, such as in-line inspection or hand-held scanning, requiring fast and accurate measurement of complex geometries. Optical metrology techniques such as deflectometry achieve high accuracy but typically rely on multi-shot acquisition, making them unsuitable for dynamic environments. Fourier-based single-shot approaches alleviate this constraint, yet their performance deteriorates when measuring surfaces with high spatial frequency structure or large curvature. Alternatively, polarimetric 3D imaging in computer vision operates in a single-shot fashion and exhibits robustness to geometric complexity. However, its accuracy is fundamentally limited by the orthographic imaging assumption. In this paper, we propose a physics-informed deep learning framework for single-shot 3D imaging of complex specular surfaces. Polarization cues provide orientation priors that assist in interpreting geometric information encoded by structured illumination. These complementary cues are processed through a dual-encoder architecture with mutual feature modulation, allowing the network to resolve their nonlinear coupling and directly infer surface normals. The proposed method achieves accurate and robust normal estimation in single-shot with fast inference, enabling practical 3D imaging of complex specular surfaces.
Figures & tables
Figure 1: Overview of the proposed physics-informed learning framework. Polarimetric inputs, including Stokes parameters and DoLP, are first processed by U-Nets to obtain coarse depth and normal estimates. Then the coarse correspondence map is analytically calculated. These physics priors are processed separately through two encoder branches to extract modality-specific features Feature-wise Linear Modulation layers are employed to adaptively fuse polarimetric cues and geometric correspondence features, enabling robust normal estimation.
Figure 2: Real experimental prototype. The prototype consists of a polarization camera and an unpolarized display. The polarization cues encode the surface orientation information. Decoding the reflected deformed stucture light pattern reveals the surface geometrical cues.
Figure 3: Normal comparison with different polarimetric 3D imaging methods. From left to right: measurement image, ground truth, the classical single-view physical SfP method of Smith [ 18 ] , the previous multi-modal physics-based method Wang [ 19 ] , the learning-based method DeepSfP [ 2 ] , and the proposed method. The Mean Angular Error (MAE) for each method is reported. Physics-based methods suffer from large reconstruction errors due to inherent modeling issues, while the learning-based DeepSfP improves the performance but is still limited by the orthographic imaging assumption. In contrast, the proposed method leverages complementary physical priors to achieve significantly improved accuracy, reducing the MAE to 0.79∘ .
Figure 4: Error maps of surface normal estimation. (a) Ideal SfP where the normal ambiguity is manually resolved, leaving the orthographic imaging assumption as the primary error source. The error increases toward the image periphery. (b) DeepSfP [ 2 ] exhibits a similar spatial error distribution, indicating that learning-based SfP methods remain constrained by the same physical limitation. (c) The proposed method significantly reduces the reconstruction error across the entire surface by incorporating complementary physical priors, achieving a Mean Angular Error (MAE) of 0.79∘ .
Figure 5: Quantitative evaluation on real measurement. We perform quantitative evaluation using a precisely manufactured ball bearing with a known radius of 1 inch. Our proposed method achieves an MAE of 1.56∘ , outperforming previous methods. It is worth noting that while the previous multi-modal physics-based method by Wang [ 19 ] also demonstrated reconstruction of a ball bearing, in our experiment, the object has smaller size, which leads to larger angular coverage and more severe spatial pattern deformation, making single-shot correspondence retrieval significantly more challenging and leading to degraded performance of the physics-based method.
Figure 6: Qualitative evaluation on complex real object. (a) Surface normal estimated using the previous physics-based method [ 19 ] with multi-shot sequential captures. (b) Surface normal estimated by the proposed method with single-shot capture. The proposed method produces a more consistent normal field, especially in the cheek region of the horse.
Underwater optical imaging is severely hindered by scattering, but polarization imaging offers the unique dual advantages of descattering and shape-from-polarization (SfP) 3D reconstruction. To exploit these advantages, this paper proposes UD-SfPNet, an underwater descattering shape-from-polarization network that leverages polarization cues for improved 3D surface normal prediction. The framework jointly models polarization-based image descattering and SfP normal estimation in a unified pipeline, avoiding error accumulation from sequential processing and enabling global optimization across both tasks. UD-SfPNet further incorporates a novel color embedding module to enhance geometric consistency by exploiting the relationship between color encodings and surface orientation. A detail enhancement convolution module is also included to better preserve high-frequency geometric details that are lost under scattering. Experiments on the MuS-Polar3D dataset show that the proposed method significantly improves reconstruction accuracy, achieving a mean surface normal angular error of 15.12∘ (the lowest among compared methods). These results confirm the efficacy of combining descattering with polarization-based shape inference, and highlight the practical significance and potential applications of UD-SfPNet for optical 3D imaging in challenging underwater environments. The code is available at https://github.com/WangPuyun/UD-SfPNet.
Recovering surface normals from single view polarization images in complex media remains challenging. This paper focuses on ice as a representative complex medium, where intricate light matter interactions lead to a nonlinear mapping between polarization observations and surface normals. To address this, a structure-aware polarization prior based on autocorrelation functions is proposed to capture the local spatial consistency of AoLP. Building on this, a dual-branch network (IceSfP) is designed to integrate raw polarization features with priors via cross modal attention and multi-scale feature fusion, enabling accurate surface normal estimation under complex media conditions. To evaluate the method, the first real-world ice SfP dataset is constructed. Experimental results show that the method outperforms existing approaches across all metrics, achieving a MAE of 16.01 deg, which is 2.74 deg lower than the second-best method. The framework provides a generalizable solution for high-precision geometric perception in complex media.
Kaimin Yu, Puyun Wang, Huayang He +1
The School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, China · Research Institute of Highway, Ministry of Transport, Beijing, China
Accurate reconstruction of reflective surfaces remains a fundamental challenge in computer vision, with broad applications in real-time virtual reality and digital content creation. Although 3D Gaussian Splatting (3DGS) enables efficient novel-view rendering with explicit representations, its performance on reflective surfaces still lags behind implicit neural methods, especially in recovering fine geometry and surface normals. To address this gap, we propose PolGS++, a physically-guided polarimetric Gaussian Splatting framework for fast reflective surface reconstruction. Specifically, we integrate a polarized BRDF (pBRDF) model into 3DGS to explicitly decouple diffuse and specular components, providing physically grounded reflectance modeling and stronger geometric cues for reflective surface recovery. Furthermore, we introduce a depth-guided visibility mask acquisition mechanism that enables angle-of-polarization (AoP)-based tangent-space consistency constraints in Gaussian Splatting without costly ray-tracing intersections. This physically guided design improves reconstruction quality and efficiency, requiring only about 10 minutes of training. Extensive experiments on both synthetic and real-world datasets validate the effectiveness of our method.
Yufei Han, Chu Zhou, Youwei Lyu +6
Beijing University of Posts and Telecommunications, China · National Institute of Informatics, Japan · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., Shanghai, China +2