3D imaging of specular surfaces remains challenging in real-world scenarios, such as in-line inspection or hand-held scanning, requiring fast and accurate measurement of complex geometries. Optical metrology techniques such as deflectometry achieve high accuracy but typically rely on multi-shot acquisition, making them unsuitable for dynamic environments. Fourier-based single-shot approaches alleviate this constraint, yet their performance deteriorates when measuring surfaces with high spatial frequency structure or large curvature. Alternatively, polarimetric 3D imaging in computer vision operates in a single-shot fashion and exhibits robustness to geometric complexity. However, its accuracy is fundamentally limited by the orthographic imaging assumption. In this paper, we propose a physics-informed deep learning framework for single-shot 3D imaging of complex specular surfaces. Polarization cues provide orientation priors that assist in interpreting geometric information encoded by structured illumination. These complementary cues are processed through a dual-encoder architecture with mutual feature modulation, allowing the network to resolve their nonlinear coupling and directly infer surface normals. The proposed method achieves accurate and robust normal estimation in single-shot with fast inference, enabling practical 3D imaging of complex specular surfaces.
Figures & tables
Figure 1: Overview of the proposed physics-informed learning framework. Polarimetric inputs, including Stokes parameters and DoLP, are first processed by U-Nets to obtain coarse depth and normal estimates. Then the coarse correspondence map is analytically calculated. These physics priors are processed separately through two encoder branches to extract modality-specific features Feature-wise Linear Modulation layers are employed to adaptively fuse polarimetric cues and geometric correspondence features, enabling robust normal estimation.
Figure 2: Real experimental prototype. The prototype consists of a polarization camera and an unpolarized display. The polarization cues encode the surface orientation information. Decoding the reflected deformed stucture light pattern reveals the surface geometrical cues.
Figure 3: Normal comparison with different polarimetric 3D imaging methods. From left to right: measurement image, ground truth, the classical single-view physical SfP method of Smith [ 18 ] , the previous multi-modal physics-based method Wang [ 19 ] , the learning-based method DeepSfP [ 2 ] , and the proposed method. The Mean Angular Error (MAE) for each method is reported. Physics-based methods suffer from large reconstruction errors due to inherent modeling issues, while the learning-based DeepSfP improves the performance but is still limited by the orthographic imaging assumption. In contrast, the proposed method leverages complementary physical priors to achieve significantly improved accuracy, reducing the MAE to 0.79∘ .
Figure 4: Error maps of surface normal estimation. (a) Ideal SfP where the normal ambiguity is manually resolved, leaving the orthographic imaging assumption as the primary error source. The error increases toward the image periphery. (b) DeepSfP [ 2 ] exhibits a similar spatial error distribution, indicating that learning-based SfP methods remain constrained by the same physical limitation. (c) The proposed method significantly reduces the reconstruction error across the entire surface by incorporating complementary physical priors, achieving a Mean Angular Error (MAE) of 0.79∘ .
Figure 5: Quantitative evaluation on real measurement. We perform quantitative evaluation using a precisely manufactured ball bearing with a known radius of 1 inch. Our proposed method achieves an MAE of 1.56∘ , outperforming previous methods. It is worth noting that while the previous multi-modal physics-based method by Wang [ 19 ] also demonstrated reconstruction of a ball bearing, in our experiment, the object has smaller size, which leads to larger angular coverage and more severe spatial pattern deformation, making single-shot correspondence retrieval significantly more challenging and leading to degraded performance of the physics-based method.
Figure 6: Qualitative evaluation on complex real object. (a) Surface normal estimated using the previous physics-based method [ 19 ] with multi-shot sequential captures. (b) Surface normal estimated by the proposed method with single-shot capture. The proposed method produces a more consistent normal field, especially in the cheek region of the horse.
The School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, China · Research Institute of Highway, Ministry of Transport, Beijing, China
Beijing University of Posts and Telecommunications, China · National Institute of Informatics, Japan · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., Shanghai, China +2