cs.CVMar 5, 2026

Revisiting Shape from Polarization in the Era of Vision Foundation Models

Authors: Chenhao Li, Taishi Ono, Takeshi Uemori, Yusuke Moriuchi

Organizations: Sony Semiconductor Solutions Corporation

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We show that, with polarization cues, a lightweight model trained on a small dataset can outperform RGB-only vision foundation models (VFMs) in single-shot object-level surface normal estimation. Shape from polarization (SfP) has long been studied due to the strong physical relationship between polarization and surface geometry. Meanwhile, driven by scaling laws, RGB-only VFMs trained on large datasets have recently achieved impressive performance and surpassed existing SfP methods. This situation raises questions about the necessity of polarization cues, which require specialized hardware and have limited training data. We argue that the weaker performance of prior SfP methods does not come from the polarization modality itself, but from domain gaps. These domain gaps mainly arise from two sources. First, existing synthetic datasets use limited and unrealistic 3D objects, with simple geometry and random texture maps that do not match the underlying shapes. Second, real-world polarization signals are often affected by sensor noise, which is not well modeled during training. To address the first issue, we render a high-quality polarization dataset using 1,954 3D-scanned real-world objects. We further incorporate pretrained DINOv3 priors to improve generalization to unseen objects. To address the second issue, we introduce polarization sensor-aware data augmentation that better reflects real-world conditions. With only 40K training scenes, our method significantly outperforms both state-of-the-art SfP approaches and RGB-only VFMs. Extensive experiments show that polarization cues enable a 33x reduction in training data or an 8x reduction in model parameters, while still achieving better performance than RGB-only counterparts.

Figures & tables

Figure 1: Our method surpasses the previous best SfP approach (SfPUEL [ 48 ] ), a leading discriminative VFM (MoGe2 [ 69 ] ), a generative VFM (StableNormal [ 74 ] ), and a commercial inverse rendering tool (SwitchLight3 [ 6 ] ). Moreover, the benefit of using polarization cues is clear by comparing with our RGB-only ablation. The numbers shown below each method indicate frames per second (FPS) and mean angular error (MAE). Inference speed for all models is tested on a V100 GPU with a resolution of 512 × 612 and FP16 precision. 2 2 2 We used a commercially available Schleich dinosaur figurine purchased by the authors. No endorsement by the manufacturer is implied.
DatasetImage type#ScenesObject type#Objects
DeepSfP [ 1 ]Real263Real-world25
SfPW [ 39 ]Real522Real-world–
Kondo et al. [ 35 ]Synthetic44KManually designed–
SfPUEL [ 48 ]Synthetic20KManually designed244
DTC-p (Ours)Synthetic40K3D scanned1,954
Table 1: Comparison of SfP datasets
Figure 2: Data augmentation pipeline and model architecture.
Figure 3: Visualization of a plastic ball in real and synthetic data with noise simulation. In real-world measurements, AoLP is consistently noisy due to sensor and acquisition artifacts. In contrast, rendered AoLP appears overly clean because of the idealized sensor model. Directly injecting noise into RGB or AoLP is not realistic (the noise level is amplified here for visualization). Instead, applying augmentation before polarization signal processing better matches real noise characteristics: RGB domain is less affected and AoLP noise is concentrated in regions with rapid AoLP direction changes.
Figure 4: Qualitative comparisons on Our real w/ GT .
Figure 5: Qualitative comparisons on Our real w/o GT (Please zoom in for details).
MethodTraining DataMAE ↓\downarrowAcc ↑\uparrow
 Our real w/ GTPISR [ 8 ]SfPUEL [ 48 ]Avg<11.25∘<11.25^{\circ}<22.50∘<22.50^{\circ}
Ours40K12.63 ∘12.32 ∘12.76 ∘12.54 ∘58.2%88.5%
w/o polar40K17.82 ∘22.99 ∘15.90 ∘18.43 ∘32.8%70.9%
w/o aug40K14.26 ∘15.93 ∘13.88 ∘14.55 ∘48.2%83.5%
w/ post aug40K13.31 ∘15.31 ∘13.31 ∘13.84 ∘52.2%85.4%
w/o DINO40K14.83 ∘15.33 ∘14.99 ∘15.03 ∘45.2%82.3%
Table 2: Quantitative comparison on three real-world datasets. SwitchLight3 [ 6 ] is a commercial tool and is not included in the ranking.
Figure 6: Out of distribution tests. Our training dataset does not contain transparent objects or conductors. Our full model is still robust to these unseen objects, while the ablated versions show various types of degradations.
Figure 7: Ablation study of number of training scenes, model size, and number of objects and environment maps used for rendering the training data. MAE of synthetic data are calculated on the test part of DTC-p. MAE of real data are calculated using three datasets. Log scaling is applied to horizontal axis for better visualization.
Figure 8: Failure cases. (a) Although the model recovers fine-grained geometry of individual objects, it fails to correctly understand background structures such as walls or buildings. (b) When the polarization signal is strong (bottom), the model produces high-fidelity geometry. However, when the target object is nearly unpolarized (top), the DoLP is close to zero and the AoLP becomes extremely noisy, showing no noticeable improvement over RGB-only VFMs.
More in this topic

Surface Normal Estimation

Surface Normal Estimation recovers the orientation of visible 3D surfaces from images and other sensor data for shape understanding and reconstruction.

See all 25 papers

Explore similar work

CardsList
Sep 8, 2026

DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision

Polarimetric vision is gaining increasing attention because it provides physical cues about scene shape, material, and reflection that are difficult to recover from RGB alone. Recent work has therefore explored predicting polarization directly from conventional RGB images; however, the fidelity of these methods strongly depends on the polarization supervision used for training. Most existing datasets rely on Division-of-Focal-Plane (DoFP) cameras with four spatially interleaved analyzer orientations, which provide limited angular redundancy and introduce interpolation and instantaneous-field-of-view errors. We introduce DensePol, a high-redundancy RGB--polarization dataset based on Division-of-Time (DoT) acquisition, capturing 180 full-resolution analyzer orientations at 1∘1^\circ intervals. DensePol contains 2,018 paired RGB--polarization images with the angular measurements and fitting residuals retained. Dense angular sampling substantially improves polarization stability, reducing AoLP deviation from 13.36∘13.36^\circ to 2.21∘2.21^\circ. We further introduce a deterministic diffusion-based RGB-to-polarization framework with cyclic AoLP representation and a local DoLP refiner. Experiments demonstrate improved polarization prediction and downstream surface-normal estimation. The dataset and code will be publicly available.

Param Sangani, Ahmad Moori, Erik Blasch +2

Department of Computer Science, Saint Louis University, St. Louis, MO, USA · MOVEJ Analytics, Fairborn, OH, USA · U.S. Naval Research Laboratory, Washington, DC, USA

Polarimetric Imaging
AskPDF
Mar 29, 2026

Poppy: Polarization-based Plug-and-Play Guidance for Enhancing Monocular Normal Estimation

Monocular surface normal estimators trained on large-scale RGB-normal data often perform poorly in the edge cases of reflective, textureless, and dark surfaces. Polarization encodes surface orientation independently of texture and albedo, offering a physics-based complement for these cases. Existing polarization methods, however, require multi-view capture or specialized training data, limiting generalization. We introduce Poppy, a training-free framework that refines normals from any frozen RGB backbone using single-shot polarization measurements at test time. Keeping backbone weights frozen, Poppy optimizes per-pixel offsets to the input RGB and output normal along with a learned reflectance decomposition. A differentiable rendering layer converts the refined normals into polarization predictions and penalizes mismatches with the observed signal. Across seven benchmarks and three backbone architectures (diffusion, flow, and feed-forward), Poppy reduces mean angular error by 23-26% on synthetic data and 6-16% on real data. These results show that guiding learned RGB-based normal estimators with polarization cues at test time refines normals on challenging surfaces without retraining.

Irene Kim, Sai Tanmay Reddy Chakkera, Alexandros Graikos +2

Stony Brook University

Surface Normal EstimationTest-Time Optimization
AskPDF
May 30, 2026

Structure-Aware Consistency Priors for Shape from Polarization in Complex Media

Recovering surface normals from single view polarization images in complex media remains challenging. This paper focuses on ice as a representative complex medium, where intricate light matter interactions lead to a nonlinear mapping between polarization observations and surface normals. To address this, a structure-aware polarization prior based on autocorrelation functions is proposed to capture the local spatial consistency of AoLP. Building on this, a dual-branch network (IceSfP) is designed to integrate raw polarization features with priors via cross modal attention and multi-scale feature fusion, enabling accurate surface normal estimation under complex media conditions. To evaluate the method, the first real-world ice SfP dataset is constructed. Experimental results show that the method outperforms existing approaches across all metrics, achieving a MAE of 16.01 deg, which is 2.74 deg lower than the second-best method. The framework provides a generalizable solution for high-precision geometric perception in complex media.

Kaimin Yu, Puyun Wang, Huayang He +1

The School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, China · Research Institute of Highway, Ministry of Transport, Beijing, China

Surface Normal EstimationPolarimetric Imaging
AskPDF