PolarVis: A Dataset and Diffusion Framework for RGB-to-Polarization Perception in Adverse Driving Scenes

Ruilin Huang; Jue Wang; Zhen Tian; Xia Hua; F. Richard Yu

cs.CVOct 18, 2026

PolarVis: A Dataset and Diffusion Framework for RGB-to-Polarization Perception in Adverse Driving Scenes

Authors: Ruilin Huang, Jue Wang, Zhen Tian, Xia Hua, F. Richard Yu

Organizations: School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen 518107, China · Department of Electrical Engineering, Carleton University, Ottawa, Ontario K1S 5B6, Canada

Abstract

Reflections, glare, transparent structures, and material-dependent appearance variations can introduce ambiguity into RGB-based visual perception in vehicle-relevant environments. Polarization provides complementary reflection- and material-sensitive information, but its acquisition requires dedicated sensing hardware. This paper presents RGB-P Wild, a real-world dataset containing 10,000 paired RGB and four-angle linear-polarization image sets, together with PolarVis, a diffusion-based framework for recovering polarization representations from a single RGB image. PolarVis combines hierarchical RGB feature extraction, diffusion-based feature refinement, four angle-specific decoders, and multi-scale attention recalibration to reconstruct polarization intensity images at $0^\circ$ , $45^\circ$ , $90^\circ$ , and $135^\circ$ . On RGB-P Wild, PolarVis achieves a normalized-domain MSE of 0.0465, a PSNR of 27.83 dB, and an SSIM of 0.8597. Compared with the released PolarAnything model under the adopted evaluation protocol, PolarVis obtains an AoLP error of $25.66^\circ$ and a DoLP MAE of 0.0404, compared with $46.03^\circ$ and 0.1084, respectively. Cross-dataset evaluation on vehicle-mounted imagery further examines transfer beyond the training domain. In downstream evaluation, a frozen PCDNet achieves AP50/AP50:95 of 86.3%/59.1% on 990 RGBP-Car test images using PolarVis-generated polarization, compared with 87.0%/59.8% using measured polarization. These results indicate that RGB-derived polarization retains useful polarimetric information for the evaluated perception tasks, while remaining distinct from calibrated physical polarization measurements.

Figures & tables

Fig. 1: Overview of PolarVis. Reflective and transparent surfaces commonly encountered in vehicle-relevant environments can significantly impair conventional RGB-based visual perception. We first construct RGB-P Wild, a large-scale RGB-polarization dataset containing diverse outdoor reflective scenes. Leveraging this dataset, PolarVis employs a diffusion-based RGB-to-polarization generation framework to recover polarization representations, including multi-angle polarization images and their corresponding AoLP and DoLP maps. These recovered polarization cues provide complementary information about material properties, reflections, surface geometry, and transparency, which can facilitate reflection-aware visual perception tasks such as reflection removal, transparent object segmentation, surface normal estimation, and 3D reconstruction.

Fig. 2: Qualitative polarization generation results of PolarVis on RGB-P Wild. Given a single RGB image, PolarVis estimates four linear polarization images at $0^{\circ}$ , $45^{\circ}$ , $90^{\circ}$ , and $135^{\circ}$ . The generated polarization images are visually compared with the corresponding ground-truth polarization observations, showing that the proposed framework can reconstruct the main reflection- and material-related structures across different polarization angles.

Panel of Figure 3 — Fig. 3: Physical basis and representative applications of polarization imaging. (a) illustrates light–surface interaction; (b)–(c) compare RGB and polarization observations; and (d)–(g) summarize applications reported in prior work, including transparent-object segmentation with PGSNet [ 1 ] , 3D reconstruction with Polarimetric MVIR [ 2 ] , reflection removal with PolarFree [ 3 ] , and surface-normal estimation with TransSfP [ 4 ] . These examples provide application context and are not experimental results of PolarVis.

Fig. 4: Detailed architecture of PolarVis. (a) Overall RGB-to-polarization generation pipeline, including the Swin-Transformer-based feature encoder, optimized diffusion module, and four angle-specific polarization decoders. (b) Illustration of the forward diffusion and reverse denoising processes. (c) Decoder data flow for polarization image reconstruction. (d) Upsampling module in the decoder. (e) Multi-Scale Feature Fusion and Attention Feature Recalibration (FFA) module used to enhance reconstructed polarization features.

Fig. 5: Sample overview of RGB-P Wild. 121 RGB images arranged in an $11\times 11$ grid are selected from 10,000 images in total. The subset covers diverse illumination conditions (daytime/nighttime), object materials (glass, plastic, metal, liquid, etc.), and scene types (indoor/outdoor), reflecting the dataset’s variability in environmental context, material property, and lighting condition.

Panel of Figure 6 — Fig. 6: Acquisition characteristics of RGB-P Wild: (a) color-polarization sampling and (b) natural-illumination distribution.

Dataset	Size	Resolution	Collection	Public	Level	Environment
RGB-P Glass [ 1 ]	4511	$612\times 512$	real-world	✓	Scene	Indoor
SPW [ 18 ]	522	$1224\times 1024$	real-world	✓	Scene	Outdoor
DeepSfP [ 37 ]	300	$256\times 256$	real-world	✓	Object	Outdoor
Spectro-Polarimetric Dataset [ 17 ]	2022/311	$1900\times 2100$ / $512\times 612$	real-world	✓	Scene	Outdoor
Kondo [ 53 ]	44305	$256\times 192$	synthetic	$\times$	Object	Outdoor
PolarAnything [ 11 ]	1148	$1224\times 1024$	real-world	✓	Scene	Outdoor

TABLE I: Comparison of representative polarization datasets. RGB-P Wild provides large-scale real-world paired RGB-polarization samples at the scene level, covering diverse materials and illumination conditions that are useful for RGB-to-polarization generation and vehicle-relevant reflective-scene perception studies.

GT AoLP ( ∘ )	PolarVis AoLP ( ∘ )	GT DoLP	PolarVis DoLP
117.5878	106.1014	0.1361	0.0929
110.9761	101.2570	0.1387	0.1026
114.4993	106.8212	0.0836	0.0773
148.3800	110.9311	0.1824	0.1146

TABLE II: Image-level comparison of GM ground-truth and PolarVis-predicted polarization attributes. AoLP is summarized by the axial circular mean over the common mask defined by ground-truth DoLP greater than 0.05, and DoLP is averaged over all pixels. These descriptive statistics do not replace pixel-wise error metrics.

Fig. 7: Qualitative transfer to the General Motors Polarimetric Imaging for Perception dataset [ 21 ] . PolarVis predicts four-angle images from RGB, from which AoLP and DoLP are derived. Reference four-angle images are reconstructed using eq. 19 , not directly measured; reference AoLP and DoLP are provided by the dataset.

Fig. 8: Supplementary in-domain qualitative comparison on RGB-P Wild. Complementing the vehicle-scene cross-dataset evaluation in Fig. 7 , PolarVis-generated polarization images are compared with direct ground-truth polarization observations from RGB-P Wild. The generated results preserve the main structural appearance and polarization-related intensity patterns across diverse reflective materials and illumination conditions, while some fine local differences remain in highly reflective or low-texture regions.

Method	Mean Squared Error (MSE) (normalized) $\downarrow$	Mean Squared Error (MSE) $\downarrow$	Peak Signal-to-Noise Ratio (PSNR) $\uparrow$	Structural Similarity Index (SSIM) $\uparrow$
Traditional diffusion model	1.5879	5273.0545	11.51	0.3686
Rombach’s decoder [ 14 ]	0.0499	165.4929	27.65	0.8434
StarGAN v2’s decoder [ 69 ]	0.0608	202.0217	26.28	0.8018
Ours	0.0465	155.0706	27.83	0.8597

TABLE III: Quantitative comparison of different diffusion and decoder configurations on RGB-P Wild. The proposed configuration achieves the lowest MSE and the highest PSNR/SSIM among the compared settings.

Fig. 9: Qualitative comparison with PolarAnything [ 11 ] on RGB-P Wild. Each scene shows the RGB input and reference, PolarAnything, and PolarVis AoLP/DoLP maps. Reference maps are derived from measured four-angle images. Grayscale and color-coded views are shown for each source; each pair displays the same underlying attribute. Quantitative results are reported in Table IV .

(a) AoLP/DoLP evaluation
Method	AoLP error ( ∘ ) $\downarrow$	DoLP MAE $\downarrow$	DoLP RMSE $\downarrow$
PolarAnything	46.03	0.1084	0.1528
PolarVis	25.66	0.0404	0.0614
(b) Controlled reconstruction with reference $S_{0}$
Method	Four-angle MAE $\downarrow$	PSNR (dB) $\uparrow$	SSIM $\uparrow$
PolarAnything	0.03469	25.19	0.8385

TABLE IV: Polarimetric comparison on the 2,500-image RGB-P Wild evaluation split: (a) AoLP/DoLP errors; (b) four-angle reconstruction with a common reference

S_{0}

. Part (b) is a controlled analysis, not end-to-end inference.

Steps	Latency (ms) $\downarrow$	FPS $\uparrow$	Peak memory (MB) $\downarrow$
1	88.74	11.27	2682.76
5	95.19	10.51	2683.17
10	103.82	9.63	2682.76

TABLE V: PolarVis inference efficiency at

512\times 512

resolution on an NVIDIA RTX 4090. The model contains 157.35 million parameters. Latency excludes file I/O, polarization conversion, and downstream inference.

Method	Mean Squared Error (MSE) (normalized) $\downarrow$	Mean Squared Error (MSE) $\downarrow$	Peak Signal-to-Noise Ratio (PSNR) $\uparrow$	Structural Similarity Index (SSIM) $\uparrow$
Only channel attention mechanism	0.0567	187.8577	26.79	0.8131
Only spatial attention mechanism	0.1769	588.6359	21.30	0.5939
Only $3\times 3$ convolution layer (remove $1\times 1$ and $5\times 5$ convolutions)	1.6987	5644.3328	11.38	0.3743
Full	0.0465	155.0706	27.83	0.8597

TABLE VI: Ablation study of the Multi-Scale Feature Fusion and Attention Feature Recalibration (FFA) module on RGB-P Wild. The comparison is conducted in terms of normalized Mean Squared Error (MSE), Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index (SSIM). The results show the contribution of multi-scale feature fusion and attention-based recalibration to polarization image reconstruction.

Fig. 10: Transparent and reflective object segmentation results using real and PolarVis-generated polarization cues. The figure visualizes RGB images, real DoLP/AoLP maps, PolarVis-generated DoLP/AoLP maps, and segmentation masks. The results indicate that generated polarization cues can provide useful boundary and reflection-related information for glass-like scene perception.

PGSNet	IoU $\uparrow$	$F_{\beta}^{w}$ $\uparrow$	MAE $\downarrow$	BER $\downarrow$
RGB-P Glass	81.08	0.842	0.091	9.63
Generated by PolarVis	74.11	0.776	0.140	13.79

TABLE VII: Comparison of polarization-assisted segmentation performance via PGSNet. Two input configurations are compared: (1) original RGB-P Glass inputs with real polarization information; and (2) RGB inputs paired with PolarVis-synthesized polarization information. The experiment evaluates whether generated polarization cues can support transparent and reflective object segmentation. Note:

\uparrow

indicates higher values are better;

\downarrow

indicates lower values are better.

Fig. 11: Vehicle detections produced by frozen PCDNet [ 22 ] on four RGBP-Car examples. The top and bottom rows use measured and PolarVis-generated AoLP/DoLP, respectively. Each column uses the same RGB image and detector weights. Aggregate results are reported in Table VIII .

Polarization input	P (%) $\uparrow$	R (%) $\uparrow$	AP50 (%) $\uparrow$	AP50:95 (%) $\uparrow$
Measured	87.6	82.3	87.0	59.8
PolarVis-generated	87.6	81.3	86.3	59.1

TABLE VIII: Vehicle detection using frozen PCDNet on 990 RGBP-Car test images. Both rows use identical detector weights and evaluation settings. Measured-polarization results are reproduced locally. All metrics are percentages.

Polarimetric Imaging

Polarimetric imaging measures light’s polarization to reveal material, shape, and scene properties that ordinary images can miss.

See all 35 papers

Sep 8, 2026

DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision

Polarimetric vision is gaining increasing attention because it provides physical cues about scene shape, material, and reflection that are difficult to recover from RGB alone. Recent work has therefore explored predicting polarization directly from conventional RGB images; however, the fidelity of these methods strongly depends on the polarization supervision used for training. Most existing datasets rely on Division-of-Focal-Plane (DoFP) cameras with four spatially interleaved analyzer orientations, which provide limited angular redundancy and introduce interpolation and instantaneous-field-of-view errors. We introduce DensePol, a high-redundancy RGB--polarization dataset based on Division-of-Time (DoT) acquisition, capturing 180 full-resolution analyzer orientations at

1^\circ

intervals. DensePol contains 2,018 paired RGB--polarization images with the angular measurements and fitting residuals retained. Dense angular sampling substantially improves polarization stability, reducing AoLP deviation from

13.36^\circ

2.21^\circ

. We further introduce a deterministic diffusion-based RGB-to-polarization framework with cyclic AoLP representation and a local DoLP refiner. Experiments demonstrate improved polarization prediction and downstream surface-normal estimation. The dataset and code will be publicly available.

Param Sangani, Ahmad Moori, Erik Blasch +2

Department of Computer Science, Saint Louis University, St. Louis, MO, USA · MOVEJ Analytics, Fairborn, OH, USA · U.S. Naval Research Laboratory, Washington, DC, USA

Polarimetric Imaging

Ask PDF

Jul 23, 2026

Stokes-Informed Diffusion for Robust Linear Polarization Estimation

Polarization cues benefit applications such as material detection and de-reflection, yet acquiring them typically requires dedicated hardware. This motivates us to estimate the linear polarization from a single RGB image. However, the task is inherently ill-posed, with the Angle of Polarization (AoP) becoming particularly unstable in weak polarization regions, where the polarimetric signal is overwhelmed by noise, leading to erratic angle estimates. To address these limitations, we propose GenPolar, a Stokes-informed diffusion framework grounded in the Mueller formalism from an intensity observation. Specifically, GenPolar predicts channel-wise linear Stokes components (S1,S2) from intensity S0, from which degree of linear polarization (DoLP) and AoP are analytically derived; AoP is further supervised with an observability-aware loss. In addition, to enable efficient and high-fidelity inference, we adopt a two-stage training strategy. Firstly, a multi-step conditional diffusion model is trained with a physics-based loss. Subsequently, we distill it into a one-step generator, which further supports stable Low-Rank Adaptation (LoRA) of the VAE encoder to mitigate domain-specific autoencoding bias. Extensive experiments across rotating-polarizer, division-of-focal-plane, and hybrid datasets demonstrate that GenPolar achieves state-of-the-art performance in both DoLP fidelity and AoP stability. Crucially, these improvements translate to significant and consistent gains in downstream applications, including material detection and de-reflection.

Yidong Luo, Chenggong Li, Yuchao Feng +3

Zhejiang University, Hangzhou, China · School of Automation, Central South University, Changsha, China · College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China +2

Physics-Informed ML Polarimetric Imaging Diffusion Models

Ask PDF

Oct 18, 2026

MPIG: Multi-angle Polarization Image Generation from a Single Polarization Observation

Polarization imaging provides informative cues for material analysis, scene understanding, and visual perception. However, acquiring complete polarization observations typically requires specialized hardware and multi-angle measurements, thereby increasing system complexity and limiting practical deployment. In this work, we address the problem of predicting the missing

45^\circ

90^\circ

, and

135^\circ

polarization views from a single

0^\circ

observation by leveraging a learned scene prior. To this end, we propose MPIG (Multi-angle Polarization Image Generation), a Swin Transformer-based framework that couples cross-angle feature interaction with residual decoding to predict angle-dependent intensity variations while preserving shared scene structure. Specifically, measured Stokes differences and polarization-strength-aware supervision guide the recovery of polarization modulation, orientation, and magnitude, while low-rank factorization further reduces the decoding cost. We also construct HighPol, a large-scale real-world polarization dataset comprising 5,474 groups of high-resolution four-angle polarization images collected from diverse indoor and outdoor scenes. Experimental results demonstrate improved image and polarization accuracy, achieving 29.729 dB PSNR, 0.9505 SSIM, and

21.514^\circ

AoLP MAngE while requiring 85.86 M parameters and 1.126 T FLOPs. Furthermore, downstream validation on glass and semantic-segmentation tasks suggests that the generated polarization images can preserve task-relevant information, demonstrating the effectiveness of MPIG and the practical utility of predicted polarization observations for downstream vision applications.

Jue Wang, Ruilin Huang, Liquan Shen +2

School of Communication and Information Engineering, Shanghai University, Shanghai, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, China

Polarimetric Imaging Image Reconstruction

Ask PDF

Abstract

Polarimetric Imaging

Explore similar work

DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision

Stokes-Informed Diffusion for Robust Linear Polarization Estimation

MPIG: Multi-angle Polarization Image Generation from a Single Polarization Observation