PolarVis: A Dataset and Diffusion Framework for RGB-to-Polarization Perception in Adverse Driving Scenes

Ruilin Huang; Jue Wang; Zhen Tian; Xia Hua; F. Richard Yu

Fig. 1: Overview of PolarVis. Reflective and transparent surfaces commonly encountered in vehicle-relevant environments can significantly impair conventional RGB-based visual perception. We first construct RGB-P Wild, a large-scale RGB-polarization dataset containing diverse outdoor reflective scenes. Leveraging this dataset, PolarVis employs a diffusion-based RGB-to-polarization generation framework to recover polarization representations, including multi-angle polarization images and their corresponding AoLP and DoLP maps. These recovered polarization cues provide complementary information about material properties, reflections, surface geometry, and transparency, which can facilitate reflection-aware visual perception tasks such as reflection removal, transparent object segmentation, surface normal estimation, and 3D reconstruction.

Fig. 2: Qualitative polarization generation results of PolarVis on RGB-P Wild. Given a single RGB image, PolarVis estimates four linear polarization images at

0^{\circ}

,

45^{\circ}

,

90^{\circ}

, and

135^{\circ}

. The generated polarization images are visually compared with the corresponding ground-truth polarization observations, showing that the proposed framework can reconstruct the main reflection- and material-related structures across different polarization angles.

Fig. 3: Physical basis and representative applications of polarization imaging. (a) illustrates light–surface interaction; (b)–(c) compare RGB and polarization observations; and (d)–(g) summarize applications reported in prior work, including transparent-object segmentation with PGSNet [ 1 ] , 3D reconstruction with Polarimetric MVIR [ 2 ] , reflection removal with PolarFree [ 3 ] , and surface-normal estimation with TransSfP [ 4 ] . These examples provide application context and are not experimental results of PolarVis.

Fig. 4: Detailed architecture of PolarVis. (a) Overall RGB-to-polarization generation pipeline, including the Swin-Transformer-based feature encoder, optimized diffusion module, and four angle-specific polarization decoders. (b) Illustration of the forward diffusion and reverse denoising processes. (c) Decoder data flow for polarization image reconstruction. (d) Upsampling module in the decoder. (e) Multi-Scale Feature Fusion and Attention Feature Recalibration (FFA) module used to enhance reconstructed polarization features.

Fig. 5: Sample overview of RGB-P Wild. 121 RGB images arranged in an

11\times 11

grid are selected from 10,000 images in total. The subset covers diverse illumination conditions (daytime/nighttime), object materials (glass, plastic, metal, liquid, etc.), and scene types (indoor/outdoor), reflecting the dataset’s variability in environmental context, material property, and lighting condition.

Fig. 6: Acquisition characteristics of RGB-P Wild: (a) color-polarization sampling and (b) natural-illumination distribution.

Dataset	Size	Resolution	Collection	Public	Level	Environment
RGB-P Glass [ 1 ]	4511	$612\times 512$	real-world	✓	Scene	Indoor
SPW [ 18 ]	522	$1224\times 1024$	real-world	✓	Scene	Outdoor
DeepSfP [ 37 ]	300	$256\times 256$	real-world	✓	Object	Outdoor
Spectro-Polarimetric Dataset [ 17 ]	2022/311	$1900\times 2100$ / $512\times 612$	real-world	✓	Scene	Outdoor
Kondo [ 53 ]	44305	$256\times 192$	synthetic	$\times$	Object	Outdoor
PolarAnything [ 11 ]	1148	$1224\times 1024$	real-world	✓	Scene	Outdoor

TABLE I: Comparison of representative polarization datasets. RGB-P Wild provides large-scale real-world paired RGB-polarization samples at the scene level, covering diverse materials and illumination conditions that are useful for RGB-to-polarization generation and vehicle-relevant reflective-scene perception studies.

GT AoLP ( ∘ )	PolarVis AoLP ( ∘ )	GT DoLP	PolarVis DoLP
117.5878	106.1014	0.1361	0.0929
110.9761	101.2570	0.1387	0.1026
114.4993	106.8212	0.0836	0.0773
148.3800	110.9311	0.1824	0.1146

TABLE II: Image-level comparison of GM ground-truth and PolarVis-predicted polarization attributes. AoLP is summarized by the axial circular mean over the common mask defined by ground-truth DoLP greater than 0.05, and DoLP is averaged over all pixels. These descriptive statistics do not replace pixel-wise error metrics.

Fig. 7: Qualitative transfer to the General Motors Polarimetric Imaging for Perception dataset [ 21 ] . PolarVis predicts four-angle images from RGB, from which AoLP and DoLP are derived. Reference four-angle images are reconstructed using eq. 19 , not directly measured; reference AoLP and DoLP are provided by the dataset.

Fig. 8: Supplementary in-domain qualitative comparison on RGB-P Wild. Complementing the vehicle-scene cross-dataset evaluation in Fig. 7 , PolarVis-generated polarization images are compared with direct ground-truth polarization observations from RGB-P Wild. The generated results preserve the main structural appearance and polarization-related intensity patterns across diverse reflective materials and illumination conditions, while some fine local differences remain in highly reflective or low-texture regions.

Method	Mean Squared Error (MSE) (normalized) $\downarrow$	Mean Squared Error (MSE) $\downarrow$	Peak Signal-to-Noise Ratio (PSNR) $\uparrow$	Structural Similarity Index (SSIM) $\uparrow$
Traditional diffusion model	1.5879	5273.0545	11.51	0.3686
Rombach’s decoder [ 14 ]	0.0499	165.4929	27.65	0.8434
StarGAN v2’s decoder [ 69 ]	0.0608	202.0217	26.28	0.8018
Ours	0.0465	155.0706	27.83	0.8597

TABLE III: Quantitative comparison of different diffusion and decoder configurations on RGB-P Wild. The proposed configuration achieves the lowest MSE and the highest PSNR/SSIM among the compared settings.

Fig. 9: Qualitative comparison with PolarAnything [ 11 ] on RGB-P Wild. Each scene shows the RGB input and reference, PolarAnything, and PolarVis AoLP/DoLP maps. Reference maps are derived from measured four-angle images. Grayscale and color-coded views are shown for each source; each pair displays the same underlying attribute. Quantitative results are reported in Table IV .

(a) AoLP/DoLP evaluation
Method	AoLP error ( ∘ ) $\downarrow$	DoLP MAE $\downarrow$	DoLP RMSE $\downarrow$
PolarAnything	46.03	0.1084	0.1528
PolarVis	25.66	0.0404	0.0614
(b) Controlled reconstruction with reference $S_{0}$
Method	Four-angle MAE $\downarrow$	PSNR (dB) $\uparrow$	SSIM $\uparrow$
PolarAnything	0.03469	25.19	0.8385

TABLE IV: Polarimetric comparison on the 2,500-image RGB-P Wild evaluation split: (a) AoLP/DoLP errors; (b) four-angle reconstruction with a common reference

S_{0}

. Part (b) is a controlled analysis, not end-to-end inference.

Steps	Latency (ms) $\downarrow$	FPS $\uparrow$	Peak memory (MB) $\downarrow$
1	88.74	11.27	2682.76
5	95.19	10.51	2683.17
10	103.82	9.63	2682.76

TABLE V: PolarVis inference efficiency at

512\times 512

resolution on an NVIDIA RTX 4090. The model contains 157.35 million parameters. Latency excludes file I/O, polarization conversion, and downstream inference.

Method	Mean Squared Error (MSE) (normalized) $\downarrow$	Mean Squared Error (MSE) $\downarrow$	Peak Signal-to-Noise Ratio (PSNR) $\uparrow$	Structural Similarity Index (SSIM) $\uparrow$
Only channel attention mechanism	0.0567	187.8577	26.79	0.8131
Only spatial attention mechanism	0.1769	588.6359	21.30	0.5939
Only $3\times 3$ convolution layer (remove $1\times 1$ and $5\times 5$ convolutions)	1.6987	5644.3328	11.38	0.3743
Full	0.0465	155.0706	27.83	0.8597

TABLE VI: Ablation study of the Multi-Scale Feature Fusion and Attention Feature Recalibration (FFA) module on RGB-P Wild. The comparison is conducted in terms of normalized Mean Squared Error (MSE), Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index (SSIM). The results show the contribution of multi-scale feature fusion and attention-based recalibration to polarization image reconstruction.

Fig. 10: Transparent and reflective object segmentation results using real and PolarVis-generated polarization cues. The figure visualizes RGB images, real DoLP/AoLP maps, PolarVis-generated DoLP/AoLP maps, and segmentation masks. The results indicate that generated polarization cues can provide useful boundary and reflection-related information for glass-like scene perception.

PGSNet	IoU $\uparrow$	$F_{\beta}^{w}$ $\uparrow$	MAE $\downarrow$	BER $\downarrow$
RGB-P Glass	81.08	0.842	0.091	9.63
Generated by PolarVis	74.11	0.776	0.140	13.79

TABLE VII: Comparison of polarization-assisted segmentation performance via PGSNet. Two input configurations are compared: (1) original RGB-P Glass inputs with real polarization information; and (2) RGB inputs paired with PolarVis-synthesized polarization information. The experiment evaluates whether generated polarization cues can support transparent and reflective object segmentation. Note:

\uparrow

indicates higher values are better;

\downarrow

indicates lower values are better.

Fig. 11: Vehicle detections produced by frozen PCDNet [ 22 ] on four RGBP-Car examples. The top and bottom rows use measured and PolarVis-generated AoLP/DoLP, respectively. Each column uses the same RGB image and detector weights. Aggregate results are reported in Table VIII .

Polarization input	P (%) $\uparrow$	R (%) $\uparrow$	AP50 (%) $\uparrow$	AP50:95 (%) $\uparrow$
Measured	87.6	82.3	87.0	59.8
PolarVis-generated	87.6	81.3	86.3	59.1

TABLE VIII: Vehicle detection using frozen PCDNet on 990 RGBP-Car test images. Both rows use identical detector weights and evaluation settings. Measured-polarization results are reproduced locally. All metrics are percentages.

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

PolarVis: A Dataset and Diffusion Framework for RGB-to-Polarization Perception in Adverse Driving Scenes

Abstract

Figures & tables

Polarimetric Imaging

DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision

Stokes-Informed Diffusion for Robust Linear Polarization Estimation

MPIG: Multi-angle Polarization Image Generation from a Single Polarization Observation

Abstract

Polarimetric Imaging

Explore similar work

DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision

Stokes-Informed Diffusion for Robust Linear Polarization Estimation

MPIG: Multi-angle Polarization Image Generation from a Single Polarization Observation