MPIG: Multi-angle Polarization Image Generation from a Single Polarization Observation

Jue Wang; Ruilin Huang; Liquan Shen; Zhen Tian; Xia Hua

cs.CVOct 18, 2026

MPIG: Multi-angle Polarization Image Generation from a Single Polarization Observation

Authors: Jue Wang, Ruilin Huang, Liquan Shen, Zhen Tian, Xia Hua

Organizations: School of Communication and Information Engineering, Shanghai University, Shanghai, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, China

Abstract

Polarization imaging provides informative cues for material analysis, scene understanding, and visual perception. However, acquiring complete polarization observations typically requires specialized hardware and multi-angle measurements, thereby increasing system complexity and limiting practical deployment. In this work, we address the problem of predicting the missing $45^\circ$ , $90^\circ$ , and $135^\circ$ polarization views from a single $0^\circ$ observation by leveraging a learned scene prior. To this end, we propose MPIG (Multi-angle Polarization Image Generation), a Swin Transformer-based framework that couples cross-angle feature interaction with residual decoding to predict angle-dependent intensity variations while preserving shared scene structure. Specifically, measured Stokes differences and polarization-strength-aware supervision guide the recovery of polarization modulation, orientation, and magnitude, while low-rank factorization further reduces the decoding cost. We also construct HighPol, a large-scale real-world polarization dataset comprising 5,474 groups of high-resolution four-angle polarization images collected from diverse indoor and outdoor scenes. Experimental results demonstrate improved image and polarization accuracy, achieving 29.729 dB PSNR, 0.9505 SSIM, and $21.514^\circ$ AoLP MAngE while requiring 85.86 M parameters and 1.126 T FLOPs. Furthermore, downstream validation on glass and semantic-segmentation tasks suggests that the generated polarization images can preserve task-relevant information, demonstrating the effectiveness of MPIG and the practical utility of predicted polarization observations for downstream vision applications.

Figures & tables

Fig. 1: Motivation and practical utility of MPIG. (a) Conventional polarization acquisition requires either four sequential captures with a rotating polarizer or single-shot acquisition using a costly division-of-focal-plane (DoFP) camera. (b) The target acquisition setting uses a standard camera with a fixed $0^{\circ}$ polarizer to capture one observation $I_{0}$ , from which MPIG predicts the missing views $\hat{I}_{45}$ , $\hat{I}_{90}$ , and $\hat{I}_{135}$ . (c) The completed observations support polarization-assisted perception, illustrated by DoLP and glass-segmentation results alongside those obtained from four measured views. Both segmentation masks are predictions of the same frozen model, with blue indicating glass. Acquisition setups and angle tiles are schematic.

Fig. 2: Overview of MPIG. Shared encoding and LiteFeature feed three angle-specific decoders with gated native-resolution skips and rank-40 fusion. ACAIM exchanges cross-angle features before residual prediction. Measured images and Stokes differences supervise image fidelity; normalized-Stokes, direction, and magnitude losses emphasize polarization modulation according to GT polarization strength. Insets show LiteFeature and final-stage decoding.

Panel of Figure 3 — Fig. 3: Overview of the proposed HighPol dataset. (a) Category distribution of the proposed dataset. (b) Representative samples of six major categories, covering different materials, reflection characteristics, and polarization properties.

Dataset	Size	Resolution	Public	Diversity	Pol.strength
PolaRGB [ 12 ]	6500	1224 $\times$ 1024	Yes	Med.	High
DeepSfP [ 39 ]	338	1224 $\times$ 1024	Yes	Low	Med.
RGBP-Glass [ 14 ]	4511	612 $\times$ 512	Yes	Low	High
Film Removal [ 15 ]	315	2448 $\times$ 2048	Yes	Low	Med.
Kondo [ 40 ]	44305	256 $\times$ 192	No	Synth.	Low
HighPol (Ours)	5474	1224 $\times$ 1024	Yes	High	High

TABLE I: Comparison of existing polarization datasets. Diversity denotes scene-type coverage. DeepSfP size follows the authors’ released-set description.

Fig. 4: Qualitative comparison on a HighPol water scene from the test set. Rows show the $90^{\circ}$ intensity image, DoLP, and AoLP; GT uses the measured four-angle images. DoLP and AoLP share display ranges of $[0,1]$ and $[0,180^{\circ})$ across methods.

Method	Image fidelity		DoLP			AoLP	Physical diagnostics		Complexity
Method	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAbsE $\downarrow$	MAngE $\downarrow$	$E_{\Sigma}\downarrow$	$r_{>1}\downarrow$	Params.	FLOPs
	(dB)		(dB)			( ∘ )	( $10^{-2}$ )	(%)	(M)	(T)
Pix2Pix [ 34 ]	27.715	0.9263	21.960	0.5525	0.06055	27.217	2.605	0.0863	34.26	0.498
Swin [ 19 ]	26.994	0.9140	21.717	0.5298	0.06483	28.125	2.745	0.0788	39.21	0.287
DDPM [ 35 ]	24.952	0.8631	18.007	0.3521	0.09277	37.330	2.586	0.2570	63.37	18.940
RGB2Pol-MAE [ 16 ]	28.904	0.9437	23.174	0.6125	0.05129	21.718	0.153	0.0621	329.94	1.208

TABLE II: Comparison on the 750-group test set. Bold indicates the best image and polarization accuracy and the lowest computational cost among learned methods.

Fig. 5: Downstream segmentation under frozen-input substitution. (a) ShareCMP and (b) CSFNet-2 on ZJU-RGB-P; (c) building glazing and (d) a vehicle window on HighPol. Within each panel, the intensity input and model weights are fixed. Annotation denotes the reference segmentation, whereas Measured denotes predictions using measured polarization inputs.

	ZJU-RGB-P						HighPol
	(a) Frozen-input substitution				(b) Input-specific training		(c) Frozen-input substitution
	ShareCMP		CSFNet-2		ShareCMP		Glass model (three seeds)
Input condition	mIoU	Glass IoU	mIoU	Glass IoU	mIoU	Glass IoU	Glass IoU	Dice	BF1
Measured	91.967	87.158	91.405	86.114	91.967	87.158	68.09 $\pm$ 1.09	80.13 $\pm$ 0.94	22.83 $\pm$ 1.30
MPIG	91.536	85.889	90.445	83.811	92.042	87.074	65.56 $\pm$ 4.27	78.26 $\pm$ 3.32	21.76 $\pm$ 1.67
Copy- $I_{0}$	91.392	84.998	89.162	79.538	91.866	86.343	51.80 $\pm$ 18.89	65.88 $\pm$ 18.01	14.72 $\pm$ 4.35

TABLE III: Downstream segmentation performance (%). HighPol results are mean

\pm

standard deviation across three seeds.

Fig. 6: Visual ablation on a reflective car window. Columns show MPIG without ACAIM, polarization supervision, or output residual prediction, followed by full MPIG and GT. DoLP (top) and AoLP (bottom) use a common $192\times 192$ crop and the display ranges of Fig. 4 .

Variant	Image		DoLP	AoLP
Variant	PSNR $\uparrow$	SSIM $\uparrow$	MAbsE $\downarrow$	MAngE $\downarrow$
w/o ACAIM	29.576	0.9499	0.05134	22.078
w/o polar. supervision	29.688	0.9502	0.05215	21.983
w/o output residual	29.469	0.9488	0.05175	23.161
Full MPIG	29.729	0.9505	0.05037	21.514

TABLE IV: Ablation of MPIG on the HighPol/PolarVis test set.

Polarimetric Imaging

Polarimetric imaging measures light’s polarization to reveal material, shape, and scene properties that ordinary images can miss.

See all 35 papers

Sep 25, 2026

Learning Polarization Image Restoration with General Restoration Priors

Polarization imaging captures distinctive surface and geometric cues that benefit a wide range of vision tasks. However, real-world polarization acquisition is often affected by multiple coupled degradations, making image restoration essential for practical polarization vision. Existing methods are largely tailored to specific degradations and remain constrained by the limited scale and quality of polarization data. To address these limitations, we develop an all-in-one polarization restoration framework for diverse and composite degradations. We first study the impact of different polarization representations on restoration performance and identify the normalized Stokes representation as an effective choice for separating intensity and polarization information. Accordingly, we devise a dual-branch architecture that separates intensity and polarization modeling. To overcome the limitations of polarization-specific training, the intensity branch leverages pretrained general restoration priors and a mixture-of-experts extension for composite degradations, while its restoration knowledge is adaptively distilled into the symmetric polarization branch via a cross-domain feature transform. In addition, we establish a composite-degradation polarization benchmark to support all-in-one restoration research. Extensive experiments on public datasets and our proposed benchmark demonstrate the effectiveness of the proposed method.

Chenggong Li, Jinhao Liu, Caiyun Wu +3

School of Automation Central South University · School of Engineering Westlake University

Image Restoration Polarimetric Imaging All-in-One Image Restoration

Ask PDF

Sep 8, 2026

DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision

Polarimetric vision is gaining increasing attention because it provides physical cues about scene shape, material, and reflection that are difficult to recover from RGB alone. Recent work has therefore explored predicting polarization directly from conventional RGB images; however, the fidelity of these methods strongly depends on the polarization supervision used for training. Most existing datasets rely on Division-of-Focal-Plane (DoFP) cameras with four spatially interleaved analyzer orientations, which provide limited angular redundancy and introduce interpolation and instantaneous-field-of-view errors. We introduce DensePol, a high-redundancy RGB--polarization dataset based on Division-of-Time (DoT) acquisition, capturing 180 full-resolution analyzer orientations at

1^\circ

intervals. DensePol contains 2,018 paired RGB--polarization images with the angular measurements and fitting residuals retained. Dense angular sampling substantially improves polarization stability, reducing AoLP deviation from

13.36^\circ

2.21^\circ

. We further introduce a deterministic diffusion-based RGB-to-polarization framework with cyclic AoLP representation and a local DoLP refiner. Experiments demonstrate improved polarization prediction and downstream surface-normal estimation. The dataset and code will be publicly available.

Param Sangani, Ahmad Moori, Erik Blasch +2

Department of Computer Science, Saint Louis University, St. Louis, MO, USA · MOVEJ Analytics, Fairborn, OH, USA · U.S. Naval Research Laboratory, Washington, DC, USA

Polarimetric Imaging

Ask PDF

Jul 23, 2026

Stokes-Informed Diffusion for Robust Linear Polarization Estimation

Polarization cues benefit applications such as material detection and de-reflection, yet acquiring them typically requires dedicated hardware. This motivates us to estimate the linear polarization from a single RGB image. However, the task is inherently ill-posed, with the Angle of Polarization (AoP) becoming particularly unstable in weak polarization regions, where the polarimetric signal is overwhelmed by noise, leading to erratic angle estimates. To address these limitations, we propose GenPolar, a Stokes-informed diffusion framework grounded in the Mueller formalism from an intensity observation. Specifically, GenPolar predicts channel-wise linear Stokes components (S1,S2) from intensity S0, from which degree of linear polarization (DoLP) and AoP are analytically derived; AoP is further supervised with an observability-aware loss. In addition, to enable efficient and high-fidelity inference, we adopt a two-stage training strategy. Firstly, a multi-step conditional diffusion model is trained with a physics-based loss. Subsequently, we distill it into a one-step generator, which further supports stable Low-Rank Adaptation (LoRA) of the VAE encoder to mitigate domain-specific autoencoding bias. Extensive experiments across rotating-polarizer, division-of-focal-plane, and hybrid datasets demonstrate that GenPolar achieves state-of-the-art performance in both DoLP fidelity and AoP stability. Crucially, these improvements translate to significant and consistent gains in downstream applications, including material detection and de-reflection.

Yidong Luo, Chenggong Li, Yuchao Feng +3

Zhejiang University, Hangzhou, China · School of Automation, Central South University, Changsha, China · College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China +2

Physics-Informed ML Polarimetric Imaging Diffusion Models

Ask PDF

Abstract

Polarimetric Imaging

Explore similar work

Learning Polarization Image Restoration with General Restoration Priors

DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision

Stokes-Informed Diffusion for Robust Linear Polarization Estimation