MPIG: Multi-angle Polarization Image Generation from a Single Polarization Observation

Jue Wang; Ruilin Huang; Liquan Shen; Zhen Tian; Xia Hua

cs.CVOct 18, 2026

MPIG: Multi-angle Polarization Image Generation from a Single Polarization Observation

Authors: Jue Wang, Ruilin Huang, Liquan Shen, Zhen Tian, Xia Hua

Organizations: School of Communication and Information Engineering, Shanghai University, Shanghai, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, China

Abstract

Polarization imaging provides informative cues for material analysis, scene understanding, and visual perception. However, acquiring complete polarization observations typically requires specialized hardware and multi-angle measurements, thereby increasing system complexity and limiting practical deployment. In this work, we address the problem of predicting the missing $45^\circ$ , $90^\circ$ , and $135^\circ$ polarization views from a single $0^\circ$ observation by leveraging a learned scene prior. To this end, we propose MPIG (Multi-angle Polarization Image Generation), a Swin Transformer-based framework that couples cross-angle feature interaction with residual decoding to predict angle-dependent intensity variations while preserving shared scene structure. Specifically, measured Stokes differences and polarization-strength-aware supervision guide the recovery of polarization modulation, orientation, and magnitude, while low-rank factorization further reduces the decoding cost. We also construct HighPol, a large-scale real-world polarization dataset comprising 5,474 groups of high-resolution four-angle polarization images collected from diverse indoor and outdoor scenes. Experimental results demonstrate improved image and polarization accuracy, achieving 29.729 dB PSNR, 0.9505 SSIM, and $21.514^\circ$ AoLP MAngE while requiring 85.86 M parameters and 1.126 T FLOPs. Furthermore, downstream validation on glass and semantic-segmentation tasks suggests that the generated polarization images can preserve task-relevant information, demonstrating the effectiveness of MPIG and the practical utility of predicted polarization observations for downstream vision applications.

Figures & tables

Fig. 1: Motivation and practical utility of MPIG. (a) Conventional polarization acquisition requires either four sequential captures with a rotating polarizer or single-shot acquisition using a costly division-of-focal-plane (DoFP) camera. (b) The target acquisition setting uses a standard camera with a fixed $0^{\circ}$ polarizer to capture one observation $I_{0}$ , from which MPIG predicts the missing views $\hat{I}_{45}$ , $\hat{I}_{90}$ , and $\hat{I}_{135}$ . (c) The completed observations support polarization-assisted perception, illustrated by DoLP and glass-segmentation results alongside those obtained from four measured views. Both segmentation masks are predictions of the same frozen model, with blue indicating glass. Acquisition setups and angle tiles are schematic.

Fig. 2: Overview of MPIG. Shared encoding and LiteFeature feed three angle-specific decoders with gated native-resolution skips and rank-40 fusion. ACAIM exchanges cross-angle features before residual prediction. Measured images and Stokes differences supervise image fidelity; normalized-Stokes, direction, and magnitude losses emphasize polarization modulation according to GT polarization strength. Insets show LiteFeature and final-stage decoding.

Panel of Figure 3 — Fig. 3: Overview of the proposed HighPol dataset. (a) Category distribution of the proposed dataset. (b) Representative samples of six major categories, covering different materials, reflection characteristics, and polarization properties.

Dataset	Size	Resolution	Public	Diversity	Pol.strength
PolaRGB [ 12 ]	6500	1224 $\times$ 1024	Yes	Med.	High
DeepSfP [ 39 ]	338	1224 $\times$ 1024	Yes	Low	Med.
RGBP-Glass [ 14 ]	4511	612 $\times$ 512	Yes	Low	High
Film Removal [ 15 ]	315	2448 $\times$ 2048	Yes	Low	Med.
Kondo [ 40 ]	44305	256 $\times$ 192	No	Synth.	Low
HighPol (Ours)	5474	1224 $\times$ 1024	Yes	High	High

TABLE I: Comparison of existing polarization datasets. Diversity denotes scene-type coverage. DeepSfP size follows the authors’ released-set description.

Fig. 4: Qualitative comparison on a HighPol water scene from the test set. Rows show the $90^{\circ}$ intensity image, DoLP, and AoLP; GT uses the measured four-angle images. DoLP and AoLP share display ranges of $[0,1]$ and $[0,180^{\circ})$ across methods.

Method	Image fidelity		DoLP			AoLP	Physical diagnostics		Complexity
Method	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAbsE $\downarrow$	MAngE $\downarrow$	$E_{\Sigma}\downarrow$	$r_{>1}\downarrow$	Params.	FLOPs
	(dB)		(dB)			( ∘ )	( $10^{-2}$ )	(%)	(M)	(T)
Pix2Pix [ 34 ]	27.715	0.9263	21.960	0.5525	0.06055	27.217	2.605	0.0863	34.26	0.498
Swin [ 19 ]	26.994	0.9140	21.717	0.5298	0.06483	28.125	2.745	0.0788	39.21	0.287
DDPM [ 35 ]	24.952	0.8631	18.007	0.3521	0.09277	37.330	2.586	0.2570	63.37	18.940
RGB2Pol-MAE [ 16 ]	28.904	0.9437	23.174	0.6125	0.05129	21.718	0.153	0.0621	329.94	1.208

TABLE II: Comparison on the 750-group test set. Bold indicates the best image and polarization accuracy and the lowest computational cost among learned methods.

Fig. 5: Downstream segmentation under frozen-input substitution. (a) ShareCMP and (b) CSFNet-2 on ZJU-RGB-P; (c) building glazing and (d) a vehicle window on HighPol. Within each panel, the intensity input and model weights are fixed. Annotation denotes the reference segmentation, whereas Measured denotes predictions using measured polarization inputs.

	ZJU-RGB-P						HighPol
	(a) Frozen-input substitution				(b) Input-specific training		(c) Frozen-input substitution
	ShareCMP		CSFNet-2		ShareCMP		Glass model (three seeds)
Input condition	mIoU	Glass IoU	mIoU	Glass IoU	mIoU	Glass IoU	Glass IoU	Dice	BF1
Measured	91.967	87.158	91.405	86.114	91.967	87.158	68.09 $\pm$ 1.09	80.13 $\pm$ 0.94	22.83 $\pm$ 1.30
MPIG	91.536	85.889	90.445	83.811	92.042	87.074	65.56 $\pm$ 4.27	78.26 $\pm$ 3.32	21.76 $\pm$ 1.67
Copy- $I_{0}$	91.392	84.998	89.162	79.538	91.866	86.343	51.80 $\pm$ 18.89	65.88 $\pm$ 18.01	14.72 $\pm$ 4.35

TABLE III: Downstream segmentation performance (%). HighPol results are mean

\pm

standard deviation across three seeds.

Fig. 6: Visual ablation on a reflective car window. Columns show MPIG without ACAIM, polarization supervision, or output residual prediction, followed by full MPIG and GT. DoLP (top) and AoLP (bottom) use a common $192\times 192$ crop and the display ranges of Fig. 4 .

Variant	Image		DoLP	AoLP
Variant	PSNR $\uparrow$	SSIM $\uparrow$	MAbsE $\downarrow$	MAngE $\downarrow$
w/o ACAIM	29.576	0.9499	0.05134	22.078
w/o polar. supervision	29.688	0.9502	0.05215	21.983
w/o output residual	29.469	0.9488	0.05175	23.161
Full MPIG	29.729	0.9505	0.05037	21.514

TABLE IV: Ablation of MPIG on the HighPol/PolarVis test set.

Polarimetric Imaging

Polarimetric imaging measures light’s polarization to reveal material, shape, and scene properties that ordinary images can miss.

See all 35 papers

Learning Polarization Image Restoration with General Restoration Priors
Sep 25, 2026Chenggong Li, Jinhao Liu, Caiyun Wu +3Image Restoration Polarimetric Imaging
School of Automation Central South University · School of Engineering Westlake University
DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision
Sep 8, 2026Param Sangani, Ahmad Moori, Erik Blasch +2Polarimetric Imaging
Department of Computer Science, Saint Louis University, St. Louis, MO, USA · MOVEJ Analytics, Fairborn, OH, USA · U.S. Naval Research Laboratory, Washington, DC, USA
Stokes-Informed Diffusion for Robust Linear Polarization Estimation
Jul 23, 2026Yidong Luo, Chenggong Li, Yuchao Feng +3Physics-Informed ML Polarimetric Imaging
Zhejiang University, Hangzhou, China · School of Automation, Central South University, Changsha, China · College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China +2

Sep 25, 2026J/K move · Enter open · S save

Learning Polarization Image Restoration with General Restoration Priors

Chenggong Li, Jinhao Liu, Caiyun Wu, Yidong Luo+2

School of Automation Central South University · School of Engineering Westlake University

Image Restoration Polarimetric Imaging All-in-One Image Restoration

Open paper AskPDF arXiv

Polarization imaging captures distinctive surface and geometric cues that benefit a wide range of vision tasks. However, real-world polarization acquisition is often affected by multiple coupled degradations, making image restoration essential for practical polarization vision. Existing methods are largely tailored to specific degradations and remain constrained by the limited scale and quality of polarization data. To address these limitations, we develop an all-in-one polarization restoration framework for diverse and composite degradations. We first study the impact of different polarization representations on restoration performance and identify the normalized Stokes representation as an effective choice for separating intensity and polarization information. Accordingly, we devise a dual-branch architecture that separates intensity and polarization modeling. To overcome the limitations of polarization-specific training, the intensity branch leverages pretrained general restoration priors and a mixture-of-experts extension for composite degradations, while its restoration knowledge is adaptively distilled into the symmetric polarization branch via a cross-domain feature transform. In addition, we establish a composite-degradation polarization benchmark to support all-in-one restoration research. Extensive experiments on public datasets and our proposed benchmark demonstrate the effectiveness of the proposed method.

Abstract

Polarimetric Imaging

Explore similar work

Learning Polarization Image Restoration with General Restoration Priors

DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision

Stokes-Informed Diffusion for Robust Linear Polarization Estimation