Polarization Uncertainty-Guided Diffusion Model for Color Polarization Image Demosaicking

Chenggong Li; Yidong Luo; Junchao Zhang; Degui Yang

eess.IVFeb 27, 2026

Polarization Uncertainty-Guided Diffusion Model for Color Polarization Image Demosaicking

Authors: Chenggong Li, Yidong Luo, Junchao Zhang, Degui Yang

Organizations: School of Automation, Central South University, Changsha, China · Hunan Provincial Key Laboratory of Optic-Electronic Intelligent Measurement and Control, Changsha, China · Zhejiang University, Hangzhou, China · School of Engineering, Westlake University, Hangzhou, China

Abstract

Color polarization demosaicking (CPDM) aims to reconstruct full-resolution polarization images of four directions from the color-polarization filter array (CPFA) raw image. Due to the challenge of predicting numerous missing pixels and the scarcity of high-quality training data, existing network-based methods, despite effectively recovering scene intensity information, still exhibit significant errors in reconstructing polarization characteristics (degree of polarization, DOP, and angle of polarization, AOP). To address this problem, we introduce the image diffusion prior from text-to-image (T2I) models to overcome the performance bottleneck of network-based methods, with the additional diffusion prior compensating for limited representational capacity caused by restricted data distribution. To effectively leverage the diffusion prior, we explicitly model the polarization uncertainty during reconstruction and use uncertainty to guide the diffusion model in recovering high error regions. Extensive experiments demonstrate that the proposed method accurately recovers scene polarization characteristics with both high fidelity and strong visual perception.

Figures & tables

Figure 1: The workflow of color polarization imaging. The second row presents the AOP results of different CPDM methods, where AOP reflects the accuracy of polarization property reconstruction. Our method exhibits the least noise interference and achieves the most visually faithful results.

Figure 2: The framework of the proposed PUGDiff. (a) The architecture of the base branch, the mosaic array is initialized as a low-quality color polarization image and fed into the base branch for processing, and the network is updated via $\mathcal{L}_{B}$ ; (b) The architecture of the SD branch, results of the base branch is further refined by the SD branch. LoRA layers are injected into SD to adapt it to the CPDM task, while the cross-attention modules are pruned to improve efficiency, the LoRA parameters are updated via $\mathcal{L}_{SD}$ ; (c) The fusion process of the two branches, which is also used during inference. The contribution of each branch is adaptively weighted by polarization uncertainty. In low-uncertainty regions, the base branch is favored for high fidelity; in high-uncertainty regions, the SD branch is used to enhance the polarization property.

Figure 3: The architecture of the uncertainty estimation network. The network’s backbone shares the same architecture as the base branch, but augmented with an additional estimation head to output polarization uncertainty. The entire network is updated via $\mathcal{L}_{PU}$ , and during the training phase in Fig. 2 (c), only the polarization uncertainty is used. The right part of the figure shows the reconstructed DOP and the polarization uncertainty. High values in the heatmap indicate regions where DOP reconstruction is inaccurate.

	Method	$PSNR_{mean}$	$PSNR_{S_{0}}$	$PSNR_{DOP}$	$MAE$	$SSIM_{mean}$	$SSIM_{S_{0}}$	$SSIM_{DOP}$
MQ’s	Polanalyser	38.2149	35.2560	32.5532	14.8220	0.9653	0.9541	0.8387
	NLCSR	36.8877	33.5610	34.0964	11.1297	0.9523	0.9310	0.8557
	CPDNet	39.4374	36.6062	32.6586	12.5138	0.9700	0.9631	0.8465
	TCPDNet	42.5602	39.7694	36.0613	10.1765	0.9815	0.9765	0.8921
	DCPM	42.0221	40.0263	36.1496	11.7376	0.9807	0.9759	0.8958
	PIDSR	42.4990	40.6269	36.5413	9.6463	0.9822	0.9794	0.8950

Table 1: Quantitative comparison with state-of-the-art methods. The higher value means better effects. The bold black indicates the best result. The proposed method shows the overall optimal performance.

Figure 4: Visual comparisons for CPDM of different methods. Both on synthetic and real-captured images, our method achieves the most outstanding polarization reconstruction performance in terms of AOP and DOP.

Figure 5: Visualization of results from different branches. Our method fuses the two branches to achieve complementary strengths: using the SD branch in high-uncertainty regions to improve reconstruction, and the base branch in low-uncertainty regions to preserve fidelity.

Method	$PSNR_{S_{0}}$	$PSNR_{DOP}$	$PSNR_{AOP}$
Intensity	42.1747	40.2787	15.8751
Intensity ( $S_{0}$ )	42.1165	40.3096	16.4514
Polarization	41.9823	40.6696	16.8207

Table 2: Ablation study on the forms of uncertainty. “Intensity”, “Intensity (

S_{0}

)” and “Polarization” denote the uncertainty computed from the four direction polarization images,

S_{0}

, and DOP, respectively. The bold black indicates the best result.

Method	$PSNR_{S_{0}}$	$PSNR_{DOP}$	$PSNR_{AOP}$
All weight	–	–	–
Only U-Net	40.8976	40.2579	16.0951
rank 2	–	–	–
rank 4 (Ours)	41.9823	40.6696	16.8207
rank 8	42.0145	40.6652	16.5832

Table 3: Ablation study on the SD configuration. “All weight” denotes full-parameter fine-tuning, “Only U-Net” indicates that only the diffusion U-Net is updated, “rank” is a hyperparameter of LoRA and “–” indicates training failure. The bold black indicates the best result.

Figure 6: Visualization of different forms of uncertainty. Polarization uncertainty provides more direct information about polarization reconstruction, thus leading to the most significant improvement in DOP.

Figure 7: Visual comparisons for PRR of different methods. Our method achieves the clearest reflection removal results.

Polarimetric Imaging

Polarimetric imaging measures light’s polarization to reveal material, shape, and scene properties that ordinary images can miss.

See all 35 papers

Quaternion Tensor Modeling for Joint Color-Polarization Demosaicking
Aug 3, 2026Yanqing Song, Jifei Miao, Chaoqian Li +3Image Reconstruction Computational Imaging
School of Mathematics and Statistics, Yunnan University, Kunming, Yunnan, 650091, China · Department of Mathematics, Faculty of Science and Technology, University of Macau, Macau 999078, China · School of Computing and Artificial Intelligence Southwestern University of Finance and Economics Chengdu, Sichuan, China
CPDDNet: Color-Polarization Denoising and Demosaicking Network
Jul 1, 2026Qihang Zhang, Yusuke Monno, Masayuki Tanaka +1Image Reconstruction Image Restoration
Institute of Science Tokyo
Learning Polarization Image Restoration with General Restoration Priors
Sep 25, 2026Chenggong Li, Jinhao Liu, Caiyun Wu +3Image Restoration Polarimetric Imaging
School of Automation Central South University · School of Engineering Westlake University

Aug 3, 2026J/K move · Enter open · S save

Quaternion Tensor Modeling for Joint Color-Polarization Demosaicking

Yanqing Song, Jifei Miao, Chaoqian Li, Rui Mei+2

School of Mathematics and Statistics, Yunnan University, Kunming, Yunnan, 650091, China · Department of Mathematics, Faculty of Science and Technology, University of Macau, Macau 999078, China · School of Computing and Artificial Intelligence Southwestern University of Finance and Economics Chengdu, Sichuan, China

Image Reconstruction Computational Imaging Image Restoration Polarimetric Imaging

Open paper AskPDF arXiv

Division-of-focal-plane (DoFP) color polarization cameras enable snapshot acquisition of color polarization mosaic images, but the inherently sparse sampling pattern makes color polarization demosaicking severely ill-posed. Existing methods often fail to jointly exploit the correlations among polarization channels and the physical constraints inherent in polarization imaging, resulting in noticeable demosaicking artifacts. To address this issue, a quaternion-tensor-based color polarization demosaicking (CPDM) method incorporating Stokes-domain total variation (TV) regularization is proposed. Correlation analysis shows that the correlations among polarization channels are stronger than those among color channels. Accordingly, the color polarization images acquired at

0^\circ

45^\circ

90^\circ

, and

135^\circ

are encoded into the four components of a third-order quaternion tensor, with the color channels organized along its third mode. A low-rank prior is then imposed on the quaternion tensor to exploit the global structural redundancy in the color polarization data. Moreover, spatial gradients are mapped to the Stokes domain through an orthogonal transformation to separate intensity, polarization and residual variations, with adaptive quaternion weights enabling component-specific regularization and preserving the energy consistency of the reconstructed Stokes vectors. An efficient optimization algorithm is derived for the resulting model. Extensive experiments demonstrate the superior demosaicking performance of the proposed method.

Abstract

Polarimetric Imaging

Explore similar work

Quaternion Tensor Modeling for Joint Color-Polarization Demosaicking

CPDDNet: Color-Polarization Denoising and Demosaicking Network

Learning Polarization Image Restoration with General Restoration Priors