Architectural Unification for Polarimetric Imaging Across Multiple Degradations
Authors: Chu Zhou, Yufei Han, Junda Liao, Linrui Dai, Wangze Xu, Art Subpa-Asa, Heng Guo, Boxin Shi, +1 more
Organizations: Digital Content and Media Sciences Research Division, National Institute of Informatics, Tokyo 101-8430, Japan. · Pattern Recognition and Intelligent System Laboratory, School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China · State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing 100080, China, and also with the National Engineering Research Center of Visual Technology, School of Computer Science, Peking University, Beijing 100080, China.
Polarimetric imaging aims to recover polarimetric parameters, including Total Intensity (TI), Degree of Polarization (DoP), and Angle of Polarization (AoP), from captured polarized measurements. In real-world scenarios, these measurements are frequently affected by diverse degradations such as low-light noise, motion blur, and mosaicing artifacts. Due to the nonlinear dependency of DoP and AoP on the measured intensities, accurately retrieving physically consistent polarimetric parameters from degraded observations remains highly challenging. Existing approaches typically adopt task-specific network architectures tailored to individual degradation types, limiting their adaptability across different restoration scenarios. Moreover, many methods rely on multi-stage processing pipelines that suffer from error accumulation, or operate solely in a single domain (either image or Stokes domain), failing to fully exploit the intrinsic physical relationships between them. In this work, we propose a unified architectural framework for polarimetric imaging that is structurally shared across multiple degradation scenarios. Rather than redesigning network structures for each task, our framework maintains a consistent architectural design while being trained separately for different degradations. The model performs single-stage joint image-Stokes processing, avoiding error accumulation and explicitly preserving physical consistency. Extensive experiments show that this unified architectural design, when trained for specific degradation types, consistently achieves state-of-the-art performance across low-light denoising, motion deblurring, and demosaicing tasks, establishing a versatile and physically grounded solution for degraded polarimetric imaging.
Figures & tables
Fig. 1: Comparison of inference pipelines ( multi-stage vs . single-stage ) and representation domains ( single-domain vs . multi-domain ) in polarimetric imaging. Existing approaches occupy three quadrants: (a) ( - , - ) multi-stage single-domain methods ( e.g . , PolDeblur [ 61 ] and PIDSR [ 64 ] ); (b) ( - , + ) multi-stage multi-domain methods ( e.g . , ColorPolarNet [ 51 ] ); (c) ( + , - ) single-stage single-domain methods ( e.g . , IPLNet [ 15 ] and PLIE [ 60 ] ). (d) ( + , + ) Our framework fills the previously unexplored first quadrant, providing a single-stage multi-domain architecture. (e) Experimental results demonstrating that our unified architecture, when optimized for distinct tasks, consistently achieves superior DoP/AoP recovery across diverse degradation scenarios, including low-light denoising, motion deblurring, and demosaicing; please zoom in for better details.
Fig. 2: Empirical validation of cross-domain semantic consistency. (a) A sanity-check experiment demonstrating that CLIP-based embeddings preserve coarse semantic relationships in both the image ( i.e . , Iα1 ) and Stokes ( i.e . , S1 ) domains. The measured semantic distances ( dsem ) between scenes of the same class are consistently smaller than those between different classes. (b) Quantitative analysis across three distinct degradation tasks, including low-light noise (using the PLIE dataset [ 60 ] ), motion blur (using the PolDeblur dataset [ 61 ] ), and mosaicing artifacts (using the PIDSR dataset [ 64 ] ). The results indicate that the expected cross-domain semantic distance ( E[dsem] ) remains remarkably stable before and after degradation, with a relative deviation consistently below 10% .
Fig. 3: Overview of our single-stage multi-domain architecture. The framework is built upon a dual-branch U-shaped backbone that jointly processes representations from both the image and Stokes domains. To facilitate the information flow between these domains, the core components, Cross-Domain Collaborative Interaction (CDCI) units, inherently integrate Collaborative Attention Feature Aggregation (CAFA) and Cross-Domain Feature Modulation (CDFM).
Fig. 4: Qualitative comparisons on the real-world PLIE dataset [ 60 ] for polarimetric recovery under low-light noise. Note that the input TI ( I ) has been digitally amplified for visualization purposes, as the original low-light captures are nearly entirely dark. Our method effectively suppresses noise while accurately preserving the structural details of DoP and AoP. Please zoom in for better details.
Method
PSNR-DoP
SSIM-DoP
PSNR-AoP
SSIM-AoP
PSNR-TI
SSIM-TI
Ours
30.61
0.842
16.69
0.446
40.78
0.986
IPLNet [ 15 ]
25.32
0.715
16.21
0.276
22.84
0.930
ColorPolarNet [ 51 ]
28.32
0.785
16.37
0.332
40.06
0.983
PLIE [ 60 ]
27.15
0.765
16.42
0.336
39.19
0.977
TABLE I: Quantitative comparisons on the real-world PLIE dataset [ 60 ] for polarimetric recovery under low-light noise. The best and second-best results are highlighted in bold and underline , respectively.
Fig. 5: Qualitative comparisons on the synthetic PolDeblur dataset [ 61 ] for handling motion blur. Our method effectively restores sharp textures ( e.g . , the text below the character “D”) while accurately reconstructing polarimetric states without ringing artifacts. Please zoom in for better details.
Fig. 6: Qualitative evaluations on real-captured degraded polarized images. Despite being trained solely on synthetic pairs, our method demonstrates remarkable generalization capability in removing real-world motion blur and recovering faithful polarization structures. Please zoom in for better details.
Method
PSNR-DoP
SSIM-DoP
PSNR-AoP
SSIM-AoP
PSNR-TI
SSIM-TI
Ours
30.31
0.804
17.92
0.378
34.56
0.932
PolDeblur [ 61 ]
29.12
0.774
17.77
0.330
29.67
0.868
ColorPolarNet [ 51 ]
28.41
0.740
16.99
0.279
25.09
0.812
PLIE [ 60 ]
26.32
0.744
17.24
0.269
28.40
0.853
TABLE II: Quantitative comparisons on the synthetic PolDeblur dataset [ 61 ] for handling motion blur. The best and second-best results are highlighted in bold and underline , respectively.
Fig. 7: Qualitative comparisons on the PIDSR dataset [ 64 ] for handling mosaicing artifacts. While existing methods struggle with hallucinated textures or underestimated values in polarimetric parameters, our method recovers physically faithful and artifact-free DoP and AoP structures. Please zoom in for better details.
Method
PSNR-DoP
SSIM-DoP
PSNR-AoP
SSIM-AoP
PSNR-TI
SSIM-TI
Ours
41.09
0.944
16.77
0.510
48.98
0.994
TCPDNet [ 32 ]
39.15
0.929
15.16
0.457
43.06
0.984
NLCSR [ 27 ]
35.75
0.891
16.68
0.384
39.83
0.975
PIDSR [ 64 ]
40.42
0.939
15.15
0.465
47.53
0.992
Bilinear interpolation
31.08
0.813
14.78
0.308
37.57
0.956
TABLE III: Quantitative comparisons on the PIDSR dataset [ 64 ] for handling mosaicing artifacts. The best and second-best results are highlighted in bold and underline , respectively.
Types
Model variants
PSNR-DoP
SSIM-DoP
PSNR-AoP
SSIM-AoP
PSNR-TI
SSIM-TI
Macro-architecture
W/o Stokes domain
30.19
0.799
17.24
0.366
32.19
0.903
W/o image domain
30.28
0.801
17.91
0.367
32.53
0.906
Micro-level modules
W/o CAFA (replaced with MDTA [ 54 ] )
30.19
0.798
17.04
0.354
32.85
0.913
W/o CDFM (replaced with GDFN [ 54 ] )
29.90
0.797
17.21
0.360
32.34
0.906
W/o refinement module
30.26
0.798
17.52
0.370
32.58
0.909
Objective functions
W/o physical loss Ri
30.30
0.801
18.33
0.381
33.96
0.916
TABLE IV: Quantitative ablation study of different architectural components and physical constraints on the PolDeblur dataset [ 61 ] . The best and second-best results are highlighted in bold and underline , respectively.
Fig. 8: Impact of polarimetric restoration on downstream physics-based vision applications. For visual simplicity, only the TI ( I ) is presented. Left: Polarization-guided dehazing [ 59 ] under motion blur in a driving scenario. Right: Polarization-guided reflection removal [ 28 ] under low-light noise in an indoor scenario (the input is digitally amplified for visualization). Please zoom in for better details.
Method
DoP (PSNR / SSIM)
AoP (PSNR / SSIM)
TI (PSNR / SSIM)
Low-Light Noise (PLIE dataset [ 60 ] )
Restormer [ 54 ] (Original)
26.42 / 0.761
15.91 / 0.328
38.50 / 0.975
Restormer [ 54 ] (Modified)
28.97 / 0.816
16.35 / 0.332
40.12 / 0.984
Ours
30.61 / 0.842
16.69 / 0.446
40.78 / 0.986
Motion Blur (PolDeblur dataset [ 61 ] )
Restormer [ 54 ] (Original)
28.69 / 0.763
16.93 / 0.307
28.34 / 0.864
TABLE V: Quantitative comparison with general-purpose RGB restoration models. Our unified framework consistently outperforms both the original and modified versions of Restormer [ 54 ] .
Polarization imaging captures distinctive surface and geometric cues that benefit a wide range of vision tasks. However, real-world polarization acquisition is often affected by multiple coupled degradations, making image restoration essential for practical polarization vision. Existing methods are largely tailored to specific degradations and remain constrained by the limited scale and quality of polarization data. To address these limitations, we develop an all-in-one polarization restoration framework for diverse and composite degradations. We first study the impact of different polarization representations on restoration performance and identify the normalized Stokes representation as an effective choice for separating intensity and polarization information. Accordingly, we devise a dual-branch architecture that separates intensity and polarization modeling. To overcome the limitations of polarization-specific training, the intensity branch leverages pretrained general restoration priors and a mixture-of-experts extension for composite degradations, while its restoration knowledge is adaptively distilled into the symmetric polarization branch via a cross-domain feature transform. In addition, we establish a composite-degradation polarization benchmark to support all-in-one restoration research. Extensive experiments on public datasets and our proposed benchmark demonstrate the effectiveness of the proposed method.
Chenggong Li, Jinhao Liu, Caiyun Wu +3
School of Automation Central South University · School of Engineering Westlake University
Polarimetric imaging enables advanced vision applications such as normal estimation and de-reflection by capturing unique surface-material interactions. However, existing applications (alternatively called downstream tasks) rely on datasets constructed by naively regrouping raw measurements from division-of-focal-plane sensors, where pixels of the same polarization angle are extracted and aligned into sparse images without proper demosaicking. This reconstruction strategy results in suboptimal, incomplete targets that limit downstream performance. Moreover, current demosaicking methods are task-agnostic, optimizing only for photometric fidelity rather than utility in downstream tasks. Towards this end, we propose PolarAPP, the first framework to jointly optimize demosaicking and its downstream tasks. PolarAPP introduces a feature alignment mechanism that semantically aligns the representations of demosaicking and downstream networks via meta-learning, guiding the reconstruction to be task-aware. It further employs an equivalent imaging constraint for demosaicking training, enabling direct regression to physically meaningful outputs without relying on rearranged data. Finally, a task-refinement stage fine-tunes the task network using the stable demosaicking front-end to further enhance accuracy. Extensive experimental results demonstrate that PolarAPP outperforms existing methods in both demosaicking quality and downstream performance. Code is available upon acceptance.
Yidong Luo, Chenggong Li, Yunfeng Song +4
Zhejiang University, Hangzhou, China · School of Automation, Central South University, Changsha, China · School of Engineering, Westlake University, Hangzhou, China +1
Polarization-based vision has gained increasing attention for providing richer physical cues beyond RGB images. While achieving single-shot capture is highly desirable for practical applications, existing Division-of-Focal-Plane (DoFP) sensors inherently suffer from reduced spatial resolution and artifacts due to their spatial multiplexing mechanism. To overcome these limitations without sacrificing the snapshot capability, we propose EasyPolar, a multi-view polarimetric imaging framework. Our system is grounded in the physical insight that three independent intensity measurements are sufficient to fully characterize linear polarization. Guided by this, we design a triple-camera setup consisting of three synchronized RGB cameras that capture one unpolarized view and two polarized views with distinct orientations. Building upon this hardware design, we further propose a confidence-guided polarization reconstruction network to address the potential misalignment in multi-view fusion. The network performs multi-modal feature fusion under a confidence-aware physical guidance mechanism, which effectively suppresses warping-induced artifacts and enforces explicit geometric constraints on the solution space. Experimental results demonstrate that our method achieves high-quality results and benefits various downstream tasks.
Shuangfan Zhou, Chu Zhou, Heng Guo +4
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China · National Institute of Informatics, Japan · State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, China +1