Guided Lensless Polarization Imaging

Noa Kraicer; Erez Yosef; Raja Giryes

eess.IVMar 28, 2026

Guided Lensless Polarization Imaging

Authors: Noa Kraicer, Erez Yosef, Raja Giryes

Organizations: Tel Aviv University School of Electrical Engineering, Faculty of Engineering

Abstract

Polarization imaging captures the polarization state of light, revealing information invisible to the human eye yet valuable in domains such as biomedical diagnostics, autonomous driving, and remote sensing. However, conventional polarization cameras are often expensive, bulky, or both, limiting their practical use. Lensless imaging offers a compact, low-cost alternative by replacing the lens with a simple optical element like a diffuser and performing computational reconstruction, but existing lensless polarization systems suffer from limited reconstruction quality. To overcome these limitations, we introduce a RGB-guided lensless polarization imaging system that combines a compact polarization-RGB sensor with an auxiliary, widely available conventional RGB camera providing structural guidance. We reconstruct multi-angle polarization images for each RGB color channel through a two-stage pipeline: a physics-based inversion recovers an initial polarization image, followed by a Transformer-based fusion network that refines this reconstruction using the RGB guidance image from the conventional RGB camera. Our two-stage method significantly improves reconstruction quality and fidelity over lensless-only baselines, generalizes across datasets and imaging conditions, and achieves high-quality real-world results on our physical prototype lensless camera without any fine-tuning.

Figures & tables

Panel of Figure 1 — Figure 1: RGB-guided lensless polarization imaging system: (a) optical setup; (b) custom polarization mask; (c) captured lensless image under front illumination with two orthogonally polarized projectors; and (d) reconstructed grayscale polarization result, visualized by mapping the $0^{\circ}$ , $45^{\circ}$ , and $90^{\circ}$ outputs to the R, G, and B channels.

Figure 2: Overview of the proposed RGB-guided reconstruction pipeline. The process consists of two stages: (1) polarization intensity images (color or grayscale) are reconstructed from lensless measurements using a physics-based algorithm (FISTA/ADMM); and (2) the initial reconstruction and a registered RGB image of the same scene are separately encoded and fused through cross-domain attention to produce a refined polarization reconstruction. For visualization, the grayscale reconstructions at three polarization angles ( $0^{\circ}$ , $45^{\circ}$ , $90^{\circ}$ ) are mapped to the R, G, and B channels. The pipeline is compatible with more general input configurations.

Figure 3: Real and simulated polarization mask responses for four polarization angles in grayscale. (Top) Real masks captured with our prototype; (center) simulated masks; and (bottom) measured grayscale PSF of our diffuser in spatial and frequency domains. The PSF is low-pass, leading to loss of high-frequency information.

Panel of Figure 4 — Figure 4: Qualitative reconstruction results on UPLight and ZJU-RGB-P . Columns: RGB guidance, FISTA reconstruction, FISTA + Transformer w/o RGB, our full RGB-guided model, its fine-tuned version, and the ground-truth polarization image. Each polarization grayscale triplet ( $0^{\circ}$ , $45^{\circ}$ , $90^{\circ}$ ) is visualized as an RGB composite. Note how the RGB guidance improves the high-frequency recovery.

Modality	Model	PIP			UPLight			ZJU-RGB-P
		PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Color	FISTA	14.76	0.47	0.44	12.20	0.18	0.60	15.27	0.48	0.45
	ADMM	12.92	0.30	0.57	10.93	0.14	0.65	14.69	0.34	0.53
	FISTA + Transf.	28.31	0.86	0.12	15.98	0.37	0.36	25.65	0.86	0.15
	ADMM + Transf.	25.41	0.81	0.17	14.63	0.35	0.41	22.38	0.79	0.22
	Ours (ADMM input)	32.48	0.95	0.04	19.63	0.51	0.29	29.11	0.96	0.04

Table 1: Quantitative results (PSNR

\uparrow

/ SSIM

\uparrow

/ LPIPS

\downarrow

) for both four-angle RGB and three-angle grayscale configurations.

Model	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
FlatNet	21.57	0.68	0.45
PolarAnything (FISTA input)	21.51	0.64	0.31
PolarAnything (RGB input)	22.02	0.66	0.29
FISTA + Transf.	28.85	0.88	0.12
Ours (FISTA input)	35.53	0.97	0.03

Table 2: Comparison of FlatNet, PolarAnything, FISTA + Transformer, and our method under the three-angle grayscale configuration on the PIP dataset.

Panel of Figure 5 — Figure 5: Qualitative results on real lensless polarization data (3-angle grayscale). Each reconstructed polarization triplet ( $0^{\circ}$ , $45^{\circ}$ , $90^{\circ}$ ) is visualized as an RGB composite. Note the significant improvement in the structural details achieved by RGB guidance.

Dataset \ Model	FISTA			FISTA + Transf.			Ours (FISTA input)
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
PIP (simple fusion)	13.87	0.45	0.45	-	-	-	34.69	0.97	0.03
UPLight (simple fusion)	16.72	0.26	0.53	-	-	-	21.86	0.54	0.30
ZJU-RGB-P (simple fusion)	14.50	0.46	0.44	-	-	-	30.80	0.97	0.08
UPLight (PSF #1)	15.01	0.27	0.62	17.96	0.34	0.63	20.78	0.53	0.32
UPLight (PSF #2)	16.03	0.26	0.54	19.09	0.44	0.53	20.68	0.52	0.32

Table 3: Ablation study results (PSNR

\uparrow

/ SSIM

\uparrow

/ LPIPS

\downarrow

). Additional details appear in the supplementary.

Figure 1: Illustration of our polarization mask, containing four repetitions of striped polarizers at orientations $0^{\circ},45^{\circ},90^{\circ},\text{and }135^{\circ}$

Figure 2: The experimental optical setup from two viewpoints. Left: Front view showing the imaging target and the two-sensor setup: (1) lensless polarization camera and (2) RGB reference camera. Right: Back view of the setup showing (3) the rotation stage with a mounted linear polarizer positioned in front of (2) the RGB camera for capturing reference images. The lensless camera prototype (1) consists of a diffuser and a manually assembled polarization mask mounted on the sensor.

Model	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
UPLight
FlatNet	10.78	0.27	0.98
PolarAnything (FISTA input)	11.84	0.36	0.98
PolarAnything (RGB input)	11.98	0.40	0.93
FISTA	16.72	0.26	0.53
FISTA + Transf.	17.93	0.44	0.53

Table 1: Baseline comparison under the three-angle grayscale configuration on the UPLight and ZJU-RGB-P evaluation datasets.

Panel of Figure 3 — Figure 3: Qualitative comparison on four real scenes: plastic bag, horizontal speaker, metals, and vertical speaker. For each scene, rows correspond to the four polarization angles, and columns show FISTA, FISTA+Transformer, our method, the reference polarization image, and the RGB guidance image used by our method (rightmost column).

Model	UPLight			ZJU-RGB-P
Model	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Color
Base	20.06	0.51	0.28	30.36	0.96	0.04
Fine-tuned	21.99	0.55	0.18	31.66	0.97	0.03
Grayscale
Base	20.49	0.52	0.32	31.16	0.97	0.07

Table 2: Performance before and after fine-tuning on 10 training pairs from ZJU-RGB-P or UPLight, under the four-angle RGB and three-angle grayscale configurations, using the base model with FISTA input.

Model	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Color
Base - before	33.05	0.95	0.04
FT (UPLight)	28.95	0.92	0.06
FT (ZJU-RGB-P)	31.53	0.95	0.05
Grayscale
Base - before	35.13	0.97	0.03

Table 3: Performance on the PIP dataset before and after fine-tuning on 10 training pairs from UPLight or ZJU-RGB-P, evaluated under the four-angle RGB and three-angle grayscale configurations using the base model with FISTA input.

Figure 5: PSFs used in ablation study (#1 and #2) and the training PSF (measured from our system). Left: PSF #1. Middle: PSF #2. Right: Measured PSF used for training.

Model	PIP		UPLight
	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
Polar (FISTA+T)	28.85	0.88	17.93	0.44
RGB (RGB+T)	32.28	0.97	13.11	0.48
Ours	35.13	0.97	20.49	0.52

Table 4: RGB ablation under the same training pipeline as our method. FISTA+T denotes the baseline derived from our architecture, in which the Transformer refines only the FISTA reconstruction without RGB guidance (i.e., the same input is fed to both branches). RGB+T uses the same architecture with RGB-only input. UPLight is out-of-distribution (OOD) relative to the training data.

Dataset	0px	1px	2px	3px	4px
Ours (no translation augmentation, grayscale)
UPLight	19.92 / .54	19.84 / .52	19.74 / .51	19.61 / .50	19.46 / .50
RGBP	31.74 / .97	25.84 / .94	23.65 / .91	22.56 / .90	21.94 / .89
PIP	35.53 / .97	27.91 / .94	25.36 / .92	24.05 / .91	23.23 / .90
Ours (trained with translation augmentation, grayscale)
UPLight	20.49 / .52	20.49 / .52	20.48 / .52	20.47 / .52	20.46 / .52

Table 5: Robustness to test-time translations (total shift: 0–4 px). Metrics are reported as PSNR / SSIM, averaged over two seeds.

Explore similar work

A Lensless Polarization Camera
Mar 17, 2026Noa Kraicer, Shay Elmalem, Erez Yosef +2Polarimetric Imaging Computational Imaging
School of Electrical Engineering, The Iby and Aladar Fleischman Faculty of Engineering, Tel Aviv University, Tel Aviv, Israel · Current affiliation: Department of Physics of Complex Systems, Faculty of Physics, Weizmann Institute of Science, Rehovot, Israel. · Triangle Regional Research and Development Center, Kfar Qara’ 3007500, Israel
High-Resolution Single-Shot Polarimetric Imaging Made Easy
Apr 7, 2026Shuangfan Zhou, Chu Zhou, Heng Guo +4Polarimetric Imaging
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China · National Institute of Informatics, Japan · State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, China +1
Learning Polarization Image Restoration with General Restoration Priors
Sep 25, 2026Chenggong Li, Jinhao Liu, Caiyun Wu +3Image Restoration Polarimetric Imaging
School of Automation Central South University · School of Engineering Westlake University

Mar 17, 2026J/K move · Enter open · S save

A Lensless Polarization Camera

Noa Kraicer, Shay Elmalem, Erez Yosef, Hani Barhum+1

School of Electrical Engineering, The Iby and Aladar Fleischman Faculty of Engineering, Tel Aviv University, Tel Aviv, Israel · Current affiliation: Department of Physics of Complex Systems, Faculty of Physics, Weizmann Institute of Science, Rehovot, Israel. · Triangle Regional Research and Development Center, Kfar Qara’ 3007500, Israel

Polarimetric Imaging Computational Imaging Image Reconstruction

Open paper AskPDF arXiv

Polarization imaging is a technique that creates a pixel map of the polarization state in a scene. Although invisible to the human eye, polarization can assist various sensing and computer vision tasks. Existing polarization cameras use spatial or temporal multiplexing, which increases the camera volume, weight, cost, or all of the above. Recent lensless imaging approaches, such as DiffuserCam, have demonstrated that compact imaging systems can be realized by replacing the lens with a coding element and performing computational reconstruction. In this work, we propose a compact lensless polarization camera composed of a diffuser and a simple striped polarization mask. By combining this optical design with a reconstruction algorithm that explicitly models the polarization-encoded lensless measurements, four linear polarization images are recovered from a single snapshot. Our results demonstrate the potential of lensless approaches for polarization imaging and reveal the physical factors that govern reconstruction quality, guiding the development of high-quality practical systems.

Abstract

Explore similar work

A Lensless Polarization Camera

High-Resolution Single-Shot Polarimetric Imaging Made Easy

Learning Polarization Image Restoration with General Restoration Priors