Seeing Through the PRISM: Compound & Controllable Restoration of Scientific Images

Rupa Kurinchi-Vendhan; Pratyusha Sharma; Antonio Torralba; Sara Beery

cs.CVMar 14, 2026

Seeing Through the PRISM: Compound & Controllable Restoration of Scientific Images

Authors: Rupa Kurinchi-Vendhan, Pratyusha Sharma, Antonio Torralba, Sara Beery

Organizations: Massachusetts Institute of Technology

Abstract

Scientific and environmental imagery often suffer from complex mixtures of noise related to the sensor and the environment. Existing restoration methods typically remove one degradation at a time, leading to cascading artifacts, overcorrection, or loss of meaningful signal. In scientific applications, restoration must be able to simultaneously handle compound degradations while allowing experts to selectively remove subsets of distortions without erasing important features. To address these challenges, we present PRISM (Precision Restoration with Interpretable Separation of Mixtures). PRISM is a prompted conditional diffusion framework which combines compound-aware supervision over mixed degradations with a weighted contrastive disentanglement objective that aligns primitives and their mixtures in the latent space. This compositional geometry enables high-fidelity joint removal of overlapping distortions while also allowing flexible, targeted fixes through natural language prompts. Across microscopy, wildlife monitoring, remote sensing, and urban weather datasets, PRISM outperforms state-of-the-art baselines on complex compound degradations, including zero-shot mixtures not seen during training. Importantly, we show that selective restoration significantly improves downstream scientific accuracy in several domains over standard "black-box" restoration. These results establish PRISM as a generalizable and controllable framework for high-fidelity restoration in domains where scientific utility is a priority.

Figures & tables

Figure 1: Expert-in-the-Loop Restoration with PRISM. PRISM enables robust compound restoration and zero-shot handling of unseen mixtures. It supports both automatic restoration and prompt-driven, selective correction for scientific analysis.

Figure 2: Overview of PRISM. We first fine-tune CLIP’s image encoder to disentangle image embeddings by distortion. The degraded input and user prompt are then used to condition the latent diffusion backbone, the coarse outputs of which are refined with a Semantic Content Preservation Module (SCPM) to yield the final restored output. Appendix Figure 12 shows the SCPM’s architecture.

Category	Method	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$
All-in-One	AirNet	14.76	0.742	78.55	0.382
	Restormer A	16.32	0.768	70.11	0.365
	NAFNet A	16.98	0.776	68.30	0.352
	PromptIR	18.11	0.801	62.78	0.298
Diffusion	DiffPlugin	19.07	0.821	53.88	0.255
	MPerceiver	20.84	0.829	48.18	0.235

Table 1: PRISM outperforms baselines with manual prompting on MDB, where each test image has up to three distortions. Best results are bolded , second-best are underlined .

Figure 3: PRISM trained on composite examples scales best with the number of distortions. This outperforms our model trained on each degradation separately as well as comparable baselines, emphasized by the

\Delta

PSNR of test images with 1 vs. 4 distortions) above each bar.

Figure 4: Latent disentanglement enables both stepwise and single-shot restoration, closing the gap between prompting strategies.

Category	Method	UIEB			POLED			ThapaSet
Category	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
All-in-One	AirNet	17.51	0.768	0.468	13.61	0.582	0.529	18.41	0.388	0.609
	Restormer A	18.13	0.792	0.454	14.38	0.608	0.512	19.31	0.413	0.588
	NAFNet A	17.76	0.756	0.479	13.02	0.591	0.543	18.94	0.401	0.596
	PromptIR	19.76	0.858	0.417	16.42	0.651	0.468	20.63	0.439	0.564
Diffusion	DiffPlugin	20.72	0.874	0.392	17.01	0.659	0.451	21.15	0.454	0.536

Table 2: PRISM achieves state-of-the-art zero-shot performance across underwater (UIEB), under-display camera (POLED), and fluid lensing (ThapaSet) benchmarks. Best results are bolded , second-best are underlined .

Figure 5: Structured, compositional latent geometry supports both automated (left) and expert-driven (right) generalization. With an expert-in-the-loop, prompts progressively target distortions.

Domain	Degraded Input	Full Restoration	Selective Restoration	p-value
Remote sensing (Acc. $\uparrow$ )	0.781 $\pm$ 0.013	0.842 $\pm$ 0.011	0.836 $\pm$ 0.012	0.11 (n.s.)
Camera Traps (Acc. $\uparrow$ )	0.921 $\pm$ 0.004	0.976 $\pm$ 0.008	0.984 $\pm$ 0.004	0.032 $<$ 0.05
Microscopy (mIoU $\uparrow$ )	0.353 $\pm$ 0.015	0.475 $\pm$ 0.012	0.580 $\pm$ 0.010	0.018 $<$ 0.05
Urban scenes (mIoU $\uparrow$ )	0.548 $\pm$ 0.018	0.615 $\pm$ 0.014	0.650 $\pm$ 0.012	0.041 $<$ 0.05

Table 3: Selective controllability outperforms full restoration across three of four downstream tasks. We report mean

\pm

std over 3 random seeds. Best results are bolded .

Figure 6: Selective restoration improves segmentation of clathrin-coated pits in microscopy. Super-resolution alone improves mIoU, while automatically detecting and removing noise suppresses faint but biologically relevant signals (see regions encircled in red), reducing accuracy.

Restoration Prompt	Segmentation	Fluorescence
	mIoU $\uparrow$	MSE $\downarrow$
Denoise	0.479	0.006
Super-Resolution	0.580	0.018
Combined	0.475	0.014
No restoration	0.356	0.025

Table 4: Task-dependent restoration performance on BioSR. Segmentation prefers structural enhancement, whereas fluorescence quantification requires accurate intensity preservation. No single restoration setting is optimal for both tasks.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Description	Size
ImageNet ( Deng et al., 2009 )	1000-class benchmark of natural images with visually diverse scenes.	1.2M
Sen12MS (Sentinel-2) ( Schmitt et al., 2019 )	RGB satellite image patches with and without clouds, used for land-cover and cloud-removal tasks.	720K
iWildCam 2022 ( Beuving et al., 2022 )	Camera trap sequences for wildlife monitoring under challenging lighting and environmental conditions.	28.8K
EUVP ( Islam et al., 2020 )	Paired underwater photos with clear vs. distorted conditions (enhanced clean split).	3.7K
Cityscapes ( Cordts et al., 2016 )	Urban street scenes captured from vehicle-mounted cameras (includes 5K labeled and 20K “extra”).	25K
BioSR ( Gong et al., 2021 )	Fluorescence microscopy slides (wide-field vs. SIM ground truth) for super-resolution and denoising tasks.	14K

Appendix

Table 5: Summary of Training Datasets. PRISM is trained on a diverse set of natural and scientific domains spanning ecological, medical, astronomical, and remote sensing imagery.

Figure 7: An overview of the data augmentation pipeline of diverse compound degradations.

Figure 8: MDB Examples. Samples from our compound degradation dataset, with paired “clean” (top) and “distorted” (bottom) images, with corresponding prompts in between.

Evaluation Setting	Task / Dataset	Dataset Size	Metric
(1) Synthetic Compound Degradations
MDB	Synthetic mixtures from clean datasets	25K	PSNR, SSIM, FID, LPIPS
(3) Zero-Shot Robustness to Unseen Real-World Distortions
Underwater Imaging	UIEB ( Li et al., 2019 )	890	PSNR, SSIM, LPIPS
Under-Display Cameras	POLED (UDC) ( Zhou et al., 2021 )	512	PSNR, SSIM, LPIPS
Fluid Lensing	ThapaSet	600	PSNR, SSIM, LPIPS

Appendix

Table 6: We group evaluations into (1) synthetic compound degradations, (2) downstream utility in real scientific datasets, and (3) zero-shot robustness to unseen real-world distortions.

	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
After SCPM	22.08	0.842	0.218
Before SCPM	16.65	0.654	0.566

Appendix

Table 7: Ablation on PRISM’s SCPM. This refinement module improves compound restoration fidelity on MDB.

Figure 9: Effect of SCPM on restoration fidelity. Without SCPM (middle), restoration reduces degradations but alters scene details, leading to blurred text/textures and distorted object boundaries. With SCPM (right), fine structures are preserved, maintaining fidelity to the ground truth (left). By reintroducing encoder features at the decoding stage, SCPM retains spatial cues that are often lost in the bottleneck representation. This cross-scale fusion constrains the decoder to stay faithful to the input structure, reducing hallucinations and over-smoothing while preserving fine details critical for scientific fidelity.

Figure 10: Ablation on temperature $\tau$ . for the contrastive objective. We show the mean cosine similarity between degraded and clean views (left) and the mean gap between positive and hardest-negative cosine (right). Results are means $\pm$ standard error margin over 3 seeds.

Similarity function	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
None	19.52	0.734	0.154
Unweighted SupCon	20.63	0.799	0.388
Cosine (labels)	21.49	0.772	0.429
Overlap coefficient	21.35	0.784	0.332
Jaccard (ours)	22.08	0.842	0.218

Appendix

Table 8: Effect of similarity function on downstream performance on the main dataset. Values show absolute performance change relative to the Jaccard-based loss (ours).

$\mathbf{N}$	Restoration Fidelity			Training	Classifier
$\mathbf{N}$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Stability $\downarrow$	F1 Score $\uparrow$
1	24.35	0.942	0.112	0.014	0.91
2	20.65	0.923	0.126	0.018	0.88
3	18.73	0.842	0.218	0.022	0.87
4	16.98	0.741	0.401	0.047	0.61

Appendix

Table 9: Impact of maximum distortion count (

N

) used in the training set. Training on more than three simultaneous degradations reduces restoration fidelity, harms distortion classification accuracy, and destabilizes prompt-following behavior.

Training Setting	Prompt Faithfulness $\uparrow$
w/o Partial or Negative Prompts	61.4%
With Partial Prompts Only	78.9%
With Negative Prompts Only	85.9%
w/o Prompt Variation	88.1%
With Partial + Negative Prompts and Prompt Variation (ours)	87.7%

Appendix

Table 10: Effect of partial and negative prompts. Including these improves prompt faithfulness (measured as proportion of outputs correctly following instructions).

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Fixed Prompt	22.08	0.842	0.218
Automated Prompt	20.84	0.824	0.229
Free-form Prompt	20.81	0.819	0.231

Appendix

Table 11: Comparison of prompting modes on MDB. Automated restoration achieves 85–95% of fixed prompting performance.

Prompt Type	PSNR $\uparrow$	LPIPS $\downarrow$	Faithfulness $\uparrow$	Alignment $\uparrow$
Composite semantic	22.08	0.218	0.98	0.93
Over-specific	22.01	0.225	0.96	0.89
Under-specified	20.30	0.241	0.78	0.70
Incorrect / negative	21.39	0.247	0.55	0.53

Appendix

Table 12: Ablation on prompt granularity and accuracy. PRISM is robust to linguistic variations but sensitive to the correctness of the distortion set.

Figure 11: Responses to intensity- and location-aware prompts. PRISM shows qualitative variability with different prompt phrasings: milder restoration tends to occur under “slightly remove the haze,” while stronger dehazing appears under “remove the haze.” Spatial specificity remains limited, and the model generally applies restoration globally even when local changes are requested (e.g., “remove haze from sky”), likely reflecting its training on global augmentations. Bottom row: heatmaps visualize pixel-wise error, and PSNR/LPIPS values are reported below.

Method	FLOPs (G)	Memory (GB)	Latency (s $\downarrow$ )
AirNet ( Li et al., 2022a )	46	2.1	0.3
Restormer A ( Zamir et al., 2022 )	118	4.6	0.5
NAFNet A ( Chen et al., 2022a )	104	4.2	0.5
OneRestore ( Guo et al., 2024 )	136	5.8	0.6
PromptIR ( Potlapalli et al., 2023a )	128	5.4	0.6
DiffPlugin ( Liu et al., 2024 )	145	6.5	2.63

Appendix

Table 13: Efficiency comparison under standardized conditions. Despite added controllability and SCPM modules, PRISM matches the runtime and memory footprint of strong diffusion-based baselines.

Figure 12: Semantic Content Preservation Module (SCPM). Encoder features

f_{\text{enc}}

are used to generate adaptive affine parameters

\gamma(f_{\text{enc}})

and

\beta(f_{\text{enc}})

, which modulate normalized decoder features

\text{Norm}(f_{\text{dec}})

. The refined features

f_{\text{refined}}

are then processed by residual and attention blocks before final decoding by

D_{\text{VAE}}

. This adaptive fusion preserves fine structures such as edges, textures, and small objects that are critical for scientific imaging tasks.

Figure 13: Contrastive disentanglement of distortion embeddings. UMAP projections of $f(I_{\text{dist}})$ from 10K samples in the MDB, across a subset of distortion classes. Left: CLIP entangles distortions with semantics. Right: Our weighted contrastive loss achieves clear separation while aligning compounds with their primitives. Overall, without contrastive disentanglement, embeddings of compound degradations collapse into unrelated regions, forcing the model to treat them as unseen categories. This can lead to artifacts or overcorrection.

Figure 14: PRISM trained on composite examples scales best with the number of distortions. This outperforms PRISM trained on each degradation separately as well as comparable baselines (MPerceiver and AutoDIR), emphasized by the

\Delta

(change in performance across test images with 1 vs. 4 distortions) above each bar.

Figure 15: Latent disentanglement of distortion types enables faithful stepwise and single-shot restoration. The contrastive loss closes the gap between sequential and composite prompting.

Figure 16: Contrastive disentanglement of distortions helps separate distortions from each other and from semantic content, enabling higher-fidelity sequential and composite restoration. Cross-attention maps (left of each output) show how the model attends to distortions. Without PRISM’s contrastive disentanglement (top), sequential restoration preserves artifacts and fails to isolate degradations. With the loss (bottom), embeddings cleanly separate distortions (e.g., clouds vs. low light). This separation not only prevents distortion types from interfering with one another, improving sequential restoration by reducing error accumulation, but also enables the model to accurately target and remove multiple degradations simultaneously, as seen in the composite restoration outputs. We report PSNR/LPIPS metric values below each output.

Figure 17: Qualitative outputs on the Mixed Degradations Benchmark (MDB). Example of a low-light + haze + snow composite evaluated across baselines. We report (PSNR/LPIPS) below, with the best results in bold . While prior methods reduce some degradations, they leave residual haze (AirNet, PromptIR), oversmooth texture (Restormer, MPerceiver), or introduce artifacts from over-correction (OneRestore, AutoDIR, DiffPlugin). PRISM produces the most faithful reconstruction, recovering both sky and foreground with minimal artifacts, closely matching the ground truth. This illustrates the strength of compositional latent disentanglement: PRISM not only removes multiple degradations simultaneously but also resists the tendency to over-restore, yielding outputs that are both high fidelity and scientifically faithful.

Figure 18: PRISM best removes unseen compound degradations. PRISM restores real-world images with degradations outside its training set in underwater imagery, under-display camera images, and fluid lensing. In all cases, it produces faithful restorations that most closely match the ground truth, showing strong single-shot generalization compared to similar diffusion baselines. We report PSNR/LPIPS metric values below, with the best results in bold .

Figure 19: Remote sensing classification under cloud occlusions requires full composite restoration. In this Sentinel-2 example, removing only clouds (middle-left) reveals incomplete information and leads to a misclassification. Full composite restoration (middle-right), correcting both clouds and low light, recovers the underlying landscape with high fidelity and matches the ground-truth class, in bold .

Figure 20: Selective restoration helps with camera trap classification under compound nighttime degradations. On the Rooftop Cityscapes dataset, frames suffer from haze and low-light conditions. Only improving contrast aids recognition of nocturnal species, while over-restoration (e.g., removing haze) can alter image content, obscure subtle texture cues, or introduce artifacts that mislead classification—sometimes even changing the perceived species. We bold the classification outputs that matches expert-provided labels.

Figure 21: Selective restoration helps with urban scene understanding under haze and low light. Rooftop Cityscapes examples show how selective restoration affects scene understanding. Removing haze alone improves mIoU, while attempting to also remove low light leads to over-correction and lower segmentation accuracy.

Figure 22: Failure modes of PRISM on challenging degradations. Top-left: Extreme unseen distortions cause incomplete restoration and overcorrection of color/contrast. Top-right: Overfitting to synthetic distortions leaves lingering artifacts when applied to real data that diverges from training augmentations. Bottom-left: Overfitting to common distortions partially reduces moire but fails to fully restore fine details. Bottom-right: Ambiguous degradations (e.g., solar flares, glare) remain difficult to generalize without explicit training examples.

Figure 23: Qualitative impact of random seed and stochasticity on restoration outcomes. Different seeds produce slightly varied outputs, reflecting both diffusion sampling variability and embedding initialization. While global structure remains stable, fine details may differ, underscoring the importance of evaluating consistency across multiple runs.

Image Restoration

Image restoration uses computational methods to recover useful visual content from noisy, blurred, low-resolution, or otherwise degraded images.

See all 372 papers

Jul 3, 2026

CURE: Controllable Unified Image Restoration for Complex Degradations

The presence of composite degradations poses a significant challenge, since the underlying corruption factors exhibit complex and interdependent interactions. Even when the degradation types are known, accurately restoring the image remains difficult due to the intertwined nature of their effects and the need for selective control during the recovery process. To address this, we introduce CURE, a unified framework that enables controllable restoration in complex degradation settings by learning disentangled and adjustable representations. CURE is driven by four complementary objectives. First, an identity embedding is incorporated, along with a reconstruction constraint, to ensure that the model can reproduce the input image when restoration is unnecessary. Second, the ratio control mechanism blends the identity embedding with degradation-specific embeddings using user-regulated mixing ratios, allowing continuous control over restoration intensity. Third, an intermediate loss is applied to supervise stepwise outputs, each encouraged to tackle the removal of only a single degradation factor within a composite mixture. Finally, a permutation-invariant loss ensures that the model achieves consistent restoration quality regardless of the order in which multiple degradations are addressed. Since CURE modifies only the training strategy and not the underlying network architecture, it can be seamlessly integrated into existing controllable restoration models. Experiments demonstrate that CURE delivers state-of-the-art performance on composite degradation benchmarks, while enabling both selective and jointly fused restoration through flexible modulation of embedding ratios. The code and dataset are available at https://github.com/bo-oseng/CURE.

Boseong Kim, Donghyeon Cho

Department of Computer Science, Hanyang University, Seoul, South Korea

Disentangled Representation Learning All-in-One Image Restoration Image Restoration Degradation-Aware Image Restoration

Ask PDF

Oct 5, 2026

EORestore-Agent: Fidelity-Guided Agentic Restoration of Remote Sensing Images with Composite Degradations

Remote sensing images often carry composite degradations, in which haze, cloud, noise, blur, low light, and low resolution coexist. Restoring them requires deciding which tool to apply, in what order, and when to stop, yet no clean reference is available at inference time to verify these decisions. All-in-one models trained on single degradations converge to a narrow PSNR band as degradations accumulate. To formulate real-world remote sensing restoration as a traceable trajectory, we present EORestore-Agent, which replaces this unmeasurable objective with reference-free, verifiable per-step decisions. A fine-tuned vision-language model reports all residual degradation types, whose tool pools are scored together, so the restoration order emerges from step-wise selection. A relative quality scorer, trained with full-reference supervision on synthetic degradation chains, predicts the changes in PSNR, SSIM, and LPIPS from the current image to each candidate. A step is accepted only when no predicted change is negative and the predicted PSNR gain is positive. Otherwise, the agent keeps the current image. On a synthetic Landsat-8 benchmark with six degradation types, EORestore-Agent improves PSNR by 2.3 to 3.2 dB over the strongest retrained all-in-one baseline on composites of two to six degradations, whereas zero-shot natural-image agents fall below the degraded input in PSNR in 17 of 18 settings. Replacing the learned scorer with no-reference quality differences costs 1.1 to 4.6 dB. The remaining harmful steps are small and cluster near the acceptance threshold. Sentinel-2 examples illustrate transfer to real atmospheric degradation without retraining.

Heli Qi, Zeqi Zhou, Jingjun Yi +5

Graduate School of Information, Production and Systems, Waseda University, Kitakyushu, Fukuoka 808-0135, Japan · RIKEN Center for Advanced Intelligence Project (AIP), RIKEN, Tokyo 103-0027, Japan · Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa, Chiba 277-8561, Japan +2

Agent-Based Image Restoration All-in-One Image Restoration Image Restoration Degradation-Aware Image Restoration

Ask PDF

Mar 27, 2026

Restore, Assess, Repeat: A Unified Framework for Iterative Image Restoration

Image restoration aims to recover high quality images from inputs degraded by various factors, such as adverse weather, blur, or low light. While recent studies have shown remarkable progress across individual or unified restoration tasks, they still suffer from limited generalization and inefficiency when handling unknown or composite degradations. To address these limitations, we propose RAR, a Restore, Assess and Repeat process, that integrates Image Quality Assessment (IQA) and Image Restoration (IR) into a unified framework to iteratively and efficiently achieve high quality image restoration. Specifically, we introduce a restoration process that operates entirely in the latent domain to jointly perform degradation identification, image restoration, and quality verification. The resulting model is fully trainable end to end and allows for an all-in-one assess and restore approach that dynamically adapts the restoration process. Also, the tight integration of IQA and IR into a unified model minimizes the latency and information loss that typically arises from keeping the two modules disjoint, (e.g. during image and/or text decoding). Extensive experiments show that our approach consistent improvements under single, unknown and composite degradations, thereby establishing a new state-of-the-art.

I-Hsiang Chen, Isma Hadji, Enrique Sanchez +5

Samsung AI Center Cambridge · National Taiwan University · Technical University of Iasi +3

Image Restoration All-in-One Image Restoration Degradation-Aware Image Restoration Image Corruption Robustness

Ask PDF

Abstract

Image Restoration

Explore similar work

CURE: Controllable Unified Image Restoration for Complex Degradations

EORestore-Agent: Fidelity-Guided Agentic Restoration of Remote Sensing Images with Composite Degradations

Restore, Assess, Repeat: A Unified Framework for Iterative Image Restoration