Learning from Failure: Leveraging Unreliable Predictions in Semi-Supervised Real-World Adverse Weather Removal

Cap Dang Xuan Kiet; Tat-Jen Cham

cs.CVOct 1, 2026

Learning from Failure: Leveraging Unreliable Predictions in Semi-Supervised Real-World Adverse Weather Removal

Authors: Cap Dang Xuan Kiet, Tat-Jen Cham

Organizations: College of Computing and Data Science, Nanyang Technological University, Singapore

Abstract

Adverse weather image restoration aims to recover images degraded by rain, haze, snow, and other weather-induced artifacts, thereby improving the robustness of outdoor vision systems. Existing unified restoration models exhibit limited generalization to real-world scenes due to their reliance on synthetic supervision and insufficient semantic constraints. In this paper, we propose a novel student--teacher semi-supervised framework that addresses both challenges. Specifically, we introduce an unreliable database that preserves failed teacher predictions as informative negative samples for contrastive learning, while a reliable database stores high-quality teacher predictions as positive samples. By jointly exploiting reliable pseudo-ground truths and unreliable teacher outputs, the proposed framework learns to enhance desirable restoration characteristics while avoiding common failures. We further propose a phase spectrum-based semantic constraint that replaces computationally expensive text-based supervision with an efficient and naturally aligned semantic prior. An adaptive phase consistency loss is also designed to dynamically balance supervision between the degraded input and teacher pseudo-ground truths according to degradation severity. Extensive experiments on real-world benchmarks demonstrate that the proposed method consistently outperforms existing state-of-the-art approaches in restoration quality and perceptual fidelity while exhibiting stronger generalization to real-world adverse weather conditions.

Figures & tables

Figure 1: The illustration of the proposed semi-supervised learning framework.

Models		Rain		Snow		Haze		Average		#Param	FLOP
Name	Year	LIQE $\uparrow$	CLIP-IQA $\uparrow$	LIQE $\uparrow$	CLIP-IQA $\uparrow$	LIQE $\uparrow$	CLIP-IQA $\uparrow$	LIQE $\uparrow$	CLIP-IQA $\uparrow$	M	G
Restormer	CVPR22	2.277	0.437	3.172	0.510	1.918	0.366	2.456	0.438	26.12	141.00
TransWeather	CVPR22	1.924	0.358	2.770	0.416	1.502	0.292	2.065	0.355	38.05	12.24
TKL	CVPR22	2.028	0.392	2.830	0.428	1.590	0.318	2.149	0.379	28.47	49.22
WeatherDiffusion	TPAMI23	2.050	0.395	2.950	0.466	1.520	0.326	2.173	0.396	82.96	475.16
WGWS-Net	CVPR23	1.965	0.389	2.619	0.395	1.506	0.310	2.030	0.365	5.19	13.61

Table 1: Quantitative comparison in all-in-one setting. Bold and Underlining indicate the best and second best results, respectively.

Models	RAIN100L
Models	PSNR $\uparrow$	SSIM $\uparrow$
UMR	32.39	0.921
SIRR	32.37	0.926
MSPFN	33.50	0.948
LPNet	33.61	0.958
AirNet	34.90	0.977

Table 2: Quantitative comparison in single-task setting. From left to right: deraining, desnowing, and dehazing.

Figure 2: Qualitative comparisons of adverse weather restoration. The red arrows highlight regions where the proposed method produces superior visual results. Please zoom in for better visualization.

Models	H	S	R	H+S	H+R	R*+S	H+S+R*
Models	PSNR $\uparrow$ /SSIM $\uparrow$	PSNR $\uparrow$ /SSIM $\uparrow$	PSNR $\uparrow$ /SSIM $\uparrow$	PSNR $\uparrow$ /SSIM $\uparrow$	PSNR $\uparrow$ /SSIM $\uparrow$	PSNR $\uparrow$ /SSIM $\uparrow$	PSNR $\uparrow$ /SSIM $\uparrow$
MODEM	19.73 / 0.8789	27.72/0.8953	23.76 / 0.9029	17.87 / 0.7918	20.16/0.8801	20.34/0.6734	14.23 /0.4327
HOGFormer	18.87/0.8693	28.96 / 0.9129	23.54/0.9051	16.51/0.7635	20.30 / 0.8876	20.41 / 0.6771	13.66/ 0.4557
Ours	19.89 / 0.8805	33.46 / 0.9568	24.01 / 0.9336	17.27 / 0.7874	21.36 / 0.9161	22.84 / 0.7390	14.45 / 0.5012

Table 3: Result with composited datasets. * indicates that this degradation is synthetically added by us.

Figure 3: Comparisons on multiple degradations (rain-haze-snow).

Baseline	w/ $L_{con}$		w/ unsupervised constraint			LIQE $\uparrow$	CLIP-IQA $\uparrow$
Baseline	w/o unreliable database	w/ unreliable database	LPIPS	Text	Phase	LIQE $\uparrow$	CLIP-IQA $\uparrow$
x						2.390	0.402
x	x					2.429	0.445
x		x				3.098	0.525
x		x	x			3.368	0.565
x		x		x		3.407	0.545

Table 4: Ablation study on effectiveness of proposed components. We report the average results of three tasks (desnowing, dehazing, and deraining).

NR-IQA	Rain		Snow		Haze
NR-IQA	LIQE $\uparrow$	CLIP-IQA $\uparrow$	LIQE $\uparrow$	CLIP-IQA $\uparrow$	LIQE $\uparrow$	CLIP-IQA $\uparrow$
LIQE	1.029	0.418	1.013	0.389	1.032	0.453
MUSIQ	1.774	0.469	2.507	0.532	1.329	0.420
NIMA	2.194	0.516	2.228	0.610	2.995	0.468
CLIP-IQA	2.445	0.553	3.120	0.603	1.696	0.508
RALI	3.466	0.597	3.852	0.611	3.263	0.507

Table 5: Performance comparison of different NR-IQA methods.

Degradation types	Metrics	Choices of $L_{sc}$
Degradation types	Metrics	L1	Contrastive	Ours
Rain	LIQE $\uparrow$	3.302	3.380	3.466
Rain	CLIP-IQA $\uparrow$	0.592	0.593	0.597
Snow	LIQE $\uparrow$	3.742	3.862	3.852
Snow	CLIP-IQA $\uparrow$	0.596	0.612	0.611
Haze	LIQUE $\uparrow$	3.136	3.192	3.263

Table 6: Performance comparison with different

L_{\text{phase}}

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Visual results on CDD11 dataset. From top to bottom: images, phase spectra, and amplitude spectra. Zoom in to see better visualization.

Degradation type	Methods
	HOGFormer		Ours
	Phase	Amplitude	Phase	Amplitude
Rain	52.645	12.331	33.735	8.836
Snow	62.657	9.616	58.644	8.953
Haze	57.057	11.604	32.623	6.545
Avg	57.453	11.184	41.667	8.111

Appendix

Table 7: Average L2

\downarrow

distance of the frequency features with GT on CDD11 dataset.

Method	Trainable Param	Latency	GPU Mem	Model Size
Method	M	ms	MB	MB
MODEM	19.96	100.83	551.21	77
HOGformer	16.65	218.82	760.43	65
Ours	19.67	21.20	497.08	225

Appendix

Table 8: Inference overhead comparison with other SOTAs. Input size of

256\times 256

on a single H100.

Method	Latency	GPU mem	Average Performance
Method	ms	MB	LIQE $\uparrow$	CLIP-IQA $\uparrow$
Using LPIPS	2.14	1774.66	3.368	0.565
Using Text	1198.14	13846.31	3.407	0.545
Using Phase	41.00	2894.09	3.527	0.572

Appendix

Table 9: Overhead comparison with other constraint methods. Input size of

256\times 256

on a single A100.

Backbone	Trainable Param	FLOP	Average Performance
Backbone	M	G	LIQE $\uparrow$	CLIP-IQA $\uparrow$
NAFNet	67.89	290.84	3.545	0.574
Uformer	5.29	42.74	3.268	0.497
MSBDN	19.67	153.94	3.527	0.572

Appendix

Table 10: Results of different backbone models. We report the average performance on three tasks using the dataset presented in the main paper.

Figure 5: Relationship between adaptive weight and rain density. Higher rain densities are assigned larger adaptive weights, enabling the model to place greater emphasis on pseudo-GT.

Figure 6: (a) Input image. Output of teacher model: (b) without unreliable database; (c) with unreliable database.

Method	AP			AP by Object Size
Method	IoU=0.5:0.95	IoU=0.5	IoU=0.75	Small	Medium	Large
Uformer	29.6	31.3	29.9	37.5	27.3	27.9
AirNet	31.0	32.9	32.1	36.1	26.6	30.7
PromptIR	31.2	33.7	32.3	42.1	29.9	31.1
DFPIR	28.8	30.9	29.8	27.4	26.4	30.1
Ours	35.3	38.7	36.6	44.4	29.9	35.3

Appendix

Table 11: Performance comparison of object detection models. We report average precision (AP) across different IoU thresholds and object sizes.

Degradation types	Metrics	Methods
Degradation types	Metrics	HOGformer	Ours
Rain	Q-Align $\uparrow$	2.8022	2.8372
Rain	MUSIQ $\uparrow$	53.958	55.776
Snow	Q-Align $\uparrow$	2.7557	2.6678
Snow	MUSIQ $\uparrow$	54.947	52.336
Haze	Q-Align $\uparrow$	2.4597	2.5241

Appendix

Table 12: Quantitative results on different metrics.

Explore similar work

Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal
Aug 7, 2026Wanshu Fan, Yunzhe Zhang, Yue Shen +5Query-Specific Degradation-Aware Representation Visual Degradation
School of Software Engineering, Dalian University, Dalian, China · School of Mathematical Sciences, Dalian University of Technology, Dalian, China · The Hong Kong Polytechnic University, Hong Kong, China +2
Pixel Ignores, Superpixel Sees: Adverse Weather Image Restoration via Semantic-Center SSM
Aug 3, 2026Dayu Li, Shihao Zhou, Leizhi Shu +3Image Restoration State Space Models
Nankai International Advanced Research Institute (SHENZHEN·FUTIAN) · College of Computer Science, Nankai University · University of Science and Technology Beijing +2
X-Restormer++: 1st Place Solution for the UG2+ CVPR 2026 All-Weather Restoration Challenge
May 13, 2026Youwei Pan, Leilei Cao, Yingfang Zhu +1Image Restoration Restoration
TEX AI, Transsion Holdings

Aug 7, 2026, cs.CVJ/K move · Enter open · S save

Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal

Wanshu Fan, Yunzhe Zhang, Yue Shen, Liyan Wang+4

School of Software Engineering, Dalian University, Dalian, China · School of Mathematical Sciences, Dalian University of Technology, Dalian, China · The Hong Kong Polytechnic University, Hong Kong, China+2

Query-Specific Degradation-Aware Representation Visual Degradation Restoration Degradation Cross-Modal Visual Fidelity

Open paper AskPDF arXiv

Adverse weather causes diverse and complex image degradations, severely compromising the reliability of computer vision systems. Existing all-in-one restoration models attempt to address multiple degradation types within a unified framework, but often lack explicit spatial and semantic modeling of degradation characteristics, limiting their adaptability to diverse weather conditions. To address this limitation, we propose a Degradation-Aware Cross-Modal Prompt Compensation Network (DCMPC-Net) that leverages cross-modal degradation cues from a pretrained vision-language model to condition restoration features within a unified backbone. Specifically, our DCMPC-Net mainly consists of the Cross-Modal Prompt Generator (CMPG), Prompt-Guided Attention Alignment Module (PGAAM), and Dual Feature Compensation Module (DFCM). The CMPG integrates textual embeddings with visual features to produce degradation-aware prompts that encode degradation-related semantic and contextual cues. These prompts are injected into the decoder via a PGAAM, which adaptively aligns semantic information with degraded regions to facilitate context-aware restoration. To further enhance structural fidelity, DFCM is introduced that disentangles degradation artifacts from scene structures, thereby improving the reconstruction of fine textures and detailed content. By integrating cross-modal semantic guidance with spatial alignment and structural enhancement, DCMPC-Net achieves robust and perceptually consistent restoration across diverse weather conditions. Extensive experiments show that DCMPC-Net outperforms state-of-the-art methods in both task-specific and unified settings, achieving superior accuracy and visual fidelity.

Abstract

Explore similar work

Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal

Pixel Ignores, Superpixel Sees: Adverse Weather Image Restoration via Semantic-Center SSM

X-Restormer++: 1st Place Solution for the UG2+ CVPR 2026 All-Weather Restoration Challenge