WeatherRemover: All-in-one Adverse Weather Removal with Multi-scale Feature Map Compression

Weikai Qu; Sijun Liang; Cheng Pan; Zikuan Yang; Guanchi Zhou; Xianjun Fu; Bo Liu; Changmiao Wang

cs.CVApr 8, 2026

WeatherRemover: All-in-one Adverse Weather Removal with Multi-scale Feature Map Compression

Authors: Weikai Qu, Sijun Liang, Cheng Pan, Zikuan Yang, Guanchi Zhou, Xianjun Fu, Bo Liu, Changmiao Wang,

Organizations: Guangdong University of Technology · Sanda University · University of Queensland · Shenzhen MSU-BIT University ·

Abstract

Photographs taken in adverse weather conditions often suffer from blurriness, occlusion, and low brightness due to interference from rain, snow, and fog. These issues can significantly hinder the performance of subsequent computer vision tasks, making the removal of weather effects a crucial step in image enhancement. Existing methods primarily target specific weather conditions, with only a few capable of handling multiple weather scenarios. However, mainstream approaches often overlook performance considerations, resulting in large parameter sizes, long inference times, and high memory costs. In this study, we introduce the WeatherRemover model, designed to enhance the restoration of images affected by various weather conditions while balancing performance. Our model adopts a UNet-like structure with a gating mechanism and a multi-scale pyramid vision Transformer. It employs channel-wise attention derived from convolutional neural networks to optimize feature extraction, while linear spatial reduction helps curtail the computational demands of attention. The gating mechanisms, strategically placed within the feed-forward and downsampling phases, refine the processing of information by selectively addressing redundancy and mitigating its influence on learning. This approach facilitates the adaptive selection of essential data, ensuring superior restoration and maximizing efficiency. Additionally, our lightweight model achieves an optimal balance between restoration quality, parameter efficiency, computational overhead, and memory usage, distinguishing it from other multi-weather models, thereby meeting practical application demands effectively. The source code is available at https://github.com/RICKand-MORTY/WeatherRemover.

Figures & tables

Fig. 1: Overview of WeatherRemover structure. The WeatherRemover model is structured around a UNet architecture. The left side of the model is dedicated to downsample, incorporating a gating mechanism to manage the flow of information, delineated as three D-Stages. The gating mechanism in D-Stage employs a dual-branch structure, where one branch is activated via the GELU function and both branches are fused through element-wise multiplication. The right side performs upsampling, merges features from matching downsampling layers, and includes three U-Stages. Each U-Stage fuses outputs from the previous layer and matching D-Stage along channels, then applies a 1 $\times$ 1 convolution to halve the channels. Each stage utilizes MS-PVT, halving spatial size and doubling channels during downsampling, and reversing this during upsampling.

Fig. 2: The MSA architecture. Within the MSA, the input matrix passes through a $1\times 1$ convolution layer and a depthwise separable convolution to obtain the query. Next, adjust the dimensions of the query to adapt the computation of multi-head attention (”HEADS” represents the number of attention heads). For the generation of the key and value, the input matrix is first compressed by linear SRA, reducing the dimensions from $B\times D\times H\times W$ to $B\times D\times P\times P$ . It then passes through a $1\times 1$ convolution layer and a depthwise separable convolution and is finally split along the channel dimension to obtain the key and value, which are adjusted to have the same dimensions as the query. The query, key, and value are then used to compute attention scores, which are scaled by a learnable parameter.

Fig. 3: The linear SRA flowchart for compressing feature maps. The primary component is an adaptive mean pooling layer. It compresses the tensor from dimensions $H\times W$ to $P\times P$ , where $P$ represents the size of the feature map output from the adaptive average pooling layer. The input tensor, with an initial dimension of $B\times D\times H\times W$ , is transformed to $B\times D\times P\times P$ after passing through the adaptive mean pooling layer. Subsequently, it is processed by a $1\times 1$ convolutional layer and normalized. Finally, the tensor is activated by an activation function.

Fig. 4: Comparison between PVTv2 and GFN structures. (a) depicts the structure of PVTv2, and (b) illustrates our proposed structure GFN. PVTv2’s serial FN structure was modified with a gating mechanism: the original single-path 1x1 and depthwise separable convolution were split into dual paths, one activated, then fused via element-wise multiplication.

Fig. 5: Overall structure of MS-PVT. MS-PVT is composed of MSA and GFN in series. The outputs of both MSA and GFN are normalized and added to the input. On the right side of both GFN and MSA, there are overall descriptions of the internal data flow. For an input with dimensions $B\times D\times H\times W$ , the output also has dimensions $B\times D\times H\times W$ .

Fig. 6: Desnowing visual comparisons. From left to right, the sequence of images includes the input images, Restormer [ 6 ] , DRSformer [ 11 ] , our model, and the ground truth images. Each image is accompanied by magnified details beneath it for closer examination. Images are selected from the Snow100K-L [ 27 ] test set, focusing on scenes where the background color closely resembles the snowflake color.

Fig. 7: Real-scene desnowing visual comparisons. The figures from left to right are: Input images, Restormer [ 6 ] , DRSformer [ 11 ] , ConvIR-B [ 16 ] , and our model. The zoomed-in details are shown below each image. Images were selected from the DAWN [ 30 ] dataset. The first two rows demonstrate the models’ ability to restore fine details in real-world scenarios, while the third row highlights the performance of the models in removing heavy snow in low-resolution settings, aiming to reveal our limitations under such conditions.

Task	Model	PSNR $\uparrow$	SSIM $\uparrow$	Time/s	Param
Snow Removal	SnowDiff 64 [ 18 ]	30.43	0.9145	>10	82.9M
	SnowDiff 128 [ 18 ]	30.28	0.9000	>10	85.6M
	Restormer [ 6 ]	31.48	0.9286	0.15	26.5M
	DRSformer [ 11 ]	31.06	0.9100	0.21	33.6M
	LMQFormer [ 17 ]	31.24	0.9100	0.02	2.18M
	ConvIR-S [ 16 ]	30.49	0.9290	0.02	5.53M

TABLE I: Desnowing results on Snow100K-L.

Fig. 8: Raindrop removal visual comparisons. From left to right, the images are: Input images, Restormer [ 6 ] , DRSformer [ 11 ] , I 2 R 2 Net [ 35 ] , our model, and ground truth images. Each image includes magnified details below. The images are from the RainDrop [ 28 ] dataset’s test set. The first row showcases restoration on backgrounds resembling rain droplet colors, the second highlights dense droplet removal, and the third contrasts models’ impacts on texture and detail preservation.

Task	Model	PSNR $\uparrow$	SSIM $\uparrow$	Time/s	Param
Raindrop Removal	AttentiveGAN [ 28 ]	31.59	0.9170	3.21	6.2M
	DuRN [ 14 ]	31.24	0.9259	0.01	10.2M
	RaindropAttn [ 36 ]	31.44	0.9263	0.19	7.3M
	IDT [ 37 ]	31.87	0.9313	1.05	16.4M
	RainDropDiff 64 [ 18 ]	32.29	0.9422	>10	82.9M
	RainDropDiff 128 [ 18 ]	32.43	0.9334	>10	85.6M

TABLE II: Raindrop removal results on RainDrop-A.

Fig. 9: Single-weather deraining and dehazing visual comparisons. From left to right, the images displayed are: The original input images, Restormer [ 6 ] , DRSformer [ 11 ] , I 2 R 2 Net [ 35 ] , our proposed model, and the ground truth images. Below each image, magnified details are provided to emphasize the distinctions in performance. The images are from the Outdoor-Rain [ 29 ] dataset. The first and third rows demonstrate the removal of rain haze from the sky, while the second row compares the impact of different models on preserving the original image details.

Fig. 10: Real-scene rain and dense fog removal visual comparisons. From left to right: Input image, Restormer [ 6 ] , DRSformer [ 11 ] , I 2 R 2 Net [ 35 ] , and our model. To show the fineness of the restoration, there are enlarged details below each image. Images with extreme dense fog from the DAWN [ 30 ] dataset were selected to demonstrate the effectiveness of different models in fog removal and their capability to restore image details.

Task	Model	PSNR $\uparrow$	SSIM $\uparrow$	Time/s	Param
Deraining & Dehazing	HRGAN [ 29 ]	21.56	0.8550	0.28	40.6M
	PCNet [ 38 ]	26.19	0.9015	0.02	0.6M
	MPRNet [ 39 ]	28.03	0.9192	0.02	3.6M
	RainHazeDiff 64 [ 18 ]	28.38	0.9320	>10	82.9M
	RainHazeDiff 128 [ 18 ]	26.84	0.9152	>10	85.6M
	Restormer [ 6 ]	31.28	0.9252	0.16	26.5M

TABLE III: Results of deraining and dehazing on the Outdoor-Rain dataset.

Dataset	GFN	D-Stage	PSNR $\uparrow$	SSIM $\uparrow$	Time/s	Param
	Gating
			31.1566	0.9164	0.0805	20.10M
	✓		32.0181	0.9250	0.0979	23.76M
		✓	31.4092	0.9184	0.0944	20.56M
Snow100K	✓	✓	32.2699	0.9268	0.1161	24.32M
			32.2264	0.9390	0.0967	20.10M

TABLE IV: Ablation experiments of the gating mechanisms within GFN and D-Stage. Ablated by removing the element-wise multiplications and branches.

Dataset	Method	PSNR $\uparrow$	SSIM $\uparrow$	Time/s	Param
	1 $\times$ 1Conv	30.3039	0.9075	0.1085	24.10M
	DWConv	31.8930	0.9230	0.1069	18.87M
	DWConv-1 $\times$ 1Conv	31.8597	0.9223	0.1151	27.94M
Snow100K	1 $\times$ 1Conv-DWConv	32.2699	0.9268	0.1161	24.32M
	1 $\times$ 1Conv	32.5468	0.9418	0.1271	24.10M
	DWConv	32.6839	0.9431	0.1272	18.87M

TABLE V: Ablation study results on different datasets using

1\times 1

convolution and depthwise separable convolution for generating query, key, and value in the MS-PVT.

Dataset	SRA	GFN	PSNR $\uparrow$	SSIM $\uparrow$	Time/s	Param
			31.7547	0.9232	0.0808	18.72M
	✓		31.8923	0.9234	0.0962	20.56M
		✓	32.1415	0.9267	0.1022	22.49M
Snow100K	✓	✓	32.2699	0.9268	0.1161	24.32M
			32.4960	0.9421	0.1027	18.72M
	✓		32.6308	0.9409	0.1210	20.56M

TABLE VI: Ablation Study of Linear SRA and GFN on Different Datasets.

	MACs/G
Model	RainDrop-A	Snow100K-L
Restormer [ 6 ]	743.5	627.8
DRSformer [ 11 ]	805.3	981.4
TransWeather [ 1 ]	32.6	28.7
LMQFormer [ 17 ]	120.8	91.1
WeatherRemover (Ours)	377.2	301.8

TABLE VII: Comparison of Transformer-based model inference computation on each dataset. Using MACs as a metric.

Model	Memory (MiB)	Param
DRSformer [ 11 ] (w/o MEFC)	140.9	32.6M
DRSformer [ 11 ]	147.8	33.6M
Restormer [ 6 ]	114.2	26.5M
TransWeather [ 1 ]	161.6	38.1M
LMQFormer [ 17 ]	27.1	2.2M
WeatherRemover (Ours)	109.6	24.3M

TABLE VIII: Comparison of Transformer-based model video memory consumption, measured in MiB.

Model	T4	4090	T4	4090	T4	4090
	Snow100K		RainDrop		Outdoor-Rain
Restormer [ 6 ]	0.66	0.07	0.99	0.11	0.78	0.08
TransWeather [ 1 ]	0.38	0.04	0.05	0.01	0.05	0.00
DRSformer [ 11 ]	1.06	0.11	2.73	0.30	0.77	0.02
WeatherRemover (Ours)	0.39	0.05	0.66	0.07	0.54	0.05

TABLE IX: Inference Time Comparison of Transformer-Based Multi-Weather Models.

Fig. 11: Model performance vs. resolution. Select raindrops as an example to measure memory usage during inference while disabling gradients.

Model	MACs/M	Memory/MiB	Param
Restormer [ 6 ]	58.06	3.59	189
DRSformer [ 11 ]	318.27	3.59	1036
TransWeather-E [ 1 ]	27.65	11.64	105
TransWeather-D [ 1 ]	55.3	11.64	207
WeatherRemover (Ours)	49.77	3.52	189

TABLE X: Performance comparison of different feed-forward networks.

All-in-One Image Restoration

All-in-one image restoration trains unified models to remove diverse and co-occurring degradations while preserving scene content.

See all 88 papers

PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
May 13, 2026Wei Dong, Han Zhou, Terry Ji +6Rectified Flow All-in-One Image Restoration
McMaster University · Shanghai Jiao Tong University
X-Restormer++: 1st Place Solution for the UG2+ CVPR 2026 All-Weather Restoration Challenge
May 13, 2026Youwei Pan, Leilei Cao, Yingfang Zhu +1All-in-One Image Restoration Image Restoration
TEX AI, Transsion Holdings
Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal
Aug 7, 2026Wanshu Fan, Yunzhe Zhang, Yue Shen +5Prompt Learning All-in-One Image Restoration
School of Software Engineering, Dalian University, Dalian, China · School of Mathematical Sciences, Dalian University of Technology, Dalian, China · The Hong Kong Polytechnic University, Hong Kong, China +2

May 13, 2026J/K move · Enter open · S save

PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow

Wei Dong, Han Zhou, Terry Ji, Guanhua Zhao+5

McMaster University · Shanghai Jiao Tong University

Rectified Flow All-in-One Image Restoration

Open paper AskPDF arXiv

Adverse weather removal (AWR) in real-world images remains challenging due to heterogeneous and unseen degradations, while distortion-driven training often yields overly smooth results. We propose PVRF, a unified framework that integrates zero-shot soft weather perceptions with velocity-constrained rectified-flow refinement. PVRF introduces an AWR-specific question answering module (AWR-QA) that uses frozen vision--language models (VLMs) to estimate soft probabilities of weather types and low-level attribute scores. These perceptions condition restoration networks via attribute-modulated normalization (AMN) and weather-weighted adapters (WWA), producing an anchor estimate for refinement. We then learn a terminal-consistent residual rectified flow with perception-adaptive source perturbation and a terminal-consistent velocity parameterization to stabilize learning near the terminal regime. Extensive experiments show that PVRF improves both fidelity and perceptual quality over state-of-the-art baselines, with strong cross-dataset generalization on single and combined degradations. Code will be released at https://github.com/dongw22/PVRF.

Abstract

All-in-One Image Restoration

Explore similar work

PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow

X-Restormer++: 1st Place Solution for the UG2+ CVPR 2026 All-Weather Restoration Challenge

Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal