Spectral-Structured Diffusion for Single-Image Rain Removal

Yucheng Xing; Xin Wang

cs.CVMar 10, 2026

Spectral-Structured Diffusion for Single-Image Rain Removal

Authors: Yucheng Xing, Xin Wang

Organizations: Department of Electrical and Computer Engineering, Stony Brook University, Stony Brook, NY 11794, USA

Abstract

Rain streaks manifest as directional and frequency-concentrated structures that overlap across multiple scales, making single-image rain removal particularly challenging. While diffusion-based restoration models provide a powerful framework for progressive denoising, standard spatial-domain diffusion does not explicitly account for such structured spectral characteristics. We introduce SpectralDiff, a spectral-structured diffusion-based framework tailored for single-image rain removal. Rather than redefining the diffusion formulation, our method incorporates structured spectral perturbations to guide the progressive suppression of multi-directional rain components. To support this design, we further propose a full-product U-Net architecture that leverages the convolution theorem to replace convolution operations with element-wise product layers, improving computational efficiency while preserving modeling capacity. Extensive experiments on synthetic and real-world benchmarks demonstrate that SpectralDiff achieves competitive rain removal performance with improved model compactness and favorable inference efficiency compared to existing diffusion-based approaches.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of the rain layer adding process. 1(a) is the original image B , 1(b) - 1(d) are example rain-streak layers $\textbf{R}_{d}$ $(d\in[1,3])$ , and 1(e) is the final rainy image O .

Figure 2: Illustration of forward / reverse diffusion process.

Panel of Figure 3 — Figure 3: Illustration of rain layers in the frequency domain.

Figure 4: Structure of our Product U-Net and Product Res-Block.

	Rain1400			RainCityscapes			SPA-Data
	PSNR	SSIM	Time	PSNR	SSIM	Time	PSNR	SSIM	Time
DSC [ 28 ]	27.60	0.825	0.320	19.90	0.795	0.341	24.26	0.696	0.321
LayerPrior [ 27 ]	24.75	0.780	7.398	13.71	0.736	4.461	33.03	0.867	5.065
DDN [ 13 ]	28.33	0.815	0.001	21.99	0.819	0.001	34.06	0.869	0.002
PReNet [ 35 ]	33.40	0.901	0.007	28.56	0.902	0.007	35.55	0.886	0.014
SIRR [ 48 ]	30.70	0.874	0.002	22.08	0.846	0.002	36.39	0.891	0.002

Table 1: Overall performance (PSNR (

\uparrow

), SSIM (

\uparrow

)) and efficiency (Time[

s^{\dagger}

] (

\downarrow

)) comparison among different deraining methods.

Diffusion ( $S=10$ )	PSNR	SSIM	FLOPs	Params
Spatial ( $\mathbf{\epsilon}_{s}$ )	30.93	0.853	376.60	6.08
Spectral ( $\lvert\mathbf{x}_{f}\rvert$ )	6.89	0.114	376.60	6.08
Spectral ( $\angle\mathbf{x}_{f}$ )	24.27	0.806	376.60	6.08
Spectral ( $\mathbf{\epsilon}_{f}$ )	14.68	0.246	1269.59	16.66
Spectral ( $\mathbf{M}_{d}\odot\mathbf{\epsilon}_{f}$ )	30.86	0.857	1269.59	16.66

Table 2: Performance comparison of diffusion models deraining in spatial domain and spectral domain on Rain1400.

( $S=10$ )	PSNR	SSIM	FLOPs	Params
SpectralDiff (Conv)	30.86	0.857	1269.59	16.66
SpectralDiff (Prod)	30.39	0.860	199.81	3.15

Table 3: Performance comparison of our SpectralDiff model with different U-Nets on Rain1400.

Image Deraining

See all 10 papers

UniRain: Unified Image Deraining with RAG-based Dataset Distillation and Multi-objective Reweighted Optimization
Mar 4, 2026Qianfeng Yang, Qiyuan Guan, Xiang Chen +3Image Deraining Image Enhancement
Dalian Polytechnic University · Nanjing University of Science and Technology
Derain-Agent: A Plug-and-Play Agent Framework for Rainy Image Restoration
Mar 12, 2026Zhaocheng Yu, Xiang Chen, Runzhe Li +4Image Deraining Agent-Based Image Restoration
FluidRain: Incompressible Rain Flow as an Attention Bias for Loop-in-Loop Video Deraining
Sep 24, 2026Pu Wang, Yongcong Wang, Wenhao Li +6Attention Mechanisms Image Deraining
Shandong University · Central South University · Nanjing University of Science and Technology +2

Mar 4, 2026J/K move · Enter open · S save

UniRain: Unified Image Deraining with RAG-based Dataset Distillation and Multi-objective Reweighted Optimization

Qianfeng Yang, Qiyuan Guan, Xiang Chen, Jiyu Jin+2

Dalian Polytechnic University · Nanjing University of Science and Technology

Image Deraining Image Enhancement

Open paper AskPDF arXiv

Despite significant progress has been made in image deraining, we note that most existing methods are often developed for only specific types of rain degradation and fail to generalize across diverse real-world rainy scenes. How to effectively model different rain degradations within a universal framework is important for real-world image deraining. In this paper, we propose UniRain, an effective unified image deraining framework capable of restoring images degraded by rain streak and raindrop under both daytime and nighttime conditions. To better enhance unified model generalization, we construct an intelligent retrieval augmented generation (RAG)-based dataset distillation pipeline that selects high-quality training samples from all public deraining datasets for better mixed training. Furthermore, we incorporate a simple yet effective multi-objective reweighted optimization strategy into the asymmetric mixture-of-experts (MoE) architecture to facilitate consistent performance and improve robustness across diverse scenes. Extensive experiments show that our framework performs favorably against the state-of-the-art models on our proposed benchmarks and multiple public datasets.

Abstract

Image Deraining

Explore similar work

UniRain: Unified Image Deraining with RAG-based Dataset Distillation and Multi-objective Reweighted Optimization

Derain-Agent: A Plug-and-Play Agent Framework for Rainy Image Restoration

FluidRain: Incompressible Rain Flow as an Attention Bias for Loop-in-Loop Video Deraining