While deep learning has advanced single-image deraining, existing models suffer from a fundamental limitation: they employ a static inference paradigm that fails to adapt to the complex, coupled degradations (e.g., noise artifacts, blur, and color deviation) of real-world rain. Consequently, restored images often exhibit residual artifacts and inconsistent perceptual quality. In this work, we present Derain-Agent, a plug-and-play refinement framework that transitions deraining from static processing to dynamic, agent-based restoration. Derain-Agent equips a base deraining model with two core capabilities: 1) a Planning Network that intelligently schedules an optimal sequence of restoration tools for each instance, and 2) a Strength Modulation mechanism that applies these tools with spatially adaptive intensity. This design enables precise, region-specific correction of residual errors without the prohibitive cost of iterative search. Our method demonstrates strong generalization, consistently boosting the performance of state-of-the-art deraining models on both synthetic and real-world benchmarks.
Figures & tables
Figure 1: Preliminary findings: a) static deraining leaves coupled residual degradations; b) agent-based adaptive refinement markedly improves visual quality.
Figure 2: Overview of the Derain-Agent framework. Given a preliminary deraining output Ibase , the perception stage analyzes residual degradations to produce an executable restoration program Π=(P,Λ) . The execution stage then applies the scheduled tools T(ℓ) sequentially, modulated by pixel-wise strength maps λ(ℓ) , to yield the final enhanced image Ifinal .
Figure 3: Visual results on LHP-Rain. Top: Baseline derainers; Bottom: Derain-Agent enhanced. Please zoom in for a better view.
Figure 4: Visual results on LHP-Rain. Top: Baseline derainers; Bottom: Derain-Agent enhanced. Please zoom in for a better view.
Figure 5: Visual results on RE-Rain. Top: Baseline derainers; Bottom: Derain-Agent enhanced. Please zoom in for a better view.
Strategy
PSNR
SSIM
LPIPS
Baseline
31.59
0.8835
0.219
IQA-Greedy
31.65
0.8864
0.214
Rollback-Replanning
31.69
0.8873
0.215
Derain-Agent
32.34
0.9074
0.177
Table 5: Ablation study on Execution Strategies under the same toolbox, backbone, and step budget.
Figure 6: Performance–throughput trade-off of different tool-path scheduling strategies across different backbones.
Figure 7: Visual comparison of object detection and instance segmentation on LHP-Rain. Top: Baseline derainers; Bottom: Derain-Agent enhanced. Our method significantly reduces false negatives and improves mask precision.
Figure 8: Qualitative comparison of local detail recovery. Strength modulation facilitates the reconstruction of fine spatial structures.
Strategy
PSNR
SSIM
Params (M)
FLOPs (G)
w / o SM
32.25
0.9037
21.42
4.804
w / SM
32.34
0.9074
22.93
4.901
Table 6: Ablation study on strength modulation (SM).
Training Stages
One
Two
Three
PSNR
32.04
32.34
32.16
SSIM
0.8986
0.9074
0.9014
Toolbox Size
One
Two
Three
PSNR
31.97
32.21
32.34
SSIM
0.8925
0.9034
0.9074
Table 7: Ablation study on training strategy and Toolbox Size.
Despite significant progress has been made in image deraining, we note that most existing methods are often developed for only specific types of rain degradation and fail to generalize across diverse real-world rainy scenes. How to effectively model different rain degradations within a universal framework is important for real-world image deraining. In this paper, we propose UniRain, an effective unified image deraining framework capable of restoring images degraded by rain streak and raindrop under both daytime and nighttime conditions. To better enhance unified model generalization, we construct an intelligent retrieval augmented generation (RAG)-based dataset distillation pipeline that selects high-quality training samples from all public deraining datasets for better mixed training. Furthermore, we incorporate a simple yet effective multi-objective reweighted optimization strategy into the asymmetric mixture-of-experts (MoE) architecture to facilitate consistent performance and improve robustness across diverse scenes. Extensive experiments show that our framework performs favorably against the state-of-the-art models on our proposed benchmarks and multiple public datasets.
Qianfeng Yang, Qiyuan Guan, Xiang Chen +3
Dalian Polytechnic University · Nanjing University of Science and Technology
Existing video deraining methods typically exploit neighboring frames through either explicit alignment or implicit spatiotemporal aggregation. Explicit alignment relies on accurate motion estimation, which can become unreliable under dense rain, while implicit aggregation avoids alignment but lacks explicit guidance on the directional and temporally coherent structure of rain. This leaves a gap between reliable temporal aggregation and explicit modeling of rain motion. To address these limitations, we propose FluidRain, a lightweight video derainer that uses divergence-free rain flow to guide Loop-in-Loop attention across scales and neighboring frames. Motivated by fluid mechanics, we model rain motion as a divergence-free image-space flow and use it to organize multi-scale and temporal aggregation. Specifically, FluidRain first estimates a rain-flow field for each frame and projects it onto the divergence-free subspace. The resulting flow steers window attention along rain streaks, enabling neighboring frames to be aggregated without explicit alignment. Since rain-flow structure is preserved across scales and nearby frames, Loop-in-Loop reuses the same attention operator across both dimensions, resulting in a three-frame model with only 0.80M parameters. Experiments on four benchmarks show that FluidRain remains competitive with substantially larger restoration models. We further examine how temporal evidence scales with different input views. To evaluate whether the model remains reliable when rain motion changes across frames, we introduce RainSyn-Gust, which injects controlled changes in rain-streak direction into existing benchmarks. We also develop a physics-based no-reference metric that evaluates real-rain removal without requiring clean targets.
Pu Wang, Yongcong Wang, Wenhao Li +6
Shandong University · Central South University · Nanjing University of Science and Technology +2
Real-world image restoration is challenging due to complex and interacting mixed degradations. Recent agent-based approaches address this problem by composing multiple task-specific restoration tools. However, empirical analysis reveals that their performance is fundamentally limited by implicitly constrained planning spaces and the lack of coordination among independently pretrained tools. To address these issues, we propose OPERA (Optimized Planning-Execution Restoration Agent), a framework that jointly optimizes restoration planning and tool execution in an end-to-end manner. On the planning side, OPERA uses reinforcement learning to directly optimize tool composition over a combinatorial plan space, with the final restoration quality as the reward. On the execution side, OPERA introduces agent-guided co-training of restoration tools, enabling them to learn cooperative behaviors under sequential composition. Extensive experiments on multi-degradation benchmarks and real-world datasets demonstrate that OPERA consistently outperforms both all-in-one restoration models and existing agent-based methods across diverse and complex degradation scenarios.
Feng Zhu, Shuyang Xie, Yihan Zeng +2
Harbin Institute of Technology · Huawei Noah’s Ark Lab