This workshop paper comprehensively reviews the First Challenge on Unified Removal of Raindrops and Reflections. The challenge aims to address a frequently encountered practical problem in the field of autonomous driving, i.e., raindrop-reflection composite degradation on rainy days. This competition attracted 149 registered participants and received 12 valid final submissions with corresponding fact sheets, significantly contributing to the progress of unified removal of raindrops and reflections. All the methods are developed and evaluated on our real-shot RainDrop and ReFlection (RDRF) dataset. A detailed analysis of the submitted methods and corresponding results is provided in this report, which highlights effective approaches and provides interesting insights for future research.
Figures & tables
Rank
Team
Leader
PSNR ↑
SSIM ↑
LPIPS ↓
Extra Data
Score ↑
1
X edit M aster
Jiagao Hu
32.3993
0.9645
0.0443
✓
41.8226
2
tysl
Minmin Yi
32.3006
0.9648
0.0416
✓
41.7405
3
GIST-IVL
Yeongjin Jeong
31.6556
0.9680
0.0388
✓
41.1416
4
ACVLAB
Jin-Hui Jiang
31.0060
0.9603
0.0432
✗
40.3933
5
SNU-ISPL
Youngjin Oh
30.7389
0.9572
0.0526
✓
40.0474
6
the_last_token
Vishwajeet Shukla
30.3074
0.9580
0.0559
✓
39.6075
Table 1: Quantitative results on the UR 3 Challenge test set. The bold and underline indicate the best and second-best results.
Rank
Team
Leader
PSNR ↑
SSIM ↑
LPIPS ↓
Extra Data
Score ↑
1
X edit M aster
Jiagao Hu
32.3993
0.9645
0.0443
✓
41.8226
2
tysl
Minmin Yi
32.3006
0.9648
0.0416
✓
41.7405
3
ACVLAB
Jin-Hui Jiang
31.0060
0.9603
0.0432
✗
40.3933
4
SNU-ISPL
Youngjin Oh
30.7389
0.9572
0.0526
✓
40.0474
5
GIST-IVL †
Yeongjin Jeong
30.5155
0.9573
0.0516
✓
39.8308
6
the_last_token
Vishwajeet Shukla
30.3074
0.9580
0.0559
✓
39.6075
Table 2: Quantitative results on the UR 3 Challenge test set. The results are obtained with submitted inference codes under strict single-frame setting. The bold and underline indicate the best and second-best results.
Figure 1: The pipeline of the method proposed by Team X edit M aster.
Figure 2: The pipeline of the method proposed by Team tysl.
Figure 3: The overall framework of Team GIST-IVL.
Figure 4: The architecture of Stage III in MAFNet-UR 3 .
Figure 5: The overall framework of Team ACVLAB.
Figure 6: The overall framework of Team SNU-ISPL.
Figure 7: The pipeline of the method proposed by Team the_last_token.
Figure 8: The pipeline of the method proposed by Team AIIA-Lab.
Figure 9: The pipeline of the method proposed by Team ustc_pi_lab.
When capturing images through glass surfaces or windshields on rainy days, raindrops and reflections frequently co-occur to significantly reduce the visibility of captured images. This practical problem lacks attention and needs to be resolved urgently. Prior de-raindrop, de-reflection, and all-in-one models have failed to address this composite degradation. To this end, we first formally define the unified removal of raindrops and reflections (UR3) task for the first time and construct a real-shot dataset, namely RainDrop and ReFlection (RDRF), which provides a new benchmark with substantial, high-quality, diverse image pairs. Then, we propose a novel diffusion-based framework (i.e., DiffUR3) with several target designs to address this challenging task. By leveraging the powerful generative prior, DiffUR3 successfully removes both types of degradations. Extensive experiments demonstrate that our method achieves state-of-the-art performance on our benchmark and on challenging in-the-wild images.
Xingyu Liu, Zewei He, Yu Chen +4
Huanjiang Laboratory, Zhuji, China · School of Aeronautics and Astronautics, Zhejiang University, Hangzhou, China · Hangzhou Institute of Technology, Xidian University, Hangzhou, China +1
Raindrops adhered to camera lens or windshield are inevitable in rainy scenes and can become an issue for many computer vision systems such as autonomous driving. Because raindrop appearance is affected by too many parameters, therefore it is unlikely to find an effective model based solution. Learning based methods are also problematic, because traditional learning method cannot properly model the complex appearance. Whereas deep learning method lacks sufficiently large and realistic training data. To solve it, in our work, we propose the first photo-realistic dataset of synthetic adherent raindrops for training. The rendering is physics based with consideration of the water dynamic, geometric and photometry. The dataset contains various types of rainy scenes and particularly the rainy driving scenes. Based on the modeling of raindrop imagery, we introduce a detection network which has the awareness of the raindrop refraction as well as its blurring. Based on that, we propose the removal network that can well recover the image structure. Rigorous experiments demonstrate the state-of-the-art performance of our proposed framework.
Zhixiang Hao, Shaodi You, Yu Li +2
State Key Laboratory of VR Technology and Systems, Beihang University, Beijing, China · 61-CSIRO · Tencent +2
Despite remarkable progress in reflection removal, current methods primarily exploit static image priors from a single frame and still suffer from severe residual artifacts due to the inherent ambiguity between the reflection and transmission layers. In this paper, we propose leveraging event signals to break this ambiguity. By employing event cameras to capture micro-dynamics, we reveal the differential motion between these two layers. We thereby present a novel event-driven reflection removal network, EvReflection, that utilizes these dynamic cues for layer separation. Specifically, we design a Micro-Dynamics Decoupler to disentangle layer-specific motions from event streams as priors, which then guide a Parallax-Attention Rectifier to cleanly remove artifacts from the RGB image. Furthermore, to address data scarcity, we develop a parallax-aware simulation pipeline and construct the EVR2 benchmark dataset, the first real-world dataset for this task. Extensive experiments demonstrate that EvReflection achieves state-of-the-art performance on both synthetic and real-world benchmarks, surpassing the best competing method by more than 1.6 dB and 1.2 dB in PSNR, respectively. The code, dataset, and pre-trained models are available at https://github.com/JiaxiaoWang/EvReflection.
Jiaxiao Wang, Dachun Kai, Huyue Zhu +3
University of Science and Technology of China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center