UniDeblur: Unified Motion Deblurring via Fourier Convolution Learning
Authors: Xintian Mao, Jiansheng Wang, Huaiyuan Wang, Qingli Li, Yan Wang
Organizations: Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, China. · Shanghai Institute of Technical Physics, Chinese Academy of Sciences, China.
Image deblurring is a fundamental task in computer vision, yet existing methods mainly specialize in isolated scenarios. Consequently, the deblurring performance of these models drops significantly across scenarios with different cameras and image contents. This motivates the development of a unified motion deblurring framework that can handle diverse blur patterns. To this end, we construct a multi-source dataset and develop a data pruning and quality control pipeline. Then, to restore blur images with various motion blur kernels across different scenarios, we propose a unified motion deblurring framework via Fourier Convolution Learning. Existing deblurring networks usually rely on pixel-level mappings and overlook the convolutional nature of blur. Thus, the network mostly fits the image content itself rather than learning the convolution process, making it difficult to handle multi-source blur images. Instead of relying on spatial mapping, we perform blur-adaptive global filter learning in Fourier space, which avoids the limited spatial support of kernel estimation. Extensive experiments demonstrate that our unified model effectively handles motion blurs across diverse sources and scenes, achieving competitive performance across both in-domain and OOD benchmarks. Code has been released at github.com/DeepMed-Lab-ECNU/Single-Image-Deblur.
Figures & tables
Figure 1: UniDeblur via Fourier Convolution Learning.
Figure 2: Visualization of blur images, deconvolution kernels and kernels by FCL on various datasets.
Dataset
Acquisition
Images
Videos
Camera
Resolution
Content
GoPro Nah et al. (2017)
Averaged
3.2K
33
GoPro HERO4
1280 × 720
Outdoor Scenes
HIDE Shen et al. (2019)
Averaged
8.4K
31
GoPro HERO
1280 × 720
Pedestrian in Outdoor
REDS Nah et al. (2021)
Averaged
30K
300
GoPro HERO6
1280 × 720
Outdoor Scenes
RSBlur Rim et al. (2022)
Beam Splitter
13.3K
697
-
≈ 1915 × 1200
Outdoor Scenes
RealBlur Rim et al. (2020)
Beam Splitter
9.1K
232
Sony A7RM3
≈ 660 × 750
Static Object at Night
LOLBlur Zhou et al. (2022)
Simulated
12K
200
Sony RX10 IV
1120 × 640
Low-light Scenes
Table 1: Summary of various motion deblurring datasets.
Figure 3: Architecture of UniDeblur, Fourier Convolution Learning and Basicblocks.
Figure 4: Various flaws in OOD datasets.
Train
Test
Dataset
Images
Patches
Resolution
Images
Removed
Resolution
Step
GoPro Nah et al. (2017)
2,103
16,824
512 × 512
1,111
0
1280 × 720
1
HIDE Shen et al. (2019)
6,397
12,794
720 × 720
203
0
1280 × 720
10
REDS Nah et al. (2021)
24,000
48,000
720 × 720
299
1
1280 × 720
10
RSBlur Rim et al. (2022)
8,878
53,268
720 × 720
336
0
≈ 1915 × 1200
10
RealBlur Rim et al. (2020)
7,516
15,032
≈ 660 × 750
1,960
0
640 × 640
1
Table 2: Summary of training and testing datasets.
GoPro
HIDE
REDS
RSBlur
RealBlur
LOLBlur
Method
PSNR
SSIM
MUSIQ
PSNR
SSIM
MUSIQ
PSNR
SSIM
MUSIQ
PSNR
SSIM
MUSIQ
PSNR
SSIM
PSNR
SSIM
GT
-
1.000
47.97
-
1.000
60.43
-
1.000
66.37
-
1.000
37.31
-
1.000
-
1.000
Blur
25.64
0.793
24.87
23.94
0.759
26.28
26.34
0.772
39.01
28.74
0.850
21.90
30.08
0.874
33.82
0.904
MIMOUNet
30.47
0.906
39.91
28.93
0.888
47.22
32.14
0.908
62.08
32.30
0.917
30.57
33.01
0.925
41.70
0.977
DeepRFT
31.92
0.932
41.11
30.84
0.921
51.70
32.36
0.912
61.21
33.45
0.937
32.40
33.79
0.938
43.38
0.982
NAFNet
32.81
0.940
43.61
31.11
0.924
54.99
33.66
0.931
64.50
33.89
0.941
34.62
33.93
0.939
44.22
0.985
Table 3: Comparisons on in-domain datasets.
MIORe
VAR-MIORe
BSD1
BSD2
BSD3
FoundIR
FLOPs
Mem
Method
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
(G)
(GB)
GT
0.3076
55.70
0.3045
56.81
0.2695
55.05
0.2919
51.42
0.2658
49.70
0.3207
61.42
-
-
Blur
0.2120
36.75
0.1953
26.85
0.2097
40.35
0.1854
34.15
0.1740
30.89
0.1477
23.76
-
-
MIMOUNet
0.2422
44.41
0.2147
32.97
0.2415
49.08
0.2306
42.24
0.2111
41.53
0.2756
48.21
67
12
DeepRFT
0.2797
52.01
0.2299
39.91
0.2656
52.70
0.2787
48.23
0.2395
47.76
0.3146
53.90
80
24
NAFNet
0.2704
51.00
0.2224
37.88
0.2886
53.74
0.2923
49.28
0.2649
48.02
0.3102
53.82
63
35
Table 4: Comparisons on OOD datasets.
Figure 5: Visual comparison on GoPro, RSBlur, RealBlur, MIORe and VAR-MIORe.
Backbone
Fourier Convolution Learning
GoPro
HIDE
RSBlur
LOLBlur
FLOPs
Mem
Exp
Basicblock
Level1 ∼ 3
Dim
Basicblock
mode
MS
Num
Dim
PSNR
PSNR
PSNR
PSNR
(G)
(GB)
1
NAFBlock
[1, 4, 8]
32
-
-
-
-
-
31.96
30.27
33.26
43.32
14.20
21
2
NAFBlock
[1, 4, 8]
32
ConvBlock
⊕
×
4
48
32.08
30.41
33.35
43.49
15.20
22
3
NAFBlock
[1, 4, 8]
32
ConvBlock
⊙
×
4
48
32.23
30.56
33.51
43.62
15.20
22
4
NAFBlock
[1, 4, 8]
32
MLPBlock
⊙
×
4
48
32.27
30.59
33.52
43.61
15.36
22
5
NAFBlock
[1, 4, 8]
32
GatedBlock
⊕
×
4
48
32.23
30.57
33.51
43.63
15.52
22
Table 5: Ablation for Fourier Convolution Learning. Exp is the index of each experiment. Dim is the channel number of Level 1. Num means the number of basicblock of FCL. Mode means get F(xout) via ⊙ (multiplication) or ⊕ (addition). MS denotes applying FCL in multi-scale or not. Exp 6 and 13 are trained without auxiliary supervision.
Figure 6: Visualization of the frequency filters F(k) and corresponding spatial kernels k at Levels 1 ∼ 3, together with the averaged xin/xout and the corresponding B/S/kblur/kdeblur on GoPro, RSBlur, and VAR-MIORe.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Visualization of the frequency filters F(k) and corresponding spatial kernels k at Levels 1 ∼ 3, together with the averaged xin/xout and the corresponding B/S/kblur/kdeblur on HIDE, REDS, LOLBlur, BSD and FoundIR.
Figure 8: Color shift and residual blur of ReLoBlur dataset.
GT
Blur
PromptIR
MoCEIR
EVSSM
UniDeblur-L
Dataset
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
CLIPIQA
MUSIQ
ReLoBlur
0.2103
40.14
0.1889
37.20
0.2546
41.30
0.2169
39.85
0.2841
43.61
0.2913
43.74
RealLOL
-
-
0.1370
26.12
0.1979
29.74
0.2028
29.69
0.1995
30.04
0.2088
30.28
Appendix
Table 6: Comparisons on ReLoBlur and RealLOL.
GoPro
HIDE
RealBlur
LOLBlur
REDS
RSBlur
train
PSNR
PSNR
PSNR
PSNR
PSNR
PSNR
GoPro
32.60
30.44
30.04
35.24
27.08
26.22
GoPro+HIDE
32.88
31.11
29.99
34.94
27.15
28.62
GoPro+HIDE+RealBlur
32.53
30.80
33.67
37.60
27.81
30.10
GoPro+HIDE+LOLBlur
32.63
30.93
30.63
43.46
28.58
28.68
GoPro+HIDE+RealBlur+LOLBlur
32.45
30.75
33.72
43.28
29.62
30.31
Appendix
Table 7: Performance of NAFNet32 trained by different datasets .
Figure 9: Visualization comparisons on GoPro dataset.
Figure 10: Visualization comparisons on HIDE dataset.
Figure 11: Visualization comparisons on REDS dataset.
Figure 12: Visualization comparisons on RSBlur dataset.
Figure 13: Visualization comparisons on RealBlur and LOLBlur dataset.
Figure 14: Visualization comparisons on MIORe and VAR-MIORe dataset.
Figure 15: Visualization comparisons on BSD dataset.
Figure 16: Visualization comparisons on FoundIR dataset.
Figure 17: Visualization comparisons on ReLoBlur dataset.
Department of Artificial Intelligence, Hanyang University · Department of Intelligence Convergence, Hanyang University · Department of Computer Science, Hanyang University