We propose Unblur-SLAM, a novel RGB SLAM pipeline for sharp 3D reconstruction from blurred image inputs. In contrast to previous work, our approach is able to handle different types of blur and demonstrates state-of-the-art performance in the presence of both motion blur and defocus blur. Moreover, we adjust the computation effort with the amount of blur in the input image. As a first stage, our method uses a feed-forward image deblurring model for which we propose a suitable training scheme that can improve both tracking and mapping modules. Frames that are successfully deblurred by the feed-forward network obtain refined poses and depth through local-global multi-view optimization and loop closure. Frames that fail the first stage deblurring are directly modeled through the global 3DGS representation and an additional blur network to model multiple blurred sub-frames and simulate the blur formation process in 3D space, thereby learning sharp details and refined sub-frame poses. Experiments on several real-world datasets demonstrate consistent improvements in both pose estimation and sharp reconstruction results of geometry and texture.
Figures & tables
Figure 2: Method overview. Unblur-SLAM robustly handles varying blur by adaptively categorizing images into sharp, blurry, and heavily blurred levels (shown in different red shades). Since typically only a subset of frames is blurred, this improves both blur handling and average runtime. Both tracking and mapping modules optionally leverage the deblurring network. The red mapping module optimizes the 3DGS reconstruction using sliding-window (Eq. ( 14 )) and global (Eq. ( 15 )) losses, incorporating depth (Eq. ( 4 )) and pose from the blue tracking module.
Figure 3: Deblurring performance of our method when compared to other state-of-the-art offline methods.
Figure 4: Trajectory comparison with Droid-SLAM on the indoor MCD dataset [ 45 ] .
Method
ArchViz-1
ArchViz-2
ArchViz-3
Avg. ATE
ATE ↓
PSNR ↑
ATE ↓
PSNR ↑
ATE ↓
PSNR ↑
MBA-SLAM [ 52 ]
0.0075
28.45
0.0036
30.16
0.0141
27.85
0.0084
Ours
0.0075
28.76
0.0027
32.71
0.0067
30.09
0.0056
Table 1: Qualitative experiments on the ArchViz [ 52 ] synthetic dataset to demonstrate the performance comparison between our method and MBA-SLAM [ 52 ] . This dataset contains extreme blur cases, and we achieve the best results using ATE RMSE [m] and PSNR metrics.
Method
Deblur-NeRF [ 29 ]
DP-NeRF [ 22 ]
BAGS [ 40 ]
Deblur-GS [ 21 ]
DOF-GS [ 53 ]
Ours
PSNR ↑
23.98
24.12
24.17
24.21
24.12
27.45
Table 2: Comparison on the Deblur-NeRF defocus blur dataset [ 29 ] . We report PSNR (dB) ↑ metric and compare to several state-of-the-art methods. Our method outpeforms other methods by a wide margin, both quantitatively and qualitatively.
Dataset
Droid-SLAM [ 49 ]
Real_J
Ours*
Ours
TUM
0.380
0.400
0.352
0.336
MCD
0.138
0.139
0.155
0.128
Table 3: Comparison of trajectory errors ATE RMSE [m] ↓ . Our method yields the best tracking results. Real_J differs from Ours by replacing the DPDD dataset with the RealBlur_J dataset. ’Ours’ differs from ’Ours*’ by training on five ReplicaBlurry [ 13 ] sequences instead of one.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Mip-Splatting [ 57 ]
21.87
0.6270
0.3066
Deblur-NeRF [ 29 ]
25.63
0.7645
0.1820
BAD-Gaussians [ 62 ]
21.69
0.6471
0.1262
Deblurring 3DGS [ 21 ]
26.61
0.8224
0.1096
BAGS [ 40 ]
26.70
0.8237
0.0956
CoMoGaussian [ 24 ]
27.85
0.8431
0.0822
Table 4: Comparison against offline deblurring methods on the Deblur-NeRF dataset [ 29 ] . The comparison on both the motion blur and the defocus blur dataset of this established benchmark for offline deblurring methods demonstrates that Unblur-SLAM is able to outperform current state-of-the-art despite solving the harder online problem in which only partial data is available.
Figure 5: Qualitative comparison with MBA-SLAM [ 52 ] on the TUM dataset [ 48 ] . Our method yields sharper reconstruction results in many scene parts. The qualitative experimental results for the shown fr1_desk sequence were obtained through communication with the MBA-SLAM authors.
fr1_desk
fr2_xyz
fr3_office
Sharp ratio
16%
100%
37%
I 2 -SLAM [ 3 ]
27.23
32.06
28.91
Deblur-SLAM [ 13 ]
19.93
27.21
21.67
Ours
28.03
31.14
29.22
Table 5: Rendering comparison (PSNR) on TUM-RGBD dataset [ 48 ] . For fair comparison, we only use the same keyframes as in I2-SLAM [ 3 ] to obtain results, which are then compared against the sharp frames of the keyframes marked by I2-SLAM. The sharp ratio indicates the percentage of sharp keyframes in the input sequence.
Figure 6: Qualitative comparison of our method with ground truth sharp frames (top row) from the Deblur-NeRF defocus dataset [ 29 ] .
Figure 7: Qualitative comparison results on sharp frames manually labeled in I2-SLAM [ 3 ] . The first row is the ground truth image, the second row is the experimental results of our system.
Method
Splat-SLAM
I2 -SLAM
Ours
Ours w/o ref.
FPS ↑
1.24
0.095
0.74
0.85
Table 6: Runtime comparison in frames per second (FPS) . Ours w/o ref. indicates that we only use the deblurring model without refinement, while ”Ours” refers to our complete pipeline. We compute the average FPS of our system on the TUM dataset sequences: fr1_desk, fr2_xyz, and fr3_office. Note that Splat-SLAM is not a deblurring method and is only listed for reference.
Scenario
PSNR [dB]
With Fallback Mechanism
29.94
Without Fallback Mechanism
29.38
Table 7: Ablation of the fallback mechanism. ReplicaBlurry dataset simulates severe motion blur by averaging 36 Replica frames. We select the sequence with the largest variation in motion blur degree to demonstrate the effectiveness and necessity of our fallback mechanism.
Figure 8: Comparison of input images and deblurred results
Method
PSNR [dB]
FPS
Ours_no_threshold
27.18
0.41
Ours
29.46
0.74
Table 15
Symbol
Definition
Context
x
Pixel coordinates x=(u, v) on image plane
All
f
Focal length used in defocus kernel calculation
Eq. ( 1 )
d(t)
Camera depth at time t during exposure
Eq. ( 1 )
σΔ
Standard Deviation of the Score Differences
Sec. 7.4
C,Dr
Rendered 2D RGB and depth image
Eq. ( 3 )
d,d’
Original Gaussian depth( d ) and updated tracker’s depth( d’ )
Eq. ( 4 )
Table 16
Effect
Consistency
Metric
Acc. [%]
size
score
ARNIQA-CSIQ [ 2 ]
92.93
37.01
34.39
MANIQA-KADID [ 56 ]
96.55
6.35
6.13
CLIPIQA+ [ 50 ]
96.58
2.26
2.18
MUSIQ [ 15 ]
95.13
2.14
2.03
BRISQUE [ 34 ]
79.78
1.33
1.06
Table 8: Blur detection metric. We compared the accuracy and consistency scores of 39 blur metrics and selected 5 representative ones for reporting. We chose the metric with the highest consistency score among them as our blur detector (Sec. 3.1 ).
Figure 9: Qualitative results on various datasets. The left column shows the input images, while the right column displays the rendered images of our reconstructed 3D models.