SSMB: Self-Supervised Local Feature Detection under Motion Blur

Zhenjun Zhao; Fabio Bellavia; Wenting Wang; Fan Zhu; Jiajun Wu; Suryansh Kumar; Mingqiang Wei; Haoang Li

Fig. 1: SSMB under motion blur. Left: a real-world blurred image from RealBlur [ 28 ] (top) and the keypoints (green circles) detected by SSMB on it (bottom), without any deblurring preprocessing. Right: image matching accuracy (Mean Matching Accuracy (MMA) [ 16 ] at a 3-pixel threshold) on the Blur-HPatches Tough split [ 27 ] under motion blur. SSMB, our self-supervised sparse detector trained without handcrafted or external supervision, substantially outperforms all sparse baselines.

Fig. 2: Network architecture of SSMB. Given an input image

I\in\mathbb{R}^{H\times W\times 3}

, the encoder progressively downsamples the feature map through four cascaded blocks, from

H\times W

to

\frac{H}{8}\times\frac{W}{8}

, while expanding the channel dimension. Each block contains an RSMA layer, within which the Local Discriminability Enhancement (LDE) module is inserted between its two internal branches, and a subsequent RCAB layer. The detector head then takes the encoder output and branches into two independent towers: a probability tower (two convolutions followed by a Softmax) that produces a keypoint probability map

\mathbf{P}\in\mathbb{R}^{H\times W}

, and an offset tower (two convolutions followed by a Sigmoid) that produces sub-pixel position offsets

\boldsymbol{\delta}\in\mathbb{R}^{2\times H_{c}\times W_{c}}

(

H_{c}=H/8

,

W_{c}=W/8

). Keypoints are localized at the cells where

\mathbf{P}

peaks, refined by

\boldsymbol{\delta}

.

Fig. 3: Architecture of the LDE module. The input feature

\mathbf{u}

is first normalised by Layer Norm to produce

\mathbf{x}=\text{LN}(\mathbf{u})

. Two parallel branches then operate on

\mathbf{x}

: a depthwise

3{\times}3

convolution

f_{\mathrm{local}}

that captures local gradient structure, and a lightweight, feature-dependent gating branch

g_{\mathrm{blur}}

consisting of a two-layer pointwise convolution bottleneck with a ReLU in between, followed by a Sigmoid activation. The branch outputs are fused by element-wise multiplication (

\odot

) and added to

\mathbf{x}

via an inner residual, and then to the original input

\mathbf{u}

via an outer residual, yielding

\text{LDE}(\mathbf{u})=\mathbf{u}+\text{LN}(\mathbf{u})+f_{\text{local}}(\text{LN}(\mathbf{u}))\odot g_{\text{blur}}(\text{LN}(\mathbf{u}))

.

Fig. 4: Two-stage self-supervised training pipeline of SSMB. (a) Stage 1 (geometric pretraining) : SSMB is pretrained on synthetic geometric shapes with corner ground truth rendered directly from the synthetic images (no handcrafted or learned detector involved), supervised by the cross-entropy loss

\mathcal{L}_{\text{syn}}

. (b) Stage 2 (blur-aware training) : a single SSMB network with shared weights, initialized from the Stage 1 checkpoint, processes sharp

I^{s}

and blurred

I^{b}

GoPro image pairs. The sharp-image branch is supervised by the homographic-adaptation loss

\mathcal{L}_{\text{ha}}

. The blur-image branch is supervised by the blur consistency loss

\mathcal{L}_{\text{blur}}

, the position consistency loss

\mathcal{L}_{\text{pos}}

, and the spatial diversity loss

\mathcal{L}_{\text{div}}

. All four terms are combined into the total training loss

\mathcal{L}_{\text{total}}=\sum_{i}\lambda_{i}\mathcal{L}_{i}

,

i\in\{\text{ha},\text{blur},\text{pos},\text{div}\}

.

	Reference: Sharp Target: Blur			Reference: Blur Target: Blur
Method	E ASY $\uparrow$	H ARD $\uparrow$	T OUGH $\uparrow$	E ASY $\uparrow$	H ARD $\uparrow$	T OUGH $\uparrow$
SIFT [ 13 ]	55.92	56.80	53.49	56.99	53.49	45.94
Key.Net [ 15 ]	60.34	54.71	44.69	62.77	58.17	49.25
SuperPoint [ 14 ]	65.64	62.22	52.84	58.60	50.03	43.28
LF-Net [ 52 ]	63.54	61.19	56.78	60.45	59.07	57.71
D2-Net [ 16 ]	49.71	47.30	44.32	51.80	51.05	50.53

TABLE I: Keypoint detection repeatability on the Blur-HPatches dataset [ 27 ] . For compactness, we only report the overall repeatability. The disaggregated results for viewpoint and illumination changes can be found in Sec. -A of the Appendix.

	Reference: Sharp Target: Deblur						Reference: Deblur Target: Deblur
	SRN-DeblurNet [ 26 ]			DeblurGAN-v2 [ 25 ]			SRN-DeblurNet [ 26 ]			DeblurGAN-v2 [ 25 ]
Method	E ASY $\uparrow$	H ARD $\uparrow$	T OUGH $\uparrow$	E ASY $\uparrow$	H ARD $\uparrow$	T OUGH $\uparrow$	E ASY $\uparrow$	H ARD $\uparrow$	T OUGH $\uparrow$	E ASY $\uparrow$	H ARD $\uparrow$	T OUGH $\uparrow$
SIFT [ 13 ]	56.62	55.36	53.83	57.63	56.52	56.50	59.75	58.13	50.63	59.44	57.98	51.21
Key.Net [ 15 ]	63.28	58.01	47.10	63.99	59.16	49.35	62.86	60.44	50.74	62.73	60.58	52.96
SuperPoint [ 14 ]	67.72	64.05	55.26	67.95	65.86	58.22	66.38	63.16	49.52	66.50	63.71	52.09
LF-Net [ 52 ]	62.22	59.90	54.73	62.59	60.24	54.81	63.06	62.03	57.28	63.00	61.79	57.85

TABLE II: Keypoint detection repeatability on the Deblur-HPatches dataset [ 27 ] , where blurred images are first restored using SRN-DeblurNet or DeblurGAN-v2 before detection.

Detector-free (Dense/Semi-dense)
	Overall MMA $\uparrow$			Illumination MMA $\uparrow$			Viewpoint MMA $\uparrow$
Method	$@3$ px	$@5$ px	$@10$ px	$@3$ px	$@5$ px	$@10$ px	$@3$ px	$@5$ px	$@10$ px
LoFTR [ 33 ]	26.40	35.28	64.96	39.97	42.32	73.23	13.29	28.48	56.96
MatchFormer [ 35 ]	38.08	48.86	73.67	57.69	60.57	86.33	19.14	37.54	61.45
ASpanFormer [ 36 ]	22.91	33.20	65.51	29.35	30.87	63.24	16.69	35.46	67.70
RoMa v2 [ 39 ]	28.51	54.58	88.54	36.24	67.24	96.47	21.04	42.34	80.89

TABLE III: Image matching on the Blur-HPatches dataset [ 27 ] . Mean matching accuracy (MMA) is reported at multiple pixel thresholds. first , second , and third best results are highlighted within sparse methods only, as detector-free methods operate on full images and are not directly comparable to sparse detectors.

Detector-free (Dense/Semi-dense)
	Blur-to-Sharp				Blur-to-Blur
Method	Pose AUC $\uparrow$				Pose AUC $\uparrow$
Method	$@5$ °	$@10$ °	$@20$ °	$@30$ °	$@5$ °	$@10$ °	$@20$ °	$@30$ °
LoFTR [ 33 ]	10.14	23.15	41.94	54.72	6.64	19.53	41.54	52.25
MatchFormer [ 35 ]	2.50	4.32	10.45	18.39	0.00	2.55	9.74	14.68
ASpanFormer [ 36 ]	4.31	11.00	20.20	28.87	3.56	7.40	15.85	22.31

TABLE IV: Relative pose estimation on the ArchViz dataset [ 44 ] . The Area Under the Cumulative Curve (AUC) is reported at multiple angular error thresholds in blur-to-sharp and blur-to-blur settings. First , second , and third best results are highlighted within sparse methods only.

Detector-free (Dense/Semi-dense)
	Correctly Localized Queries (%) $\uparrow$
Method	Day	Night
	(0.25m,2°) / (0.5m,5°) / (1.0m,10°)
LoFTR [ 33 ]	49.3 / 63.3 / 80.1	21.4 / 41.8 / 67.3
MatchFormer [ 35 ]	6.9 / 24.0 / 76.0	3.1 / 14.3 / 49.0
ASpanFormer [ 36 ]	43.3 / 57.2 / 76.5	12.4 / 29.6 / 58.4

TABLE V: Visual localization on the Aachen Day-Night benchmark [ 3 ] . The percentage of queries successfully localized within three error thresholds is reported. First , second , and third best results are highlighted within sparse methods only. † RoMa v2 could not be evaluated on this benchmark due to insufficient system memory during large-scale bundle adjustment (

4,328

database images).

Detector-free (Dense/Semi-dense)
Method	240 $\times$ 320 pixels $\downarrow$	480 $\times$ 640 pixels $\downarrow$
LoFTR [ 33 ]	30.35	75.22
MatchFormer [ 35 ]	22.81	37.92
ASpanFormer [ 36 ]	53.22	108.54
RoMa v2 [ 39 ]	486.64	497.96
Sparse

TABLE VI: Computational cost (in millisecond) for keypoint detection on a single image, measured on an NVIDIA RTX 3090 GPU (Intel Xeon E5-2686 v4 CPU for SIFT). † For D2-Net, detection and description cannot be separated in a single forward pass, so the reported time includes both.

Loss components
	Repeatability (%)
Variant	Blur-to-Sharp	Blur-to-Blur
w/o $\mathcal{L}_{\text{ha}}$	69.03	69.78
w/o $\mathcal{L}_{\text{blur}}$	65.81	67.84
w/o $\mathcal{L}_{\text{pos}}$	72.69	73.77
w/o $\mathcal{L}_{\text{div}}$	64.07	65.26

TABLE VII: Ablation study. Repeatability is reported on the Blur-HPatches Tough split [ 27 ] under blur-to-sharp and blur-to-blur settings.

Fig. 5: Effect of the LDE module on the predicted probability map. Top row: probability heatmaps on the same real-world blurred image from the RWBI dataset [ 57 ] , overlaid on a lightened version of the input. Full SSMB produces dense, well-localized responses that closely trace real structure such as storefront signage, arched window frames, and shop dividers. Without LDE, the response collapses toward near-zero almost everywhere, with only a handful of residual peaks remaining. Bottom row: the pixel-value histogram of the probability map, together with the extracted keypoints (green circles) for each variant. Full SSMB’s keypoints densely and precisely trace the storefront geometry. In contrast, the w/o LDE variant yields far fewer keypoints overall, which no longer trace real structure. Most fall on featureless regions such as bare pavement, with a spacing that resembles a uniform grid rather than genuine edge localization, consistent with the near-flat residual field revealed by the histogram. This visually explains the dramatic repeatability drop observed in Tab. VII (from

77.16\%

to

33.83\%

on the blur-to-sharp Tough setting). Without local discriminability, the network can no longer anchor its predictions to repeatable geometric structure.

Fig. 6: Qualitative detection results on real-world blurred images from the RWBI dataset [ 57 ] . SSMB detects well-localized and consistently distributed keypoints under real camera motion blur, while several baselines either miss salient structure or produce noisy responses in heavily blurred regions. The red and blue boxes mark two representative regions, shown enlarged below each image for closer comparison. Best viewed in color.

Fig. 7: Qualitative matching results on a real-world sharp-blur image pair from the RealBlur dataset [ 28 ] . Top: the original image pair (left: sharp, right: blurred). Bottom: SSMB detects well-localized and repeatable keypoints from both images for matching under real camera motion blur and viewpoint changes. Green lines indicate correct matches. Best viewed in color.

	Reference: Sharp Target: Sharp
Method	Viewpoint $\uparrow$	Illumination $\uparrow$	Overall $\uparrow$
SIFT [ 13 ]	60.29	60.44	60.36
Key.Net [ 15 ]	68.99	67.47	68.24
SuperPoint [ 14 ]	69.53	68.92	69.23
LF-Net [ 52 ]	68.41	73.61	70.96
D2-Net [ 16 ]	53.99	62.80	58.32

TABLE VIII: Keypoint detection repeatability on the HPatches dataset [ 43 ] . For each method, we report the average repeatability score for viewpoint changes, illumination changes, and for all image sequences.

	E ASY			H ARD			T OUGH
Method	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$
SIFT [ 13 ]	50.64	61.39	55.92	51.55	62.23	56.80	47.36	59.83	53.49
Key.Net [ 15 ]	58.02	62.74	60.34	50.54	59.02	54.71	39.20	50.38	44.69
SuperPoint [ 14 ]	66.06	65.21	65.64	61.96	62.49	62.22	51.60	54.13	52.84
LF-Net [ 52 ]	59.57	67.65	63.54	57.64	64.87	61.19	52.14	61.57	56.78
D2-Net [ 16 ]	44.30	55.31	49.71	41.60	53.19	47.30	38.39	50.47	44.32

TABLE IX: Keypoint detection repeatability on the Blur-HPatches dataset under the blur-to-sharp setting. SSMB achieves the highest illumination repeatability and ranks first in overall repeatability at the Tough level.

	E ASY			H ARD			T OUGH
Method	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$
SIFT [ 13 ]	56.67	57.31	56.99	53.15	53.85	53.49	46.23	46.63	45.94
Key.Net [ 15 ]	61.81	63.77	62.77	56.82	59.57	58.17	45.68	52.94	49.25
SuperPoint [ 14 ]	58.01	59.22	58.60	49.69	50.37	50.03	42.34	44.25	43.28
LF-Net [ 52 ]	52.45	68.74	60.45	51.20	67.21	59.07	49.68	66.02	57.71
D2-Net [ 16 ]	46.65	57.14	51.80	45.84	56.44	51.05	45.11	56.13	50.53

TABLE X: Keypoint detection repeatability on the Blur-HPatches dataset under the blur-to-blur setting. SSMB achieves the highest illumination and overall repeatability across all difficulty levels.

	E ASY			H ARD			T OUGH
Method	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$
SIFT [ 13 ]	53.98	59.35	56.62	52.44	58.38	55.36	47.95	59.92	53.83
Key.Net [ 15 ]	62.86	63.72	63.28	56.37	59.71	58.01	42.08	52.30	47.10
SuperPoint [ 14 ]	68.65	66.76	67.72	65.33	62.73	64.05	54.18	56.37	55.26
LF-Net [ 52 ]	54.40	70.31	62.22	52.52	67.53	59.90	47.77	61.93	54.73
D2-Net [ 16 ]	46.72	57.08	51.81	44.30	54.87	49.49	40.00	52.07	45.94

TABLE XI: Keypoint detection repeatability on deblurred images from SRN-DeblurNet [ 26 ] under deblur-to-sharp setting. The bottom row shows the results of SSMB on the corresponding blurred images.

	E ASY			H ARD			T OUGH
Method	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$
SIFT [ 13 ]	60.31	59.16	59.75	57.98	58.27	58.13	48.56	52.78	50.63
Key.Net [ 15 ]	62.71	63.01	62.86	59.69	61.22	60.44	46.57	55.06	50.74
SuperPoint [ 14 ]	67.77	64.93	66.38	64.61	61.67	63.16	49.04	50.02	49.52
LF-Net [ 52 ]	55.36	71.03	63.06	54.60	69.72	62.03	49.17	65.68	57.28
D2-Net [ 16 ]	49.05	58.32	53.60	48.58	57.57	53.00	45.70	56.33	50.93

TABLE XII: Keypoint detection repeatability on deblurred images from SRN-DeblurNet [ 26 ] under deblur-to-deblur setting. The bottom row shows the results of SSMB on the corresponding blurred images.

	E ASY			H ARD			T OUGH
Method	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$
SIFT [ 13 ]	56.28	59.03	57.63	54.02	59.10	56.52	51.29	61.88	56.50
Key.Net [ 15 ]	63.78	64.20	63.99	57.46	60.91	59.16	44.92	53.92	49.35
SuperPoint [ 14 ]	68.85	67.02	67.95	66.83	64.84	65.86	57.37	59.10	58.22
LF-Net [ 52 ]	55.01	70.43	62.59	53.03	67.70	60.24	47.73	62.14	54.81
D2-Net [ 16 ]	47.68	57.77	52.64	44.96	55.64	50.21	40.17	51.80	45.88

TABLE XIII: Keypoint detection repeatability on deblurred images from DeblurGAN-v2 [ 25 ] under deblur-to-sharp setting. The bottom row shows the results of SSMB on the corresponding blurred images.

	E ASY			H ARD			T OUGH
Method	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$	Viewpoint $\uparrow$	Illumination $\uparrow$	Total $\uparrow$
SIFT [ 13 ]	59.17	59.73	59.44	57.33	58.66	57.98	50.12	52.33	51.21
Key.Net [ 15 ]	62.38	63.08	62.73	59.39	61.81	60.58	49.12	56.93	52.96
SuperPoint [ 14 ]	67.49	65.49	66.50	64.89	62.48	63.71	51.85	52.35	52.09
LF-Net [ 52 ]	55.57	70.69	63.00	54.58	69.25	61.79	50.48	65.49	57.85
D2-Net [ 16 ]	49.40	58.62	53.93	48.65	58.08	53.29	45.69	55.97	50.74

TABLE XIV: Keypoint detection repeatability on deblurred images from DeblurGAN-v2 [ 25 ] under deblur-to-deblur setting. The bottom row shows the results of SSMB on the corresponding blurred images.

Fig. 8: Image matching on the Blur-HPatches dataset [ 27 ] . Left: Mean Matching Accuracy (MMA) curves at different thresholds. Solid lines denote sparse detector + non-learned matching methods, dashed lines with triangle markers (

\blacktriangle

) denote sparse detector + learned matcher methods, and dotted lines with square markers (

\blacksquare

) denote detector-free methods. SSMB (thick solid lines) is highlighted for clarity. Right: MMA at a 3-pixel threshold. first , second , and third best results are highlighted within sparse methods only. SSMB with HardNet achieves the highest MMA among all sparse methods, and at this strict threshold also surpasses every detector-free method.

Fig. 9: Qualitative matching results on the ArchViz dataset [ 44 ] under the blur-to-blur setting. Each subfigure displays the number of correct matches and precision in the top-left corner. SSMB, when paired with HardNet [ 53 ] or HyNet [ 54 ] , achieves more correct matches and fewer mismatches, effectively handling motion blur and viewpoint changes. Green lines indicate correct matches (epipolar error below

5\times 10^{-4}

) and red lines indicate incorrect matches. Best viewed in color.

Fig. 10: Qualitative detection results on real-world blurred images from the RealBlur dataset [ 28 ] . SSMB detects well-localized and consistently distributed keypoints under real camera motion blur, while several baselines either miss salient structure or produce noisy responses in heavily blurred regions. The red and blue boxes mark two representative regions, shown enlarged below each image for closer comparison. Best viewed in color.

Fig. 11: Qualitative matching results on real-world sharp-blur image pairs from the RealBlur dataset [ 28 ] . Each row shows a matched sharp-blur pair (left: sharp; right: blurred) with real camera motion blur and viewpoint changes. SSMB detects well-localized and repeatable keypoints across diverse indoor and outdoor scenes despite the cross-domain appearance gap. Green lines indicate correct matches. Best viewed in color.

SSMB: Self-Supervised Local Feature Detection under Motion Blur

Abstract

Figures & tables

Explore similar work

Subtle Motion Blur Detection and Segmentation from Static Image Artworks

BOCCHI: A More Realistic and Challenging Benchmark for Local Motion Blur Detection with MSDCT-UNet

Misanthrope: A Privacy-Preserving Keypoint Detector