Benefiting from high temporal resolution and dynamic range, event-based local feature methods have attracted increasing attention. However, event sparsity, noise, and limited texture still hinder robust local feature learning. Deploying such methods on resource-constrained platforms such as unmanned aerial vehicles also requires balancing accuracy and energy efficiency. To address these challenges, this paper proposes E-S2Feat, a spiking neural network framework for event-based local feature detection and description. The framework jointly optimizes local feature learning from the perspectives of feature representation and selection. First, a module-specific spiking activation mechanism preserves fine-grained structural cues and discriminative information under low-bit, energy-efficient inference, thereby improving overall representation fidelity. Furthermore, a semantic-guided feature modulation mechanism leverages semantic priors to refine keypoint response distributions and enhance local descriptor discriminability, thereby guiding the model to extract local features with greater geometric stability and stronger discriminative capability. Experiments on the ECD and EDS datasets show that the proposed method significantly outperforms baseline methods such as SuperEvent in pose estimation accuracy. It also achieves accuracy comparable to its artificial neural network counterpart while delivering an approximately 4.8-fold improvement in theoretical computational energy efficiency. Visual-inertial odometry experiments on the TUM-VIE dataset further verify the effectiveness and practical application potential of the proposed method in complete SLAM systems.
Figures & tables
Fig. 1: Overview of the proposed framework. The network is built upon spiking neural networks and incorporates a module-specific spiking activation mechanism, achieving low-bit, energy-efficient inference while preserving feature representation fidelity. Event frames are encoded by a shared spiking backbone and routed to the keypoint, descriptor, and semantic branches. The semantic branch jointly modulates keypoint responses and descriptor features to extract geometrically stable and discriminative local features. It is supervised by a pre-trained semantic model during training, which is unnecessary at inference time.
Fig. 2: Overall architecture of E-S2Feat. The spiking MaxViT and FPN extract multi-scale features, while three task branches generate keypoint heatmaps, local descriptors, and a geometric reliability mask. The mask guides feature detection and description through score modulation and feature fusion.
Methods
ECD: AUC [%]
EDS: AUC [%]
@5 ∘ ↑
@10 ∘ ↑
@20 ∘ ↑
@5 ∘ ↑
@10 ∘ ↑
@20 ∘ ↑
RATE [ 25 ]
3.3
8.4
18.0
2.1
5.1
10.3
LLAK [ 28 ]
0.7
1.4
2.1
0.5
0.7
10.3
SD2Event [ 50 ]
16.6
25.3
36.5
11.5
22.1
35.7
STPNet [ 51 ]
19.1
31.3
42.9
13.9
24.9
38.2
SuperEvent [ 18 ]
22.8
35.8
46.7
25.4
37.5
49.0
TABLE I: Pose Estimation Results on ECD [ 48 ] and EDS [ 49 ] datasets.
Methods
ECD: AUC [%]
EDS: AUC [%]
@5 ∘ ↑
@10 ∘ ↑
@20 ∘ ↑
@5 ∘ ↑
@10 ∘ ↑
@20 ∘ ↑
SuperEvent [ 18 ]
22.8
35.8
46.7
25.4
37.5
49.0
+ refined training data
21.2
38.9
55.0
33.2
47.8
60.8
+ Semantic
23.0
42.4
59.6
34.8
50.9
64.6
+ Spiking ( Ours SNNs )
22.8
42.1
59.4
34.1
50.2
64.0
TABLE II: Ablation Experiment Results on ECD [ 48 ] and EDS [ 49 ] datasets.
Fig. 3: Visualization of the semantic segmentation output produced by Ours (SNNs). The three columns show the event frame, the grayscale frame, and the prediction from the SNNs semantic branch, respectively. Blue indicates geometrically unreliable regions, such as the sky, while green indicates valid background regions retained for local feature extraction.
Fig. 4: Qualitative comparison of feature detection and matching between the proposed method and the SuperEvent baseline. Yellow boxes highlight differences in keypoint detection, while green boxes show the improvements in matching density and spatial consistency brought by semantic fusion.
Method
Modality
1d-trans
3d-trans
6dof
desk
desk2
Average
DEIO [ 60 ]
Mono E + IMU
1.08
1.12
1.39
1.41
1.19
1.24
ESVO [ 61 ]
Stereo E
12.54
17.19
13.46
12.92
4.42
12.11
ES-PTAM [ 62 ]
Stereo E
1.05
8.53
10.25
2.50
7.20
5.91
ICRA’24 [ 63 ]
Stereo E + IMU
3.85
18.90
failed
8.99
9.47
–
ESVO2 [ 64 ]
Stereo E + IMU
3.33
7.26
3.21
6.16
4.02
4.78
OKVIS2 [ 58 ] + SuperEvent [ 18 ] (our impl.)
Stereo E + IMU
0.49
1.88
2.25
2.07
1.87
1.71
TABLE III: ATE [cm] on TUM-VIE mocap sequences [ 59 ] .
Fig. 5: Trajectory comparison among our method (orange solid lines), the baseline method (blue solid lines), and the ground truth (gray dashed lines) on the TUM-VIE sequences [ 59 ] .
National University of Singapore, Singapore · CerCo, CNRS UMR 5549, Universit´e de Toulouse, France · ETIS UMR8051, CY Cergy Paris Universit´e, ENSEA, CNRS, Cergy, France +1