cs.CVAug 14, 2026

E-S2Feat:Semantic-Guided Spiking Local Feature Detection and Description for Event Cameras

Authors: Yang Yi, Juntao Hua, Jinpu Zhang, Liangwei Fan, Hui Shen, Dewen Hu

Organizations: College of Intelligence Science and Technology, National University of Defense Technology, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Benefiting from high temporal resolution and dynamic range, event-based local feature methods have attracted increasing attention. However, event sparsity, noise, and limited texture still hinder robust local feature learning. Deploying such methods on resource-constrained platforms such as unmanned aerial vehicles also requires balancing accuracy and energy efficiency. To address these challenges, this paper proposes E-S2Feat, a spiking neural network framework for event-based local feature detection and description. The framework jointly optimizes local feature learning from the perspectives of feature representation and selection. First, a module-specific spiking activation mechanism preserves fine-grained structural cues and discriminative information under low-bit, energy-efficient inference, thereby improving overall representation fidelity. Furthermore, a semantic-guided feature modulation mechanism leverages semantic priors to refine keypoint response distributions and enhance local descriptor discriminability, thereby guiding the model to extract local features with greater geometric stability and stronger discriminative capability. Experiments on the ECD and EDS datasets show that the proposed method significantly outperforms baseline methods such as SuperEvent in pose estimation accuracy. It also achieves accuracy comparable to its artificial neural network counterpart while delivering an approximately 4.8-fold improvement in theoretical computational energy efficiency. Visual-inertial odometry experiments on the TUM-VIE dataset further verify the effectiveness and practical application potential of the proposed method in complete SLAM systems.

Figures & tables

Fig. 1: Overview of the proposed framework. The network is built upon spiking neural networks and incorporates a module-specific spiking activation mechanism, achieving low-bit, energy-efficient inference while preserving feature representation fidelity. Event frames are encoded by a shared spiking backbone and routed to the keypoint, descriptor, and semantic branches. The semantic branch jointly modulates keypoint responses and descriptor features to extract geometrically stable and discriminative local features. It is supervised by a pre-trained semantic model during training, which is unnecessary at inference time.
Fig. 2: Overall architecture of E-S2Feat. The spiking MaxViT and FPN extract multi-scale features, while three task branches generate keypoint heatmaps, local descriptors, and a geometric reliability mask. The mask guides feature detection and description through score modulation and feature fusion.
MethodsECD: AUC [%]EDS: AUC [%]
@5 ∘ ↑\uparrow@10 ∘ ↑\uparrow@20 ∘ ↑\uparrow@5 ∘ ↑\uparrow@10 ∘ ↑\uparrow@20 ∘ ↑\uparrow
RATE [ 25 ]3.38.418.02.15.110.3
LLAK [ 28 ]0.71.42.10.50.710.3
SD2Event [ 50 ]16.625.336.511.522.135.7
STPNet [ 51 ]19.131.342.913.924.938.2
SuperEvent [ 18 ]22.835.846.725.437.549.0
TABLE I: Pose Estimation Results on ECD [ 48 ] and EDS [ 49 ] datasets.
MethodsECD: AUC [%]EDS: AUC [%]
@5 ∘ ↑\uparrow@10 ∘ ↑\uparrow@20 ∘ ↑\uparrow@5 ∘ ↑\uparrow@10 ∘ ↑\uparrow@20 ∘ ↑\uparrow
SuperEvent [ 18 ]22.835.846.725.437.549.0
+ refined training data21.238.955.033.247.860.8
+ Semantic23.042.459.634.850.964.6
+ Spiking ( Ours SNNs )22.842.159.434.150.264.0
TABLE II: Ablation Experiment Results on ECD [ 48 ] and EDS [ 49 ] datasets.
Fig. 3: Visualization of the semantic segmentation output produced by Ours (SNNs). The three columns show the event frame, the grayscale frame, and the prediction from the SNNs semantic branch, respectively. Blue indicates geometrically unreliable regions, such as the sky, while green indicates valid background regions retained for local feature extraction.
Fig. 4: Qualitative comparison of feature detection and matching between the proposed method and the SuperEvent baseline. Yellow boxes highlight differences in keypoint detection, while green boxes show the improvements in matching density and spatial consistency brought by semantic fusion.
MethodModality1d-trans3d-trans6dofdeskdesk2Average
DEIO [ 60 ]Mono E + IMU1.081.121.391.411.191.24
ESVO [ 61 ]Stereo E12.5417.1913.4612.924.4212.11
ES-PTAM [ 62 ]Stereo E1.058.5310.252.507.205.91
ICRA’24 [ 63 ]Stereo E + IMU3.8518.90failed8.999.47–
ESVO2 [ 64 ]Stereo E + IMU3.337.263.216.164.024.78
OKVIS2 [ 58 ] + SuperEvent [ 18 ] (our impl.)Stereo E + IMU0.491.882.252.071.871.71
TABLE III: ATE [cm] on TUM-VIE mocap sequences [ 59 ] .
Fig. 5: Trajectory comparison among our method (orange solid lines), the baseline method (blue solid lines), and the ground truth (gray dashed lines) on the TUM-VIE sequences [ 59 ] .
Methods#Params (M)FPSPower (mJ)
SuperEvent (ANNs)1.541107.50
Ours (ANNs)2.399312.02
Ours (SNNs)2.39852.52
TABLE IV: Model Efficiency Comparison.
More in this topic

Event-Based Vision

Event-based vision uses asynchronous camera measurements to perceive motion and scene changes with high temporal resolution and dynamic range.

See all 232 papers

Explore similar work

CardsList
Mar 6, 2026

EventGeM: Global-to-Local Feature Matching for Event-Based Visual Place Recognition

Event cameras are rapidly rising in popularity for robotic and computer vision tasks because their sparse activation delivers energy-efficient, high-dynamic-range, and fast sensing. Event cameras have been used in robotic navigation and localization tasks where positioning must occur in real time with sufficient accuracy. However, current event-based localization methods suffer from poor spatial understanding and are not viewpoint tolerant. In this paper, we address the problem of viewpoint-robust place recognition directly from event streams. We present EventGeM, a global-to-local feature fusion pipeline for event-based visual place recognition that combines whole-image feature detection to shortlist top candidates for 2D homography-based re-ranking with random sample consensus (RANSAC). We also contribute a regional generalized mean pooling (GeM) layer that learns to return the most relevant spatial features using per-row exponents to pool event streams into a compact global descriptor, trained on the NYC-Event-VPR dataset. These contributions overcome shortfalls in currently available event-based localization methods that fail to recognize similar places with large changes in viewpoint. To evaluate viewpoint-robust localization, we contribute a new event-based dataset that includes repeated traverses with a severe lateral shift. EventGeM improves absolute Recall@1 by 7 to 43 percentage points over the strongest baseline in each experiment. We also deploy EventGeM on a robotic platform, demonstrating real-time performance of our hierarchical pipeline. The code for EventGeM is available at https://github.com/AdamDHines/Event-GeM.

Adam D. Hines, Gokul B. Nair, Nicolás Marticorena +2

QUT Centre for Robotics, School of Electrical Engineering and Robotics, Queensland University of Technology, Brisbane, QLD 4000, Australia.

Visual Place RecognitionEvent-Based Vision
AskPDF
Mar 29, 2026

E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences

Event-based cameras capture visual information as asynchronous streams of per-pixel brightness changes, generating sparse, temporally precise data. Compared to conventional frame-based sensors, they offer significant advantages in capturing high-speed dynamics while consuming substantially less power. Predicting future event representations from past observations is an important problem, enabling downstream tasks such as future semantic segmentation or object tracking without requiring access to future sensor measurements. While recent state-of-the-art approaches achieve strong performance, they often rely on computationally heavy backbones and, in some cases, large-scale pretraining, limiting their applicability in resource-constrained scenarios. In this work, we introduce E-TIDE, a lightweight, end-to-end trainable architecture for event-tensor prediction that is designed to operate efficiently without large-scale pretraining. Our approach employs the TIDE module (Temporal Interaction for Dynamic Events), motivated by efficient spatiotemporal interaction design for sparse event tensors, to capture temporal dependencies via large-kernel mixing and activity-aware gating while maintaining low computational complexity. Experiments on standard event-based datasets demonstrate that our method achieves competitive performance with significantly reduced model size and training requirements, making it well-suited for real-time deployment under tight latency and memory budgets.

Biswadeep Sen, Benoit R. Cottereau, Nicolas Cuperlier +1

National University of Singapore, Singapore · CerCo, CNRS UMR 5549, Universit´e de Toulouse, France · ETIS UMR8051, CY Cergy Paris Universit´e, ENSEA, CNRS, Cergy, France +1

Event-Based VisionEfficient Neural Network InferenceSpatiotemporal Forecasting
AskPDF
Oct 30, 2025

Spiking Patches: Asynchronous, Sparse, and Efficient Tokens for Event Cameras

We propose tokenization of events and present a tokenizer, Spiking Patches, specifically designed for event cameras. Given a stream of asynchronous and spatially sparse events, our goal is to discover an event representation that preserves these properties. Prior works have represented events as frames or as voxels. However, while these representations yield high accuracy, both frames and voxels are synchronous and decrease the spatial sparsity. Spiking Patches gives the means to preserve the unique properties of event cameras and we show in our experiments that this comes without sacrificing accuracy. We evaluate our tokenizer using a GNN, PCN, and a Transformer on gesture recognition and object detection. Tokens from Spiking Patches yield inference times that are up to 3.4x faster than voxel-based tokens and up to 10.4x faster than frames. We achieve this while matching their accuracy and even surpassing in some cases with absolute improvements up to 3.8 for gesture recognition and up to 1.4 for object detection. Thus, tokenization constitutes a novel direction in event-based vision and marks a step towards methods that preserve the properties of event cameras.

Christoffer Koo Øhrstrøm, Ronja Güldenring, Lazaros Nalpantidis

Technical University of Denmark

Event-Based VisionVisual Tokenization
AskPDF