Unsupervised spiking feature learning for event-based pedestrian crossing detection: approaching supervised accuracy without labelled training data

Henok Teklu; Mustafa Sakhai; Matej Mertik; Maciej Wielgosz

cs.CVSep 15, 2026

Unsupervised spiking feature learning for event-based pedestrian crossing detection: approaching supervised accuracy without labelled training data

Authors: Henok Teklu, Mustafa Sakhai, Matej Mertik, Maciej Wielgosz

Organizations: Department of Applied Artificial Intelligence, Alma Mater Europaea University, Maribor, Slovenia. · AGH University of Science and Technology, Krak´ow, Poland.

Abstract

Event cameras are well suited to pedestrian crossing detection, and spiking neural networks (SNNs) can process their output natively, but current SNN detectors are trained with supervised backpropagation and therefore require costly frame-level crossing labels. We investigate crossing detection with no labels in feature learning and report the first unsupervised results on the recent DVS-PedX pedestrian-crossing benchmark. A single spiking layer trained with winner-take-all spike-timing-dependent plasticity learns a dictionary from unlabelled event-frame patches; frames are encoded by cosine similarity to the learned filters with spatial pooling and read out by a linear classifier, the only supervised component. On the 24,454-frame test split the method attains 90.3% accuracy and an area under the receiver operating characteristic curve (AUROC) of 0.936, compared with 92.0% and 0.943 for a supervised spiking network trained end-to-end on the same frames; under adverse weather the AUROC is 0.913. The result is insensitive to the choice of plasticity rule but depends strongly on the readout protocol: with the classical neuron-assignment readout the same network attains only 0.70 AUROC. On the benchmark's real converted portion, a readout refit lifts performance from chance (0.53) to 0.67 AUROC, within the published supervised range. These findings indicate that the accuracy cost of removing labels from feature learning is small on this benchmark, and that reported weaknesses of unsupervised spiking networks may be attributable to the readout protocol rather than to the learning rule.

Figures & tables

Figure 1: Dictionary learned without labels from DVS-PedX event-frame patches: oriented edges, blobs and silhouette fragments.

Figure 2: Input path on one real test frame (crossing). (a) The $64{\times}64$ event frame. (b) One $9{\times}9$ patch. (c) Poisson spike trains generated from the patch (81 input lines, $T{=}40$ timesteps). (d) WTA response of the trained dictionary to those spikes: a sparse sequence of winner identities, the code from which both STDP updates (during learning) and encodings (at readout) derive.

Method	Feature labels	Acc.	AUROC	macro-F1	F1 (crossing)
Majority class	—	0.806	0.500	0.444	0.000
Assignment readout (best rule)	none	0.652	0.695	0.514	0.425
Patch pipeline (this work)	none	0.903	0.936	0.834	0.726
novelty-modulated variant	none	0.903	0.937	0.834	0.727
Supervised SNN (same split)	all	0.920	0.943	—	—

Table 1: DVS-PedX synthetic test split (

n{=}24{,}454

). Labels are used by the pipeline only in the linear readout.

Figure 3: (a) AUROC of the same unsupervised learner under two readout protocols: neuron assignment (0.70) and the patch pipeline with a linear probe (0.936), with features learned by the same local rule in both cases. (b) Pipeline performance by weather partition.

Configuration	$K$	Acc.	AUROC
whole-frame, assignment readout (best rule)	1600	0.652	0.695
patch dictionary	49	0.876	0.886
patch dictionary	2048	0.903	0.936
novelty-modulated variant	2048	0.903	0.937

Table 2: Configuration ladder on the synthetic test split (identical linear-probe readout except the first row).

Figure 4: Operating characteristics of the unsupervised pipeline on the DVS-PedX synthetic test split, overall and per weather partition. (a) ROC. (b) Precision–recall for the crossing class.

Constraint	Crossing recall
precision $\geq 0.8$	0.674
precision $\geq 0.9$	0.422
false-positive rate $\leq 0.05$	0.716
false-positive rate $\leq 0.10$	0.835

Table 3: Operating points on the synthetic test split (combined weather): crossing recall under fixed precision and false-positive-rate constraints, computed from the readout score.

Configuration	Combined	Clear	Cloudy	Adverse
(A) zero-shot	0.534 / 0.503	0.546 / 0.506	0.510 / 0.501	0.558 / 0.489
(B) readout adaptation	0.669 / 0.601	0.660 / 0.605	0.674 / 0.594	0.685 / 0.609
(C) real-data training	0.662 / 0.596	0.664 / 0.603	0.649 / 0.584	0.698 / 0.603

Table 4: JAAD-DVS test split (

n{=}7{,}409

; 38% crossing), partitioned by the per-video JAAD weather attribute (adverse

=

rain

+

snow,

n{=}633

). AUROC / balanced accuracy.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Group	Parameter	Value
Neuron	membrane leak $\tau_{v}$	20 steps
	fixed threshold / increment / decay	0.8 / 0.05 / 200 pres.
	lateral inhibition	2.0
Encoding	timesteps $T$ / Poisson gain $\lambda$	40 / 0.5
STDP	$\eta_{+}$ / $\eta_{-}$ / $w_{\max}$	0.01 / $10^{-4}$ / 1.0
	trace constants	20 steps

Appendix

Table 5: Configuration.

Spiking Neural Networks

Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.

See all 351 papers

Pedestrian Crossing Intent Classification From Event-Based Vision Using Convolutional Spiking Neural Networks With Temporal Augmentation
Sep 11, 2026Henok Teklu, Mustafa Sakhai, Maciej Wielgosz +1Pedestrian Intention Prediction Event-Based Vision
Applied Artificial Intelligence, Alma Mater Europaea University, Maribor, Slovenia · AGH University of Kraków, Kraków, Poland
Brain-inspired spike-timing plasticity for reliable label-efficient event-camera vision
May 17, 2026Mohamad Yazan Sadoun, Sarah Sharif, Yaser Mike BanadEvent-Based Vision Spiking Neural Networks
School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK, USA
Sequence-SOD: Bio-inspired Sequence-aware Spiking ObjectDetection for Event Cameras
Jul 29, 2026Katharina Bendig, René Schuster, Didier StrickerSpiking Neural Networks Event-Based Object Detection
RPTU – Technical University Kaiserslautern-Landau, Gottlieb-Daimler-Straße 47, Kaiserslautern 67663, Germany · DFKI – German Research Center for Artificial Intelligence, Trippstadter Str. 122, Kaiserslautern 67663, Germany

Sep 11, 2026J/K move · Enter open · S save

Pedestrian Crossing Intent Classification From Event-Based Vision Using Convolutional Spiking Neural Networks With Temporal Augmentation

Henok Teklu, Mustafa Sakhai, Maciej Wielgosz, Matej Mertik

Applied Artificial Intelligence, Alma Mater Europaea University, Maribor, Slovenia · AGH University of Kraków, Kraków, Poland

Pedestrian Intention Prediction Event-Based Vision Spiking Neural Networks

Open paper AskPDF arXiv

Anticipating whether a pedestrian will cross the road is safety-critical for autonomous vehicles, requiring real-time inference under challenging conditions including motion blur, high dynamic range, and class imbalance. Conventional frame-based deep networks process redundant RGB data at fixed frame rates, limiting their temporal resolution and energy efficiency. In this work we present an end-to-end pipeline that (i) converts real-world driving footage from the Joint Attention in Autonomous Driving (JAAD) dataset into synthetic dynamic vision sensor (DVS) event streams using the v2e simulator, (ii) augments training with the CARLA-simulated DVS sequences of the DVS-PedX dataset under both normal and adverse weather conditions, and (iii) trains a novel convolutional spiking neural network (Conv-SNN) with clip-consistent DVS augmentation to classify pedestrian crossing intent as binary: crossing or non-crossing. We detail all architectural decisions, the exact leaky-integrate-and-fire neuron dynamics with surrogate-gradient learning, the class-balanced loss formulation, JAAD oversampling at 6x, and a 70/15/15 stratified splitting protocol. The trained model achieves 95.83% accuracy and F1 = 0.9695 on the JAAD DVS test set, 97.79% accuracy and F1 = 0.9478 on normal CARLA DVS, and 94.78% accuracy and F1 = 0.8369 on adverse-weather CARLA DVS, all from a 1.07M-parameter architecture trained on CPU. Compared to prior frame-based approaches on JAAD, our method closes or surpasses the reported accuracy while operating natively on sparse temporal representations. We include a thorough analysis of the convergence behaviour across all 15 training epochs, domain transfer characteristics, and a quantitative comparison with representative related work.

Abstract

Spiking Neural Networks

Explore similar work

Pedestrian Crossing Intent Classification From Event-Based Vision Using Convolutional Spiking Neural Networks With Temporal Augmentation

Brain-inspired spike-timing plasticity for reliable label-efficient event-camera vision

Sequence-SOD: Bio-inspired Sequence-aware Spiking ObjectDetection for Event Cameras