cs.NEAug 27, 2026

ANTShapes Benchmarking Datasets for Event-Based Neuromorphic Object Classification

Authors: M. Middleton, H. Kayan, B. Sen Bhattacharya, T. Ali, E. Baikas, M. Vousden, C. Perera, O. Rhodes, 

Organizations: University of York, UK · Cardiff University, UK · University of Manchester, UK · University of Stirling, UK · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Object classification in event-based computer vision is a task that is attracting considerable research attention. Event-based object classification is a fundamental task in the fields of security and applied computer vision, which typically use synchronous frame-based cameras and computing pipelines for operation. This approach has several practical flaws. The size, weight and power consumption of the device could prohibit deployment at the extreme edge or in covert sensing environments. Besides this, there are security concerns inherent in cloud-based or other off-device computation approaches due to the requirement of sending and receiving potentially sensitive data. Furthermore, this transmission of data introduces latency and requires consistent connectivity to the cloud infrastructure to function. The use of Spiking Neural Networks (SNNs) hosted on neuromorphic devices attempts to solve several issues present in this conventional approach. Research into event-based object classification methods are hindered by the lack of high-quality vision datasets to use. To this end, the ANTShapes simulation tool has been previously proposed to create and label event-based vision datasets. In this paper, four novel datasets of varying difficulties are created using the tool and are benchmarked against existing spiking datasets commonly used for event-based vision research (N-MNIST, CIFAR10-DVS, DVSGesture and POKER-DVS). Classification is performed using a convolutional SNN. This work simultaneously provides four datasets with rich details for future experiments to use and validates the output of the ANTShapes dataset simulation tool as being suitable for its purpose.

Figures & tables

Figure 1: Examples of the 0, 1, 2 and 3 classes taken from the N-MNIST dataset. A denoising process (explained later in Section 4.1 is applied before events are accumulated per-pixel for display. Pixels with more "on" than "off" events are coloured red whilst pixels of the opposite case are coloured blue.
Figure 2: Examples of the "airplane", "cat", "truck" and "horse" classes taken from the CIFAR10-DVS dataset.
Figure 3: Examples of the four classes of poker card symbols taken from the POKER-DVS dataset. The resolution of 35×3535\times 35 pixels shown here is used in practice for POKER-DVS rather than the published value of 32×3232\times 32 . Some Python implementations of the dataset, such as the Tonic package implementation, use a resolution of 35×3535\times 35 pixels.
Figure 4: Examples of the "hand clapping", "right arm CW", "left arm CW" and "arm roll" classes taken from the DVSGesture dataset. The right and left arm gestures represent moving the relevant arm in a circular motion clockwise.
DatasetPurposeSpatial ResolutionNum. ClassesType of FeaturesContains DVS Noise?
N-MNISTHandwritten digit classification34×3434\times 3410Saccadic eye mimicYes
CIFAR10-DVSImage classification128×128128\times 12810Saccadic eye mimicYes
POKER-DVSObject detail classification32×3232\times 324Natural sceneYes
DVSGestureObject detail classification128×128128\times 12810 + 1 “other”Natural sceneYes
Table 1: Overview of the four existing datasets discussed here. Only one dataset, CIFAR10-DVS, is specifically targeted towards the classification of an entire object rather than just details expressed on or by an object (such as suit printed on a card or gesture expressed by a person’s body).
ParameterSettingDescription
Initial Rotation σ\sigma , (x,y,z)(x,y,z)180Randomises the initial rotation position of the object around the (x,y,z)(x,y,z) axes.
Initial Position σ\sigma (x,y)(x,y)0.15Displaces the shape from the center of the scene on the horizontal and vertical axes. This provides variance in the dataset by randomising object positions within scenes over each example.
Scale μ\mu (x,y,z)(x,y,z)4A constant scaling value is applied to all shapes to enlarge them in the scene. Scaling is applied uniformly across the (x,y,z)(x,y,z) axes.
Surface Noise μ\mu0.2Applies normal-mapped noise to the surfaces of objects. This applies visual texture to shapes which is dependent on the incident angle of the scene light source.
Anomaly LabellingDisabledAll anomaly labelling supported by ANTShapes is disabled as it is not relevant to the object classification task.
Table 2: Configuration used to generate the standard ANTShapes dataset in the simulation program. These configurations are applied globally to all classes of objects. ANTShapes parameters are defined as normal distributions and accept values for the mean μ\mu and variance σ\sigma . Per-shape parameters are sampled from these defined distributions, causing each example rendered for the dataset to be unique.
Figure 5: Examples of four classes taken from the "standard" ANTShapes dataset. As in Figures 1 to 4 , the events pictured are accumulated over the simulation period leading to the "smearing" effect seen. Variance in the positioning of the objects is seen between examples, as each appears in a different location in the scene. The simulated data is also free from background noise.
Figure 6: Examples from the ANTShapes "translation" dataset. Compared to the standard dataset shown in Figure 5 , it can be seen that the shapes move with different trajectories and rebound from the edges of the simulated scene whilst rotating.
Figure 7: Four classes taken from the "distortion" dataset. These objects each have 3 randomly-assigned scaling vectors which are each applied to the XX , YY and ZZ dimensions to distort the object’s event signature.
Figure 8: Four examples from the "rotation" dataset. Unlike the standard dataset, where each object is assigned a random initial rotation then continues to rotate around the XX , YY and ZZ axes uniformly, each simulated object is assigned a random 3D rotation vector. This introduces additional variance between event signatures within classes by introducing independent rotational behaviour for all shapes between examples.
Figure 9: A diagram of the spiking convolutional network design used for the following benchmarking experiments. Here, TT is the total number of time steps in the dataset where the time between frames is 1ms. XX and YY are the sensor dimensions in pixels; for ANTShapes datasets, this is the spatial resolution of the simulated scene. CC is the number of classes in the dataset being evaluated and BB is the batch size. The network architecture pools the features from previous layers, learning the most relevant features through surrogate gradient training. The readout head is trained on the flattened features from the output of the final hidden layer using the cross-entropy loss function. The stage 2 and 3 layers are optional and are only created if the length of the shortest spatial dimension PnP_{n} is greater than 4 pixels.
 N-MNISTCIFAR10-DVSPOKER-DVSDVSGestureANTShapes (Standard)ANTShapes (Translation)ANTShapes (Distortion)ANTShapes (Rotation)
Mean Accuracy97%55.6%96.6%88.9%92.2%89.1%83.5%89.8%
Variance2.8×10−32.8\times 10^{-3}5.3×10−35.3\times 10^{-3}11.0×10−311.0\times 10^{-3}12.0×10−312.0\times 10^{-3}12.0×10−312.0\times 10^{-3}8.6×10−38.6\times 10^{-3}16.0×10−316.0\times 10^{-3}7.8×10−37.8\times 10^{-3}
Table 3: Summary of the mean accuracy and variance for all datasets. 10 convolutional SNNs, each with unique random initial weights, were trained. The mean accuracy and variance of the ensembles are shown.
Figure 10: Confusion matrices produced for each dataset, illustrating the class confusions predicted by the convolutional SNN. The mean over all 10 trials is plotted here to give an impression of the average confusion matrix produced by the network for a given dataset.
Figure 11: PCA plots created for each of the datasets evaluated in this paper. The DVSGesture plots show clear separation, but also class overlap between clockwise (CW) and counter-clockwise (CCW) motion, which leads to difficulties in separating the classes. POKER-DVS shows clear separation, but only contains 4 classes and a limited number of samples. CIFAR10-DVS shows extreme overlap of principal elements for all classes, demonstrating that the principal components for each class are largely similar and are difficult to separate. The standard ANTShapes dataset has clear separation between most classes, with some overlap between the L-block and T-block objects, and between the various smooth-faced objects (such as cones and capsules). The separation between principal component clusters becomes less well-defined when random rotation, translation and scaling (distortion) vectors are introduced to the objects presented in each example. For all ANTShapes datasets, the sphere objects remain the most cleanly separated from the rest of the classes. This is due to their near-uniform appearance when rotated, with events only being generated from the motion of simulated roughness on the sphere’s surface. Their event signature is therefore distinct from the other objects presented, which can appear different depending on their angle of rotation.
More in this topic

Event-Based Vision

Event-based vision uses asynchronous camera measurements to perceive motion and scene changes with high temporal resolution and dynamic range.

See all 232 papers

Explore similar work

CardsList
Feb 26, 2026

Modelling and Simulation of Neuromorphic Datasets for Anomaly Detection in Computer Vision

Limitations on the availability of Dynamic Vision Sensors (DVS) present a fundamental challenge to researchers of neuromorphic computer vision applications. In response, datasets have been created by the research community, but often contain a limited number of samples or scenarios. To address the lack of a comprehensive simulator of neuromorphic vision datasets, we introduce the Anomalous Neuromorphic Tool for Shapes (ANTShapes), a novel dataset simulation framework. Built in the Unity engine, ANTShapes simulates abstract, configurable 3D scenes populated by objects displaying randomly-generated behaviours describing attributes such as motion and rotation. The sampling of object behaviours, and the labelling of anomalously-acting objects, is a statistical process following central limit theorem principles. Datasets containing an arbitrary number of samples can be created and exported from ANTShapes, along with accompanying label and frame data, through the adjustment of a limited number of parameters within the software. ANTShapes addresses the limitations of data availability to researchers of event-based computer vision by allowing for the simulation of bespoke datasets to suit purposes including object recognition and localisation alongside anomaly detection.

Mike Middleton, Teymoor Ali, Hakan Kayan +6

Bio-Inspired Systems and Technologies Lab, University of York, York, UK · Faculty of Natural Sciences, University of Stirling, Stirling, UK · School of Computer Science and Informatics, Cardiff University, Cardiff, UK +2

Synthetic Anomaly GenerationEvent-Based VisionNeuromorphic ComputingAnomaly Detection
AskPDF
Apr 30, 2026

Real-Time Frame- and Event-based Object Detection with Spiking Neural Networks on Edge Neuromorphic Hardware: Design, Deployment and Benchmark

Real-time object detection on energy-constrained platforms is critical for applications such as UAV-based inspection, autonomous navigation, and mobile robotics. Spiking neural networks (SNNs) on neuromorphic hardware are believed to be significantly more energy-efficient than conventional artificial neural networks (ANNs). In this work, we present a comprehensive methodology for designing general SNN detection architectures targeting neuromorphic platforms, along with the engineering adaptations required to deploy them on the state-of-the-art Neuromorphic processor, Intel Loihi 2. We benchmark SNN-based object detection on Loihi 2 using both frame-based and event-based datasets, comparing performance with ANN-based detection on the NVIDIA Jetson Orin Nano, NVIDIA Jetson Nano B01, and the Apple M2 CPU. Our results show that SNNs on Loihi 2 can perform real-time detection while achieving the lowest per-inference dynamic energy among all platforms. Also, Loihi 2 outperforms the other platforms in terms of power consumption, though ANNs on Jetson Orin Nano achieve higher inference rates. Furthermore, our ANN-to-SNN distillation-aware training enables SNNs to recover 87-100% of the detection accuracy of their ANN counterparts while maintaining lower inference latency; without distillation, SNNs exhibit an 11-27% accuracy drop. These results highlight the potential of neuromorphic systems for energy-efficient, real-time object detection at the edge.

Udayanga G. W. K. N. Gamage, Yan Zeng, Cesar Cadena +2

Department of Electrical and Photonics Engineering, Technical University of Denmark, Anker Engelunds Vej 1, 2800 Kongens Lyngby, Denmark · Faculty of Engineering, University of Ruhuna, Hapugala,Galle 80000, Sri Lanka · Autonomous Systems Lab, ETH zürich, Leonhardstrasse 21, 8092 Zürich, Switzerland

Edge InferenceSpiking Neural NetworksNeuromorphic ComputingEvent-Based Object DetectionReal-Time Object Detection
AskPDF
Mar 17, 2026

SpikeCLR: Contrastive Self-Supervised Learning for Few-Shot Event-Based Vision using Spiking Neural Networks

Event-based vision sensors provide significant advantages for high-speed perception, including microsecond temporal resolution, high dynamic range, and low power consumption. When combined with Spiking Neural Networks (SNNs), they can be deployed on neuromorphic hardware, enabling energy-efficient applications on embedded systems. However, this potential is severely limited by the scarcity of large-scale labeled datasets required to effectively train such models. In this work, we introduce SpikeCLR, a contrastive self-supervised learning framework that enables SNNs to learn robust visual representations from unlabeled event data. We adapt prior frame-based methods to the spiking domain using surrogate gradient training and introduce a suite of event-specific augmentations that leverage spatial, temporal, and polarity transformations. Through extensive experiments on CIFAR10-DVS, N-Caltech101, N-MNIST, and DVS-Gesture benchmarks, we demonstrate that self-supervised pretraining with subsequent fine-tuning outperforms supervised learning in low-data regimes, achieving consistent gains in few-shot and semi-supervised settings. Our ablation studies reveal that combining spatial and temporal augmentations is critical for learning effective spatio-temporal invariances in event data. We further show that learned representations transfer across datasets, contributing to efforts for powerful event-based models in label-scarce settings.

Maxime Vaillant, Axel Carlier, Lai Xing Ng +2

CNRS, IPAL IRL 2955, Singapore · Université de Toulouse, IRIT, France · Institute for Infocomm Research, A*STAR, Singapore +2

Self-Supervised Visual Representation LearningEvent-Based VisionContrastive LearningSpiking Neural Networks
AskPDF