Event-Based Vision

Ask about this topic

Overview

Event-based vision studies cameras that report changes in brightness asynchronously rather than capturing fixed-rate images, making them useful for fast motion, low light, and high-dynamic-range scenes. Methods convert events into time surfaces, voxel grids, or continuous-time features, then apply neural networks, spiking models, or geometric estimation for tasks such as detection, optical flow, depth, and visual odometry. Work also combines events with RGB, inertial, or LiDAR data and evaluates systems on benchmarks such as DSEC, MVSEC, Gen1, and DVS Gesture. A central challenge is making results robust to sensor noise and hardware differences while balancing accuracy, latency, energy use, and performance against frame-based methods in real-world settings.

Latest papers

CardsList

232 papers20 papers in the last four weeks, down 26% on the four weeks before. 0.1% of all new papers.

Feb 28, 2026

High Dynamic Range Imaging Based on an Asymmetric Event-SVE Camera System

High dynamic range (HDR) imaging under extreme illumination remains challenging for conventional cameras due to overexposure. Event cameras provide microsecond temporal resolution and high dynamic range, while spatially varying exposure (SVE) sensors offer single-shot radiometric diversity.We present a hardware--algorithm co-designed HDR imaging system that tightly integrates an SVE micro-attenuation camera with an event sensor in an asymmetric dual-modality configuration. To handle non-coaxial geometry and heterogeneous optics, we develop a two-stage cross-modal alignment framework that combines feature-guided coarse homography estimation with a multi-scale refinement module based on spatial pooling and frequency-domain filtering. On top of aligned representations, we develop a cross-modal HDR reconstruction network with convolutional fusion, mutual-information regularization, and a learnable fusion loss that adaptively balances intensity cues and event-derived structural constraints. Comprehensive experiments on both synthetic benchmarks and real captures demonstrate that the proposed system consistently improves highlight recovery, edge fidelity, and robustness compared with frame-only or event-only HDR pipelines. The results indicate that jointly optimizing optical design, cross-modal alignment, and computational fusion provides an effective foundation for reliable HDR perception in highly dynamic and radiometrically challenging environments.

Pengju Sun, Banglei Guan, Jing Tao +4

College of Aerospace Science and Engineering, National University of Defense Technology, Hunan 410073, China · Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation, Hunan 410073, China

HDR ImagingMulti-Source Image FusionEvent-Based VisionCross-Modal Image Registration
AskPDF
Feb 26, 2026

Modelling and Simulation of Neuromorphic Datasets for Anomaly Detection in Computer Vision

Limitations on the availability of Dynamic Vision Sensors (DVS) present a fundamental challenge to researchers of neuromorphic computer vision applications. In response, datasets have been created by the research community, but often contain a limited number of samples or scenarios. To address the lack of a comprehensive simulator of neuromorphic vision datasets, we introduce the Anomalous Neuromorphic Tool for Shapes (ANTShapes), a novel dataset simulation framework. Built in the Unity engine, ANTShapes simulates abstract, configurable 3D scenes populated by objects displaying randomly-generated behaviours describing attributes such as motion and rotation. The sampling of object behaviours, and the labelling of anomalously-acting objects, is a statistical process following central limit theorem principles. Datasets containing an arbitrary number of samples can be created and exported from ANTShapes, along with accompanying label and frame data, through the adjustment of a limited number of parameters within the software. ANTShapes addresses the limitations of data availability to researchers of event-based computer vision by allowing for the simulation of bespoke datasets to suit purposes including object recognition and localisation alongside anomaly detection.

Mike Middleton, Teymoor Ali, Hakan Kayan +6

Bio-Inspired Systems and Technologies Lab, University of York, York, UK · Faculty of Natural Sciences, University of Stirling, Stirling, UK · School of Computer Science and Informatics, Cardiff University, Cardiff, UK +2

Synthetic Anomaly GenerationEvent-Based VisionNeuromorphic ComputingAnomaly Detection
AskPDF
Feb 26, 2026

Sensor Generalization for Adaptive Sensing in Event-based Object Detection via Joint Distribution Training

Bio-inspired event cameras have recently attracted significant research due to their asynchronous and low-latency capabilities. These features provide a high dynamic range and significantly reduce motion blur. However, because of the novelty in the nature of their output signals, there is a gap in the variability of available data and a lack of extensive analysis of the parameters characterizing their signals. This paper addresses these issues by providing readers with an in-depth understanding of how intrinsic parameters affect the performance of a model trained on event data, specifically for object detection. We also use our findings to expand the capabilities of the downstream model towards sensor-agnostic robustness.

Aheli Saha, René Schuster, Didier Stricker

Deutsches Forschungszentrum für Künstliche Intelligenz, Kaiserslautern, Germany

Adaptive SensingEvent-Based VisionEvent-Based Object DetectionDomain Generalization
AskPDF
Feb 26, 2026

Motion-aware Event Suppression for Event Cameras

Event cameras report asynchronously per-pixel brightness changes with microsecond latency, encoding dynamic visual information as a sparse stream of events. However, their extreme temporal resolution floods perception systems with entangled events from ego-motion and independently moving objects (IMOs), which existing solutions fail to efficiently decouple, relying instead on prohibitive dense 3D reconstructions or limited hand-tuned filters. In this work, we introduce the first framework for Motion-aware Event Suppression, which learns to filter events triggered by IMOs and ego-motion in real time. Our model jointly segments IMOs in the current event stream while predicting their future motion, enabling anticipatory suppression of dynamic events before they occur. Our lightweight architecture achieves 173 Hz inference on consumer-grade GPUs with less than 1 GB of memory usage, outperforming previous state-of-the-art methods on the challenging EVIMO benchmark by 67% in segmentation accuracy while operating at a 53% higher inference rate. Moreover, we demonstrate significant benefits for downstream applications: our method accelerates Vision Transformer inference by 83% via token pruning and improves event-based visual odometry accuracy, reducing Absolute Trajectory Error (ATE) by 13%.

Roberto Pellerito, Nico Messikommer, Giovanni Cioffi +2

Robotics and Perception Group, University of Zurich, Switzerland

Event-Based Vision
AskPDF
Feb 26, 2026

Locally Adaptive Decay Surfaces for High-Speed Face and Landmark Detection with Event Cameras

Event cameras record luminance changes with microsecond resolution, but converting their sparse, asynchronous output into dense tensors that neural networks can exploit remains a core challenge. Conventional histograms or globally-decayed time-surface representations apply fixed temporal parameters across the entire image plane, which in practice creates a trade-off between preserving spatial structure during still periods and retaining sharp edges during rapid motion. We introduce Locally Adaptive Decay Surfaces (LADS), a family of event representations in which the temporal decay at each location is modulated according to local signal dynamics. Three strategies are explored, based on event rate, Laplacian-of-Gaussian response, and high-frequency spectral energy. These adaptive schemes preserve detail in quiescent regions while reducing blur in regions of dense activity. Extensive experiments on the public data show that LADS consistently improves both face detection and facial landmark accuracy compared to standard non-adaptive representations. At 30 Hz, LADS achieves higher detection accuracy and lower landmark error than either baseline, and at 240 Hz it mitigates the accuracy decline typically observed at higher frequencies, sustaining 2.44 % normalized mean error for landmarks and 0.966 mAP50 in face detection. These high-frequency results even surpass the accuracy reported in prior works operating at 30 Hz, setting new benchmarks for event-based face analysis. Moreover, by preserving spatial structure at the representation stage, LADS supports the use of much lighter network architectures while still retaining real-time performance. These results highlight the importance of context-aware temporal integration for neuromorphic vision and point toward real-time, high-frequency human-computer interaction systems that exploit the unique advantages of event cameras.

Paul Kielty, Timothy Hanley, Peter Corcoran

With the College of Science and Engineering, University of Galway, Ireland.

Event-Based VisionEvent-Based Object Detection
AskPDF
Feb 26, 2026

Coded-E2LF: Coded Aperture Light Field Imaging from Events

We propose Coded-E2LF (coded event to light field), a computational imaging method for acquiring a 4-D light field using a coded aperture and a stationary event-only camera. In a previous work, an imaging system similar to ours was adopted, but both events and intensity images were captured and used for light field reconstruction. In contrast, our method is purely event-based, which relaxes restrictions for hardware implementation. We also introduce several advancements from the previous work that enable us to theoretically support and practically improve light field reconstruction from events alone. In particular, we clarify the key role of a black pattern in aperture coding patterns. We finally implemented our method on real imaging hardware to demonstrate its effectiveness in capturing real 3-D scenes. To the best of our knowledge, we are the first to demonstrate that a 4-D light field with pixel-level accuracy can be reconstructed from events alone. Our software and supplementary video are available from our project website.

Tomoya Tsuchida, Keita Takahashi, Chihiro Tsutake +2

Nagoya University, Japan · Osaka University, Japan

Computational ImagingImage ReconstructionEvent-Based Vision
AskPDF
Feb 25, 2026

RGB-Event HyperGraph Prompt for Kilometer Marker Recognition based on Pre-trained Foundation Models

Metro trains often operate in highly complex environments, characterized by illumination variations, high-speed motion, and adverse weather conditions. These factors pose significant challenges for visual perception systems, especially those relying solely on conventional RGB cameras. To tackle these difficulties, we explore the integration of event cameras into the perception system, leveraging their advantages in low-light conditions, high-speed scenarios, and low power consumption. Specifically, we focus on Kilometer Marker Recognition (KMR), a critical task for autonomous metro localization under GNSS-denied conditions. In this context, we propose a robust baseline method based on a pre-trained RGB OCR foundation model, enhanced through multi-modal adaptation. Furthermore, we construct the first large-scale RGB-Event dataset, EvMetro5K, containing 5,599 pairs of synchronized RGB-Event samples, split into 4,479 training and 1,120 testing samples. Extensive experiments on EvMetro5K and other widely used benchmarks demonstrate the effectiveness of our approach for KMR. Both the dataset and source code will be released on https://github.com/Event-AHU/EvMetro5K_benchmark

Xiaoyu Xian, Shiao Wang, Xiao Wang +2

School of Transportation Science and Engineering, Beihang University, Beijing 100191, China · CRRC Academy Co., Ltd, Beijing, 100070, China · School of Computer Science and Technology, Anhui University, Hefei 230601, China.

RGB-Event Sensor FusionEvent-Based VisionRobot Localization
AskPDF
Feb 24, 2026

Real-time Motion Segmentation with Event-based Normal Flow

Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle visual tasks in challenging scenarios. However, due to the sparse information content in individual events, directly processing the raw event data to solve vision tasks is highly inefficient, which severely limits the applicability of state-of-the-art methods in real-time tasks, such as motion segmentation, a fundamental task for dynamic scene understanding. Incorporating normal flow as an intermediate representation to compress motion information from event clusters within a localized region provides a more effective solution. In this work, we propose a normal flow-based motion segmentation framework for event-based vision. Leveraging the dense normal flow directly learned from event neighborhoods as input, we formulate the motion segmentation task as an energy minimization problem solved via graph cuts, and optimize it iteratively with normal flow clustering and motion model fitting. By using a normal flow-based motion model initialization and fitting method, the proposed system is able to efficiently estimate the motion models of independently moving objects with only a limited number of candidate models, which significantly reduces the computational complexity and ensures real-time performance, achieving nearly a 800x speedup in comparison to the open-source state-of-the-art method. Extensive evaluations on multiple public datasets fully demonstrate the accuracy and efficiency of our framework.

Sheng Zhong, Zhongyang Ren, Xiya Zhu +3

Neuromorphic Automation and Intelligence Lab (NAIL) at School of Artificial Intelligence and Robotics, Hunan University, Changsha, China. · Capital One. · University of Maryland, College Park.

Motion SegmentationEvent-Based Vision
AskPDF
Feb 22, 2026

UniE2F: A Unified Diffusion Framework for Event-to-Frame Reconstruction with Video Foundation Models

Event cameras excel at high-speed, low-power, and high-dynamic-range scene perception. However, as they fundamentally record only relative intensity changes rather than absolute intensity, the resulting data streams suffer from a significant loss of spatial information and static texture details. In this paper, we address this limitation by leveraging the generative prior of a pre-trained video diffusion model to reconstruct high-fidelity video frames from sparse event data. Specifically, we first establish a baseline model by directly applying event data as a condition to synthesize videos. Then, based on the physical correlation between the event stream and video frames, we further introduce the event-based inter-frame residual guidance to enhance the accuracy of video frame reconstruction. Furthermore, we extend our method to video frame interpolation and prediction in a zero-shot manner by modulating the reverse diffusion sampling process, thereby creating a unified event-to-frame reconstruction framework. Experimental results on real-world and synthetic datasets demonstrate that our method outperforms previous reconstruction approaches on most quantitative metrics and achieves superior qualitative results, while achieving competitive zero-shot performance compared with dedicated methods explicitly trained for interpolation and prediction tasks. We also refer the reviewers to the video demo contained in the supplementary material for video results. The code will be publicly available at https://github.com/CS-GangXu/UniE2F.

Gang Xu, Zhiyu Zhu, Junhui Hou

Department of Computer Science, City University of Hong Kong, Hong Kong, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, China · Department of Computer Science, City University of Hong Kong (Dongguan), Dongguan, China

Event-Based VisionVideo PredictionVideo Frame Interpolation
AskPDF
Feb 20, 2026

RoEL: Robust Event-based 3D Line Reconstruction

Event cameras in motion tend to detect object boundaries or texture edges, which produce lines of brightness changes, especially in man-made environments. While lines can constitute a robust intermediate representation that is consistently observed, the sparse nature of lines may lead to drastic deterioration with minor estimation errors. Only a few previous works, often accompanied by additional sensors, utilize lines to compensate for the severe domain discrepancies of event sensors along with unpredictable noise characteristics. We propose a method that can stably extract tracks of varying appearances of lines using a clever algorithmic process that observes multiple representations from various time slices of events, compensating for potential adversaries within the event data. We then propose geometric cost functions that can refine the 3D line maps and camera poses, eliminating projective distortions and depth ambiguities. The 3D line maps are highly compact and can be equipped with our proposed cost function, which can be adapted for any observations that can detect and extract line structures or projections of them, including 3D point cloud maps or image observations. We demonstrate that our formulation is powerful enough to exhibit a significant performance boost in event-based mapping and pose refinement across diverse datasets, and can be flexibly applied to multimodal scenarios. Our results confirm that the proposed line-based formulation is a robust and effective approach for the practical deployment of event-based perceptual modules. Project page: https://gwangtak.github.io/roel/

Gwangtak Bae, Jaeho Shin, Seunggu Kang +3

Dept. of Electrical and Computer Engineering, Seoul National University · University of Michigan · Dept. of Future Automotive Mobility, Seoul National University +1

3D ReconstructionEvent-Based VisionCamera Pose Estimation
AskPDF
Feb 16, 2026

Event-based Visual Deformation Measurement

Visual Deformation Measurement (VDM) aims to recover dense deformation fields by tracking surface motion from camera observations. Traditional image-based methods rely on minimal inter-frame motion to constrain the correspondence search space, which limits their applicability to highly dynamic scenes or necessitates high-speed cameras at the cost of prohibitive storage and computational overhead. We propose an event-frame fusion framework that exploits events for temporally dense motion cues and frames for spatially dense precise estimation. Revisiting the solid elastic modeling prior, we propose an Affine Invariant Simplicial (AIS) framework. It partitions the deformation field into linearized sub-regions with low-parametric representation, effectively mitigating motion ambiguities arising from sparse and noisy events. To speed up parameter searching and reduce error accumulation, a neighborhood-greedy optimization strategy is introduced, enabling well-converged sub-regions to guide their poorly-converged neighbors, effectively suppress local error accumulation in long-term dense tracking. To evaluate the proposed method, a benchmark dataset with temporally aligned event streams and frames is established, encompassing over 120 sequences spanning diverse deformation scenarios. Experimental results show that our method outperforms the state-of-the-art baseline by 1.6% in survival rate. Remarkably, it achieves this using only 18.9% of the data storage and processing resources of high-speed video methods.

Yuliang Wu, Wei Zhai, Yuxin Cui +3

MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China · Fuzhou University

Deformable Image RegistrationEvent-Based Vision
AskPDF
Dec 19, 2025

Event-based Scene Synthesis via Inter-Frame Residual Alignment

Event-based scene synthesis reconstructs target RGB frames from sparse image observations and asynchronous event streams, encompassing both video frame prediction and interpolation. Existing event-based synthesis methods commonly estimate optical flow to warp the observed frames toward the target time, but are vulnerable to inaccurate flow under large motion and occlusion and often rely on flow supervision or pretrained estimators. In this work, we propose EvFRA, an Event-based scene synthesis framework based on inter-Frame Residual Alignment. We identify a structural correspondence between event measurements and frame-to-frame scene changes, and exploit this correspondence for target frame synthesis. Our training pipeline consists of two stages: 1) an Event-to-Residual Alignment Variational Autoencoder (ER-VAE) aligns the event frame captured between the anchor and target frames with the corresponding inter-frame residual, and 2) a ControlNet-conditioned diffusion model is fine-tuned to denoise the residual latent using event data. Our method outperforms state-of-the-art methods by up to 2.61 dB and 1.85 dB in PSNR for frame prediction and interpolation, respectively, with consistent SSIM improvements. Code is available at https://github.com/jiyun-kong/EvFRA.

Jiyun Kong, Jun-Hyuk Kim, Jong-Seok Lee

Yonsei University, Korea · Chung-Ang University, Korea

Event-Based VisionVideo PredictionVideo Frame Interpolation
AskPDF
Nov 22, 2025

Hybrid Event Frame Sensors: Modeling, Calibration, and Simulation

Hybrid event-frame sensors integrate an Event Vision Sensor (EVS) and an Active Pixel Sensor (APS) within a single chip, combining the high dynamic range and low latency of the EVS with the rich spatial intensity information from the APS. While this tight integration offers compact and temporally precise imaging, the complex circuit architecture introduces nontrivial noise patterns that remain poorly understood and unmodeled. In this work, we present the first unified statistics-based imaging noise model that jointly describes the noise behavior of APS and EVS pixels. Our formulation explicitly incorporates photon shot noise, dark current noise, fixed-pattern noise, and quantization noise, and links EVS noise to illumination level and dark current. Based on this formulation, we further develop a calibration pipeline to estimate noise parameters from real data and provide a detailed analysis of both APS and EVS noise behaviors. Finally, we propose H-ESIM, a statistically grounded simulator that generates RAW frames and events under realistic jointly calibrated noise statistics. Experiments on two hybrid sensors validate our model across multiple imaging tasks, including video frame interpolation and deblurring, demonstrating strong transfer from simulation to real data.

Yunfan Lu, Nico Messikommer, Xiaogang Xu +5

1AI Thrust, HKUST(GZ) · RPG, University of Zurich · CUHK +1

Event-Based Vision
AskPDF
Oct 30, 2025

Spiking Patches: Asynchronous, Sparse, and Efficient Tokens for Event Cameras

We propose tokenization of events and present a tokenizer, Spiking Patches, specifically designed for event cameras. Given a stream of asynchronous and spatially sparse events, our goal is to discover an event representation that preserves these properties. Prior works have represented events as frames or as voxels. However, while these representations yield high accuracy, both frames and voxels are synchronous and decrease the spatial sparsity. Spiking Patches gives the means to preserve the unique properties of event cameras and we show in our experiments that this comes without sacrificing accuracy. We evaluate our tokenizer using a GNN, PCN, and a Transformer on gesture recognition and object detection. Tokens from Spiking Patches yield inference times that are up to 3.4x faster than voxel-based tokens and up to 10.4x faster than frames. We achieve this while matching their accuracy and even surpassing in some cases with absolute improvements up to 3.8 for gesture recognition and up to 1.4 for object detection. Thus, tokenization constitutes a novel direction in event-based vision and marks a step towards methods that preserve the properties of event cameras.

Christoffer Koo Øhrstrøm, Ronja Güldenring, Lazaros Nalpantidis

Technical University of Denmark

Event-Based VisionVisual Tokenization
AskPDF
Sep 21, 2025

Event-Based Visual Teach-and-Repeat via Fast Fourier-Domain Cross-Correlation

Visual teach-and-repeat (VT&R) navigation enables robots to autonomously traverse previously demonstrated paths using visual feedback. We present a novel event-camera-based VT&R system. Our system formulates event-stream matching as frequency-domain cross-correlation, transforming spatial convolutions into efficient Fourier-space multiplications. By exploiting the binary structure of event frames and applying image compression techniques, we achieve a processing latency of just 2.88 ms, about 3.5 times faster than conventional camera-based baselines that are optimised for runtime efficiency. Experiments using a Prophesee EVK4 HD event camera mounted on an AgileX Scout Mini robot demonstrate successful autonomous navigation across 3000+ meters of indoor and outdoor trajectories in daytime and nighttime conditions. Our system maintains Cross-Track Errors (XTE) below 15 cm, demonstrating the practical viability of event-based perception for real-time VT&R navigation.

Gokul B. Nair, Alejandro Fontan, Michael Milford +1

The authors are with the QUT Centre for Robotics, Faculty of Engineering, Queensland University of Technology, Brisbane, QLD Australia 4000

Visual NavigationRobot NavigationEvent-Based Vision
AskPDF
Aug 26, 2025

VibES: Induced Vibration for Persistent Event-Based Sensing

Event cameras are a bio-inspired class of sensors that asynchronously measure per-pixel intensity changes. Under fixed illumination conditions in static or low-motion scenes, rigidly mounted event cameras are unable to generate any events and become unsuitable for most computer vision tasks. To address this limitation, recent work has investigated motion-induced event stimulation, which often requires complex hardware or additional optical components. In contrast, we introduce a lightweight approach to sustain persistent event generation by employing a simple rotating unbalanced mass to induce periodic vibrational motion. This is combined with a motion-compensation pipeline that removes the injected motion and yields clean, motion-corrected events for downstream perception tasks. We develop a hardware prototype to demonstrate our approach and evaluate it on real-world datasets. Our method reliably recovers motion parameters and improves both image reconstruction and edge detection compared to event-based sensing without motion induction.

Vincenzo Polizzi, Stephen Yang, Quentin Clark +3

University of Toronto, Robotics Institute · University of Toronto, Department of Mechanical and Industrial Engineering · University of Toronto +1

Event-Based Vision
AskPDF
Apr 25, 2025

BiasBench: A reproducible benchmark for tuning the biases of event cameras

Event-based cameras are bio-inspired sensors that detect light changes asynchronously for each pixel. They are increasingly used in fields like computer vision and robotics because of several advantages over traditional frame-based cameras, such as high temporal resolution, low latency, and high dynamic range. As with any camera, the output's quality depends on how well the camera's settings, called biases for event-based cameras, are configured. While frame-based cameras have advanced automatic configuration algorithms, there are very few such tools for tuning these biases. A systematic testing framework would require observing the same scene with different biases, which is tricky since event cameras only generate events when there is movement. Event simulators exist, but since biases heavily depend on the electrical circuit and the pixel design, available simulators are not well suited for bias tuning. To allow reproducibility, we present BiasBench, a novel event dataset containing multiple scenes with settings sampled in a grid-like pattern. We present three different scenes, each with a quality metric of the downstream application. Additionally, we present a novel, RL-based method to facilitate online bias adjustments.

Andreas Ziegler, David Joseph, Thomas Gossard +2

Cognitive Systems Group, University of Tübingen, Germany

Event-Based VisionBenchmark Design
AskPDF
Jan 30, 2025

On the Theory of Bias Tuning in Event Cameras

This paper lays the foundation of a theory for bias tuning in neuromorphic cameras, a novel sensing technology also known as "event cameras". We begin by formulating the high-level effect of the sensitivity biases on the camera's event rate in mathematical terms. We then show that, as a corollary of the Poincare-Miranda theorem, the commonly used tuning principles of rate budgeting and polarity balancing lead to a unique configuration of the sensitivity biases. As a corollary, we show how by adopting these principles, the multi-variable bias-tuning problem reduces to a two-parameter problem that can be resolved experimentally.

David El-Chai Ben-Ezra, Daniel Brisk, Adar Tal

Event-Based VisionNeuromorphic Computing
AskPDF
Dec 4, 2024

GERD: Geometric event response data generation

Event-based vision sensors offer high temporal resolution, high dynamic range, and low power consumption, yet event-based vision models lag behind conventional frame-based vision methods. We argue that this gap is partly due to the lack of principled study of the transformation groups that govern event-based visual streams. Motivated by the role that geometric and group-theoretic methods have played in advancing computer vision, we present GERD: a simulator for generating event-based recordings of objects under precisely controlled affine, Galilean, and temporal scaling transformations. By providing ground-truth transformations at each timestep, GERD enables hypothesis-driven and controlled studies of geometric properties that are otherwise hard to isolate in real-world datasets or with current event simulators. GERD supports three noise models and sub-pixel motion as a complement to real sensor datasets. We demonstrate its use in training by evaluating models from the literature with geometric guarantees and release GERD as an open tool available at

Jens Egholm Pedersen, Dimitris Korakovounis, Jörg Conradt

DTU Electro, Technical University of Denmark, Copenhagen, Denmark · Department of Computational Science and Technology, KTH Royal Institute of Technology, Stockholm, Sweden

Event-Based VisionSynthetic Data Generation
AskPDF
Jul 5, 2024

Event-based vision sensing and its application to pedestrian detection for intelligent transportation and surveillance

Pedestrian detection in conventional frame-based imaging often suffers from limited temporal responsiveness and substantial data redundancy. Inspired by the biological retina, event-based vision sensing (EVS) offers ultra-low latency, high temporal resolution, wide dynamic range, and low power consumption, making it highly attractive for pedestrian perception in complex environments. This paper provides a comprehensive review of EVS and its application to pedestrian detection in intelligent transportation and surveillance scenarios. We first summarize the sensing principles, historical development, and key advantages of event-based vision in comparison with conventional frame-based imaging. We then review the major methodological components of event-based pedestrian detection, including sensing inputs, event representations, preprocessing strategies, feature extraction, detection models, datasets, and evaluation metrics. In addition, representative methods are comparatively analyzed in terms of temporal fidelity, detection accuracy, computational efficiency, and deployment complexity. Finally, we discuss the major open challenges in current EB-PD research, including benchmark standardization, event-native model design, multimodal fusion, and real-world deployment, and outline several promising directions for future development. This review aims to provide a structured and up-to-date reference for researchers working on event-based pedestrian perception and related intelligent vision systems.

Han Wang, Juntao Wu, Jingyuan Bao +5

Event-Based VisionEvent-Based Object Detection
AskPDF
Nov 30, 2023

Event-based Continuous Color Video Decompression from Single Frames

We present ContinuityCam, a novel approach to generate a continuous video from a single static RGB image and an event camera stream. Conventional cameras struggle with high-speed motion capture due to bandwidth and dynamic range limitations. Event cameras are ideal sensors to solve this problem because they encode compressed change information at high temporal resolution. In this work, we tackle the problem of event-based continuous color video decompression, pairing single static color frames and event data to reconstruct temporally continuous videos. Our approach combines continuous long-range motion modeling with a neural synthesis model, enabling frame prediction at arbitrary times within the events. Our method only requires an initial image, thus increasing the robustness to sudden motions, light changes, minimizing the prediction latency, and decreasing bandwidth usage. We also introduce a novel single-lens beamsplitter setup that acquires aligned images and events, and a novel and challenging Event Extreme Decompression Dataset (E2D2) that tests the method in various lighting and motion profiles. We thoroughly evaluate our method by benchmarking color frame reconstruction, outperforming the baseline methods by 3.61 dB in PSNR and by 33% decrease in LPIPS, as well as showing superior results on two downstream tasks.

Ziyun Wang, Friedhelm Hamann, Kenneth Chaney +3

University of Pennsylvania, USA. · TU Berlin, ECDF Berlin, SCIoI Cluster and RIG, Germany. · Archimedes, Athena RC

Event-Based Vision
AskPDF
Jun 8, 2023

Point-Voxel Absorbing Graph Representation Learning for Event Stream based Recognition

Sampled point and voxel methods are usually employed to downsample the dense events into sparse ones. After that, one popular way is to leverage a graph model which treats the sparse points/voxels as nodes and adopts graph neural networks (GNNs) to learn the representation of event data. Although good performance can be obtained, however, their results are still limited mainly due to two issues. (1) Existing event GNNs generally adopt the additional max (or mean) pooling layer to summarize all node embeddings into a single graph-level representation for the whole event data representation. However, this approach fails to capture the importance of graph nodes and also fails to be fully aware of the node representations. (2) Existing methods generally employ either a sparse point or voxel graph representation model which thus lacks consideration of the complementary between these two types of representation models. To address these issues, we propose a novel dual point-voxel absorbing graph representation learning for event stream data representation. To be specific, given the input event stream, we first transform it into the sparse event cloud and voxel grids and build dual absorbing graph models for them respectively. Then, we design a novel absorbing graph convolutional network (AGCN) for our dual absorbing graph representation and learning. The key aspect of the proposed AGCN is its ability to effectively capture the importance of nodes and thus be fully aware of node representations in summarizing all node representations through the introduced absorbing nodes. Extensive experiments on multiple event-based classification benchmark datasets fully validated the effectiveness of our framework.

Yuxiang Zhang, Chengguo Yuan, Xiao Wang +2

aSchool of Computer Science and Technology, Anhui University, Hefei 230601, China.

Event-Based VisionGraph Neural NetworksGraph Representation LearningGraph Pooling3D Representation Learning
AskPDF