Bio-inspired event cameras have recently attracted significant research due to their asynchronous and low-latency capabilities. These features provide a high dynamic range and significantly reduce motion blur. However, because of the novelty in the nature of their output signals, there is a gap in the variability of available data and a lack of extensive analysis of the parameters characterizing their signals. This paper addresses these issues by providing readers with an in-depth understanding of how intrinsic parameters affect the performance of a model trained on event data, specifically for object detection. We also use our findings to expand the capabilities of the downstream model towards sensor-agnostic robustness.
Figures & tables
Figure 1: Overview of a sensor control loop. We aim to dynamically adapt the sensor characteristics based on energy utilization and task performance feedback. In this work, we investigate and mitigate the undesired effects of dynamically varying spike transduction on task performance, i.e . , object detection.
Figure 2: Examples of two scenes with varying configurations which produce high and low event densities, respectively. The black and white points represent negative and positive events, respectively, accumulated over a 50-ms time period. The gray spaces indicate the occurrence of no events within this timeframe at the corresponding pixel locations. The corresponding instance segmentation map is provided for reference to the actual vehicular locations.
Config. ( \mathbfcalEi )
Pos. Thr. ( thp )
Neg. Thr. ( thn )
Ref. Period ( Tr ) [ms]
Field of View ( Fv )
\mathbfcalEbase
0.5
0.5
10
90∘
\mathbfcalE1
0.25
0.25
0.01
90∘
\mathbfcalE2
0.75
0.75
0.01
90∘
\mathbfcalE3
1.0
1.0
0.01
90∘
\mathbfcalE4
0.5
0.5
10
90∘
E5
0.5
0.5
25
90∘
Table 1: Data Configurations.
Figure 3: Simplified overview of the design and positioning of the distinct test sets in the parameter space of two parameters - the field of view and negative threshold. The training set and the first test set consist of distinct configurations varying along each dimension. The second test set consists of configurations obtained by interpolating along each dimension, whereas the third set is obtained by combining seen parameters into unique configurations. The fourth set represents the unseen parameter space for each parameter.
Figure 4: Comparison of results between the different models - RVT-B and SSMS-B trained on Ebase and Strain . The blue boxes indicate the predicted bounding boxes, and the red boxes represent the ground truth. Each of the three samples represents different sensor configurations to illustrate the performance across varying scenarios.
Train/Test
RVT-B
SSMS-B
Config.
\mathbfcalEbase
Strain
\mathbfcalEbase
Strain
Stest1
\mathbfcalEbase
45.63
44.12
49.10
50.58
\mathbfcalE1
44.49
45.94
51.99
53.90
\mathbfcalE3
23.74
30.23
26.72
33.01
\mathbfcalE4
45.13
44.72
48.97
50.55
\mathbfcalE6
44.61
44.13
48.73
50.05
Table 2: Comparison of performance (AP scores in % ) for each configuration.
Scorek ( f )
AP(m1)
AP50(m2)
AP75(m3)
APL(m4)
APM(m5)
RVT-B
SSMS-B
RVT-B
SSMS-B
RVT-B
SSMS-B
RVT-B
SSMS-B
RVT-B
SSMS-B
Score1(\mathbfcalEbase)
34.6 ± 14.7
38.2 ± 16.6
59.7 ± 17.9
60.9 ± 19.8
37.4 ± 19.7
42.8 ± 21.6
44.0 ± 18.0
49.6 ± 20.4
30.0 ± 13.4
32.0 ± 14.9
Score1(Strain)
37.4 ± 10.7
42.4 ± 13.1
63.4 ± 11.3
65.3 ± 14.2
40.7 ± 15.5
47.7 ± 17.5
47.4 ± 14.1
53.8 ± 17.4
32.3 ± 9.3
36.4 ± 11.4
Score2(\mathbfcalEbase)
37.5 ± 7.3
39.9 ± 8.5
64.6 ± 6.3
65.3 ± 6.9
39.7 ± 12.4
44.0 ± 12.1
46.3 ± 12.8
46.9 ± 16.1
33.5 ± 6.0
35.9 ± 6.0
Score2(Strain)
39.4 ± 4.5
44.8 ± 5.4
66.7 ± 3.6
68.9 ± 4.4
43.1 ± 7.8
51.3 ± 7.1
47.4 ± 10.7
54.4 ± 12.3
35.6 ± 4.6
39.8 ± 5.6
Score3(\mathbfcalEbase)
34.1 ± 0.2
40.6 ± 0.1
62.3 ± 4.5
66.7 ± 2.7
34.4 ± 3.7
45.2 ± 0.0
39.3 ± 3.3
54.9 ± 4.4
27.6 ± 1.5
31.9 ± 0.6
Table 3: Complete Performance Comparison on the RVT-B Model and the SSMS-B Model. We observe that the SSMS-B Model trained on Strain performs the best uniformly in each score metric, across all test sets.
Event cameras are bio-inspired sensors that asynchronously capture logarithmic intensity changes, offering inherent advantages in high-speed and high-dynamic-range scenarios. However, the sparse and asynchronous nature of event streams poses a fundamental challenge for modern deep learning architectures. To enable compatibility with standard models, most existing approaches partition the accumulation window into fixed temporal sub-bins. While effective for spatial processing, this internal discretization discards fine-grained temporal structure and constrains inference to the low temporal frequencies imposed by training supervision. To address this limitation, we propose FATE, a unified framework built upon a novel Pillar Encoding (PE). While operating over discrete macro-accumulation windows dictated by the target frequency, PE avoids internal temporal sub-binning. It organizes events into spatial pillars and approximates their intra-window evolution via projection onto a continuous-time orthogonal polynomial basis. This formulation yields an L2-optimal representation that retains rich temporal dynamics in a dense pseudo-image, mitigating information loss under sparse event conditions. To fully leverage this representation, we introduce Frequency-Aware Training (FAT), a soft mean-teacher curriculum that generates temporally dense pseudo-labels, effectively bridging the mismatch between low-frequency supervision and high-frequency inference. Extensive experiments demonstrate that FATE generalizes across architectural paradigms and consistently outperforms strong baselines. It enables robust object detection at high temporal resolutions up to 200 Hz, while incurring minimal overhead in parameter count and inference latency
Md Tawheedul Islam Bhuian, Kyoung-Don Kang
School of Computing State University of New York at Binghamton
Event-based cameras are bio-inspired sensors that detect light changes asynchronously for each pixel. They are increasingly used in fields like computer vision and robotics because of several advantages over traditional frame-based cameras, such as high temporal resolution, low latency, and high dynamic range. As with any camera, the output's quality depends on how well the camera's settings, called biases for event-based cameras, are configured. While frame-based cameras have advanced automatic configuration algorithms, there are very few such tools for tuning these biases. A systematic testing framework would require observing the same scene with different biases, which is tricky since event cameras only generate events when there is movement. Event simulators exist, but since biases heavily depend on the electrical circuit and the pixel design, available simulators are not well suited for bias tuning. To allow reproducibility, we present BiasBench, a novel event dataset containing multiple scenes with settings sampled in a grid-like pattern. We present three different scenes, each with a quality metric of the downstream application. Additionally, we present a novel, RL-based method to facilitate online bias adjustments.
Andreas Ziegler, David Joseph, Thomas Gossard +2
Cognitive Systems Group, University of Tübingen, Germany
Event cameras are a bio-inspired class of sensors that asynchronously measure per-pixel intensity changes. Under fixed illumination conditions in static or low-motion scenes, rigidly mounted event cameras are unable to generate any events and become unsuitable for most computer vision tasks. To address this limitation, recent work has investigated motion-induced event stimulation, which often requires complex hardware or additional optical components. In contrast, we introduce a lightweight approach to sustain persistent event generation by employing a simple rotating unbalanced mass to induce periodic vibrational motion. This is combined with a motion-compensation pipeline that removes the injected motion and yields clean, motion-corrected events for downstream perception tasks. We develop a hardware prototype to demonstrate our approach and evaluate it on real-world datasets. Our method reliably recovers motion parameters and improves both image reconstruction and edge detection compared to event-based sensing without motion induction.
Vincenzo Polizzi, Stephen Yang, Quentin Clark +3
University of Toronto, Robotics Institute · University of Toronto, Department of Mechanical and Industrial Engineering · University of Toronto +1