UnrealEvent: Synthetic Dataset Generation and Annotation for Event-Based Vision
Organizations: EAVISE-PSI, ESAT, KU Leuven, Belgium
Abstract
Event-based cameras provide high temporal resolution, low power consumption, and a high dynamic range, but progress in event-based object detection remains constrained by the scarcity of annotated training data. To solve this issue, this paper introduces the tool UnrealEvent, a synthetic dataset-generation pipeline implemented in Unreal Engine 5.5. The pipeline combines a V2E-inspired event-frame generator with EasySynth semantic-rendering passes, allowing rendered level sequences to be converted into paired event representations and YOLO-format bounding-box annotations without manual labeling. Events are generated from per-pixel log-luminance differences between consecutive frames, with contrast thresholds sampled independently from a normal distribution to emulate the fabrication-induced threshold mismatch of real sensors. Using this pipeline, we provide a synthetic dataset and evaluate YOLOv8 detectors under synthetic, zero-shot real, and real fine-tuning training on eTraM. The results show strong in-domain synthetic performance but a substantial synthetic-to-real gap, especially for pedestrians. Fine-tuning on a small real subset improves real-domain performance, while the remaining gap highlights the importance of asset diversity and sensor calibration.
Figures & tables
| Event-based data simulators | ||||
|---|---|---|---|---|
| Simulator | Noise | Method | Defining feature | Year |
| DAVIS Simulator [ 18 ] | Ideal Pixel | Based on Blender | Linear Time Interpolation Sampling | 2017 |
| ESIM [ 24 ] | Standard Noise Model | Based on Rendering Engine | Adaptive Sampling | 2018 |
| V2E [ 12 ] | Temporal Noise, Leak Events, Finite Bandwidth, Gaussian Threshold Distribution | RGB-video to Event Sequence | Inclusion of Motion Blur Effects | 2021 |
| DVS-Voltmeter [ 17 ] | Stochastic Process | Stochastic Process-based Event Simulator | Incorporating Sensor Physics | 2022 |
| V2CE [ 34 ] | Learned by Training on Real Event Voxels | RGB-video to Event Sequence | Local Dynamics-Aware Timestamp Inference | 2024 |
| YOLOv8 AP50 Results | ||
|---|---|---|
| Data Regime | Pedestrian | Vehicle |
| eTraM † | ||
| Synthetic data | ||
| Zero-shot on eTraM | ||
| Fine-tuned on eTraM | ||
| Configuration | AP50 | mAP@ [.5:.95] | AP50 [95% CI] |
|---|---|---|---|
| Baseline | |||
| Real-only (from scratch) | — | ||
| Synthetic pre-training (ours) | |||
| Frozen backbone | [ ] | ||
| Full fine-tune | [ ] | ||
Synthetic Data Generation
Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.
See all 658 papers