cs.CVOct 14, 2026

Event Cameras versus Frame Cameras for Synthetic-Aperture Imaging: A Simulation Study

Authors: Jakob Hinterholzer, Mohamed Youssef, Oliver Bimber

Organizations: Johannes Kepler University Linz Institute of Computer Graphics Linz, Austria

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Synthetic-aperture imaging (SAI) removes occluders such as foliage by integrating many views of a scene along a camera path. It has so far been used mainly with frame cameras. We ask whether an event camera has advantages for SAI, and answer the question in simulation, where both sensors see exactly the same scene and the same camera movement. A densely rendered Blender sequence is turned into frames with an EMVA~1288 noise model of an industrial global-shutter camera, and into an event stream with the v2e simulator parameterized like a Sony IMX636. The frames are integrated with the existing airborne optical sectioning renderer. For the events we add a new integration path that projects every single event onto the focal plane at its own interpolated pose, so no events have to be binned into frames. We compare the two sensors in three main runs and an additional dark high-speed test. In a bright scene at 25,m/s the frame camera records only three images on the path and cannot remove the occluder, while the event camera reconstructs the target, even after an event-rate controller drops most of its events to fit the camera link. At the same speed in a 5,lux scene the event camera no longer resolves the target either. In a 5,lux scene at 5,m/s both sensors reconstruct the target at similar quality. We also compare the data one traversal of the aperture costs. At high speed the capped event stream still shows the target at the same data volume as the three frames. At 1,m/s we compare both cameras at two matched data budgets, and there the event camera has no advantage over the frame camera in reconstruction quality. The data rate of the event camera depends on the scene and is highest exactly in the fast bright scenes where it is needed most.

Figures & tables

Figure 1. Overview of the pipeline with its three stages. Flowchart with one box at the top for the Blender rendering, two parallel columns below it for the frame-camera and event-camera models and their integrations, and one box at the bottom for the resulting integral images.
Figure 3. Event-based synthetic-aperture integral produced by the proposed integration path (bright scene, 2525 m/s). (a) Signed net polarity, balanced pixels render mid-gray. (b) Absolute value of the net polarity, inverted, so cancelled pixels render white and events black. The occluding foliage of Figure 4 washes out, while the bicycle on the focal plane, including spokes and frame, is reinforced. Two square grayscale reconstructions side by side: on the left a clearly outlined bicycle with visible spokes on a nearly uniform gray background, on the right the same bicycle in dark gray on a white background; the foliage visible in the earlier scene figure is gone in both.
Figure 4. One viewpoint from the shared trajectory. Near-field foliage occludes a bicycle on the focal plane; the 0.50.5 m path supplies the remaining viewpoints of the synthetic aperture. Grayscale rendered view through two bushes. A bicycle wheel is partly visible in a gap in the foliage; the ground is covered in leaves.
 High speedLow lightMatched data
Scene illuminance10001000 lux55 lux10001000 lux
Camera speed2525 m/s55 m/s11 m/s
Duration TT0.020.02 s0.10.1 s0.50.5 s
Frame views nn3316168080 / 4040
Event cutoff_hz10001000 Hz100100 Hz10001000 Hz
ERC (MEv/s)5050 / 2020off2020 / 1010
Table 1. The two evaluation points and the additional run at 11 m/s. Frame-view counts are n=160 fps⋅Tn=160\,\text{fps}\cdot T over the 0.50.5 m aperture, rounded down.
ParameterValueSource
Simulated resolution720×720720\times 720matched
Frame rate160160 fpsEMVA
Peak QE62%62\%EMVA
Pixel pitch3.453.45 µmEMVA
Read noise3.03.0 e -EMVA
Full-well capacity10.510.5 ke -EMVA
Table 2. Frame-camera model. EMVA rows are Basler’s measurement of the ace 2 a2A1920-160umBAS (Sony IMX392) ( Basler AG, 2024 ) . Exposures are chosen as the frame camera’s best case at each operating point.
ParameterValueSource
Simulated resolution720×720720\times 720matched
ERC budget (bright)5050 MEv/sSection 4.4
Contrast threshold0.250.25brief
Threshold mismatch0.030.03v2e
Refractory period0.10.1 msv2e
Shot-noise rate (bright / night)0.10.1 / 1010 Hz/pxbrief
Table 3. Event-camera model. IMX636-class target specifications ( Finateu et al., 2020 ; Prophesee and Sony Semiconductor Solutions, 2021 ) and v2e non-ideality settings. The photoreceptor cutoff is a modelling assumption (Section 4.3 ), not a datasheet number.
Figure 5. High speed ( 2525 m/s, 10001000 lux). (a) Frame SAI from the three frames the camera records on the path. (b) Event SAI from the full event stream. (c) and (d) Event SAI after the ERC drops the stream to 5050 and 2020 MEv/s. The bicycle survives both; fine structure and smoothness do not. Four square grayscale reconstructions. Top left: a wall of foliage in which only the two wheel rims of the bicycle can be made out. Top right: a clearly outlined bicycle with spokes on a nearly uniform gray background. Bottom: the same bicycle after capping, both grainier with fainter spokes, the right one grainier than the left.
Figure 6. Low light ( 55 m/s, 55 lux). (a) Frame SAI from 1616 views. (b) Event SAI at the 100100 Hz photoreceptor cutoff. Both recover the bicycle at a similar quality. Two square reconstructions. The left (frame) image shows a noisy bicycle with streaked leftover foliage. The right (event) image shows a fainter but complete bicycle on a smoother gray field.
RunStreamMEv/sMB/sMB/trav.
2525 m/s, 0.020.02 sFrames ( 33 )—83831.61.6
 Events, raw17717770970914.114.1
 Events, ERC 505050502002004.04.0
 Events, ERC 2020202080801.61.6
55 m/s, 0.10.1 s, nightFrames ( 1616 )—83838.38.3
 Events, raw212185858.68.6
Table 4. Data of the two cameras in all runs at 720×720720\times 720 . Event rates are measured from the simulated streams and converted with 44 bytes per event (Section 4.4 ). Frames are 88 bit per pixel at 160160 fps, except for the 4040 -view subset at 8080 fps. The last column is the data of one traversal of the 0.50.5 m aperture.
Figure 7. Two approximately matched data budgets in the bright scene at 11 m/s. Top: 8080 frames versus an ERC cap of 2020 MEv/s. Bottom: 4040 frames versus 1010 MEv/s. Volumes refer to one 0.50.5 s traversal. The 4040 views are every second frame of the original sequence, with unchanged exposure. Both modalities retain the bicycle at both budgets. Four grayscale reconstructions arranged by data budget in rows and sensor modality in columns. Left: the 80-frame and 40-frame reconstructions both show the bicycle against streaked residual foliage. Right: the 20 and 10 MEv/s event reconstructions show the bicycle against a more uniform but grainy background; the lower-budget image has more grain and less complete fine edges.
Figure 8. Event SAI in the dark at high speed ( 2525 m/s, 55 lux, 100100 Hz cutoff). Only a faint outline of one wheel remains, and the foliage is not averaged out. A square gray reconstruction dominated by streaked noise. A faint circle, the outline of one bicycle wheel, is visible slightly right of center.
More in this topic

Computational Imaging

Computational imaging combines optical and sensor design with algorithms to recover or infer scene information from indirect, compressed, or noisy measurements.

See all 123 papers

Explore similar work

CardsList
Jul 16, 2026

ESAR: Event-Based Synthetic Aperture Reconstruction

Event cameras report asynchronous polarity events when changes in log--radiance exceed a fixed contrast threshold, producing signed temporal contrast measurements rather than conventional image frames. We formulate monocular event-based imaging as a synthetic-aperture inverse problem for a static ground-domain log--radiance field θ∈RNgθ\in \mathbb{R}^{N_g}. Instead of reconstructing a latent pixel-time volume v∈RNpNtv \in \mathbb{R}^{N_pN_t}, we impose the geometric relation v=Pθv=Pθ, where PP maps the fixed scene into motion-dependent latent views. Aggregating events over finite time intervals gives the linearized model APθ=b+η,APθ= b+η, where AA is a temporal differencing operator, bb contains signed binned event counts, and ηη represents measurement and modeling errors. This decomposition exposes a synthetic-aperture structure: under near-nadir motion, successive projections are approximately shifted views of a common scene, while the composite operator APAP remains ill-conditioned because it combines spatial averaging with temporal differencing. We therefore use regularized inversion to recover θθ. Numerical experiments on simulated data and real near-nadir Falcon Neuro event data show that the proposed θθ-based formulation recovers coherent large-scale spatial structure, relative to dynamic latent-image and learned event-reconstruction baselines, while suppressing fine-scale texture.

Harbir Antil, Daniel Blauvelt, David Sayre

Event-Based VisionComputational ImagingImage Inverse Problems
AskPDF
Aug 9, 2026

EsaacSim: A Multimodal Event Camera Add-on for NVIDIA Isaac Sim

Event-based vision is becoming an increasingly important sensing paradigm for robotics, yet its adoption remains limited by sensor availability and the lack of integrated simulation tools for modern robotics platforms. This paper presents EsaacSim, a multimodal event camera add-on for NVIDIA Isaac Sim that enables online simulation of configurable event cameras with grayscale and Bayer RGGB event generation. The framework supports multiple event camera resolutions and provides synchronized RGB, APS, event, depth, and IMU outputs through native ROS2 interfaces. A motion-guided frame-gap synthesis strategy further increases the effective temporal resolution while preserving compatibility with the Isaac Sim rendering pipeline. Experimental evaluation demonstrates synchronized multimodal simulation across representative robotic scenes and efficient online performance over five event camera resolutions at effective event rates from 240 to 960Hz. Event stream generation requires 6.98--27.28ms for grayscale events and 7.58--29.16ms for Bayer RGGB events while using less than 400MB of additional GPU memory on an NVIDIA RTX~4060 GPU. These results show that EsaacSim enables supports online multimodal event-camera simulation for robotics research and synthetic data generation. We release an early version of the simulator and report its current architecture and performance.

Ignacio Bugueno-Cordova, Malte Kuhlmann, Nicolás Navarro-Guerrero +2

Leibniz Universität Hannover, L3S Research Center, Hanover, Germany · Ekumen, Buenos Aires, Argentina · University of Southern Denmark, Odense, Denmark +1

Event-Based VisionRobotics SimulationSynthetic Data Generation for Robotics
AskPDF
Sep 17, 2026

An Event Preserving Velocity Invariant Representation for Event Cameras

Event cameras provide low-latency, high temporal resolution perception for real-time vision tasks such as robotics.The novel circuitry (i.e. asynchronous, independent pixels) that enables these advantages also introduces new algorithmic challenges. Velocity-invariant representations alleviate missing observations under slow motion and motion blur under fast motion, but most discard temporal information by converting events into image-like representations. We propose Set of Centre Active Receptive Fields (SCARF), a real-time velocity-invariant representation that preserves raw events while consistently handling fast motion, stationary scenes, and independently moving objects. SCARF achieves state-of-the-art performance in both computational efficiency and representation quality.

Mikihiro Ikura, Luna Gava, Jiahang Wu +2

Istituto Italiano di Tecnologia, Via Morego 30 16163 Genova, Italy

Event-Based Vision
AskPDF