Rendering-Free Event Frame Simulation for Fast Reinforcement Learning of Autonomous Robot Navigation
Authors: Dequan Ou, Austin Lopez, Quentin Missinne, Christophe De Wagter, Guido C. H. E. de Croon
Organizations: All authors are with the Micro Air Vehicle Lab of the Faculty of Aerospace Engineering, Delft University of Technology, 2629 HS Delft, The Netherlands
Event cameras offer low latency, high dynamic range, and milliwatt-level power consumption, making them ideal for agile robots such as tiny drones navigating through cluttered environments. However, existing simulators generally render photorealistic images before converting them into events, making large-scale reinforcement learning prohibitively slow. We propose an analytic event simulator that exploits what event cameras fundamentally measure - moving contrast edges - to synthesize event streams directly from scene geometry and ego-motion without visual rendering. Our simulator generates a 64x64 event observation in 17 microseconds on a single GPU - over 400x faster than a state-of-the-art rendering pipeline - enabling end-to-end reinforcement learning for goal-directed obstacle avoidance to converge in under 30 minutes - three orders of magnitude faster than training on rendered events. Policies transfer zero-shot to physically based event simulations (ray-traced Isaac Sim). Our policy reaches 83% success in cluttered courses unseen during training (mid-difficulty, traversability probability approx. 0.57), at 2.4 m/s average goal velocity, and retains 70% in a simulated texture-dense indoor hall. Furthermore, we quantized the policy to a 1.2 MB int8 network running fully onboard a 56-gram Crazyflie equipped with a Prophesee GenX320 event camera and an OpenMV N6 microcontroller board. Without any fine-tuning, external processing, or positioning, the policy achieves an average success rate of 82% (5/5, 5/6, 4/6 flights) across three real-world obstacle courses of increasing difficulty. Code: https://github.com/tudelft/rendering-free-event-nav
Figures & tables
Fig. 1 : (a) A real flight through our test arena: the 56 g Crazyflie (inset) navigates the clutter fully onboard using a GenX320 event camera and an STM32N6 microcontroller. (b–d) Event frames from the three worlds (blue ON, red OFF): (b) the analytic proxy used for training and (c) the Isaac Sim+EVIS [ 1 ] rendered validation, both showing a replica of this arena from a matched viewpoint; (d) the real sensor view in flight.
Fig. 2 : The rendering-free event-proxy pipeline: one obstacle (orange) is traced through every stage. (a) A scene is only 3D edge points on depth discontinuities (trunks brown, walls gray, props green; the red triangle is the camera, the arrow its velocity). (b) Visible points project to the image plane and move under ego-motion; arrow color is polarity. (c) The two firing mechanisms: per-cell event mass follows a saturating trigger (the block-level aggregate of binary pixel crossings; its knee—the displacement at which a cell saturates—is fitted to real sensor recordings), and polarity is a hashed contrast class times the covering/revealing side of the moving edge. (d) Signed accumulation yields the 64×64 DIFF3D frame the policy observes; the traced obstacle’s streak is circled.
parallel envs
512
rollout length
128
epochs / minibatches
2 / 8
discount γ
0.99
GAE λ
0.95
clip ϵ
0.2
learning rate (Adam)
3×10−4
entropy coef.
5×10−3
value coef.
0.5
reward scale
0.05
total steps
40 M
wall-clock
34 min
TABLE I : Training configuration (identical for all policies).
Fig. 3 : Simulation evaluation under the shared action interface. Top and middle rows: the two appearance conditions with identical obstacle layouts—the shadow scene (bare surfaces, directional light) and the texture scene (court markings, panels, window bands under diffuse light)—as ray-traced views (left) and simulated 64×64 DIFF3D observations (blue ON, red OFF); the texture scene adds background events the shadow scene lacks. Bottom: all 30 evaluation episodes of our policy in each condition (every episode a distinct procedurally generated layout; blue = reached, red × = failure, dashed circle = goal), 25/30 and 21/30.
active px
ON share
p50/p90
blob
blk. var.
proxy (training)
15%
37%
2 / 6
1.19
216
Isaac+EVIS, shadow
25%
40%
5 / 14
1.59
1801
Isaac+EVIS, texture
54%
40%
3 / 13
1.61
2882
real GenX320
66%
39%
3 / 14
1.54
6947
TABLE II : Event-frame statistics of the policy’s operational worlds: proxy training scenes (champion flights, domain randomization active), the two Isaac evaluation conditions, and a real in-flight GenX320 recording. Net-count deviations from the 128 background; block variance over 4×4 -pixel sums; trajectories differ across rows.
Fig. 4 : The 56 g flight platform. Top: exploded view—Crazyflie 2.1 base (24 g), LiPo battery (9 g), OpenMV N6 board (9 g), Prophesee GenX320 event camera (3 g), printed camera mount (3 g), and flow deck (2 g) for velocity estimation. Bottom: assembled, front and side. The 56 g total includes these components, cables, and a boost converter.
sensor sim
closed loop
wall-clock to
( 64×64 frame)
(policy steps/s)
trained policy
Rendered (Isaac+EVIS)
6.9 ms
∼ 2.5
∼ 200 days †
Analytic proxy (ours)
17 μ s ‡
19–21k
34 min
† extrapolated from the measured GPU-day (195k steps).
‡ full observation path (bare event splat 4 μ s).
TABLE III : Measured event-generation and training throughput (RTX 4090, identical scenes).
input
SR (shadow)
SR (texture)
AGV
PPO, privileged depth
depth
100%
97%
2.19
PPO, analytic proxy (ours) a
events
83%
70%
2.38
Flow balancing (reactive)
events
13%
17%
1.10
PPO, rendered, 1 GPU-day
events
40%
53%
1.78
a All methods share the deployed action interface
( v∈[0,3.5] m/s at 13.3 Hz); scenes: Sec. IV-A .
TABLE IV : Baseline comparison in the Isaac protocol ( n=30 , identical seeds, props). AvoidBench metrics: success rate, average goal velocity (m/s, shadow scene), path optimality (excess over the A*-shortest path; successes only).
Fig. 5 : All real flights, overhead composites. Top: environment 1 (poles and boxes), 5/5 reached. Middle: environment 2 (extended course with block towers, plants, and screens), 5/6. Bottom: environment 3 (compact arena with tight passages), 4/6; crashes marked. All flights fully onboard—GenX320 events, int8 policy on the N6, flow-based velocity—with no external processing or positioning.
Leibniz Universität Hannover, L3S Research Center, Hanover, Germany · Ekumen, Buenos Aires, Argentina · University of Southern Denmark, Odense, Denmark +1