Event-based object detection under adverse lighting and motion: lessons for construction sites
Organizations: School of Computer Science, Faculty of Engineering, The University of Sydney, NSW, Australia · School of Project Management, Faculty of Engineering, The University of Sydney, NSW, Australia
Abstract
Vision-based monitoring on construction sites fails at night, at dawn and dusk, in tunnels and under floodlights, when conventional RGB cameras under- or over-expose. Event cameras record per-pixel brightness changes with microsecond latency and over 120 dB of dynamic range, but their benefit for site-relevant object detection has not been quantified on real data. We compare three recent event-based detectors (SMamba, SATE and EvRT-DETR) with an RGB detector trained on the same data, using the pixel-aligned PEOD dataset. On low-light frames (70% of the test set) the best event detector reaches 27.2 AP against 21.3 AP for RGB, and a training-free late fusion of both sensors is the best option in every lighting condition (33.3 AP). An object-level analysis of 53,091 objects shows that brightness has little effect on event detectors, whereas fast motion matters: above 20 pixels per frame RGB finds 11-19% of medium-size objects and EvRT-DETR 39-60%. Activation patching locates the damage caused by losing events at the 1/16-resolution features, with small objects depending on the finest scale, and shows that event detectors fail when few signal events meet added noise. A model-agnostic pruning study keeps about 95% of accuracy with 60% of the input but halves small-object recall. On rendered construction sites, used zero-shot, dimming the floodlights drops RGB to near zero while the event detector keeps most of its AP50. We close with camera-placement guidance for construction sites.
Figures & tables
| Model | Pre-training | Res. | Training on PEOD |
|---|---|---|---|
| SMamba | none | full | 120k steps |
| SMamba + 1Mpx | 1Mpx | half | 20k steps |
| SATE | none | full | 60k steps, bf16 |
| EvRT-DETR | 1Mpx (Gen4) | half | 2 stages, 30 epochs |
| RGB zero-shot | COCO | full | none |
| RGB trained | COCO | full | 15 epochs |
| Subset | Frames | RGB z-s | RGB tr. | SMamba | EvRT |
|---|---|---|---|---|---|
| Low light | 8,161 | 9.0 | 21.3 | 19.4 | 27.2 |
| Overexposure | 681 | 37.8 | 23.8 | 17.8 | 23.5 |
| Well-lit | 2,817 | 37.8 | 39.7 | 30.7 | 42.0 |
| All | 11,659 | 14.6 | 27.1 | 22.4 | 32.2 |
| Model | All | LL | OE | WL | Found LL (small) |
|---|---|---|---|---|---|
| RGB trained | 22.6 | 23.0 | 18.4 | 23.0 | 38% (3%) |
| SMamba | 18.8 | 20.0 | 5.0 | 16.9 | 53% (8%) |
| EvRT-DETR | 25.5 | 27.6 | 9.7 | 21.4 | 69% (30%) |
| Fused | 27.5 | 28.4 | 18.9 | 26.1 | 70% (30%) |
| Subset | Objects | Ev. only | RGB only | Both | Fused |
|---|---|---|---|---|---|
| Low light | 37,790 | 6,669 | 542 | 21,086 | 28,102 |
| Overexposure | 2,334 | 262 | 65 | 1,509 | 1,827 |
| Well-lit | 12,967 | 1,639 | 531 | 8,201 | 10,321 |
| Subset | RGB | EvRT | Fused |
|---|---|---|---|
| Low light | 21.3 | 27.2 | 27.8 |
| Overexposure | 23.8 | 23.5 | 26.0 |
| Well-lit | 39.7 | 42.0 | 44.7 |
| All | 27.1 | 32.2 | 33.3 |
| Model | All | LL | OE | WL | LL/WL |
|---|---|---|---|---|---|
| SMamba | 22.4 | 19.4 | 17.8 | 30.7 | 0.63 |
| SMamba + 1Mpx | 26.2 | 25.0 | 21.1 | 32.3 | 0.77 |
| SATE | 28.2 | 24.8 | 20.8 | 33.1 | 0.75 |
| EvRT-DETR | 32.2 | 27.2 | 23.5 | 42.0 | 0.65 |
| Size | Count, 60% | Local, 60% | Count, 20% | |||
|---|---|---|---|---|---|---|
| box | ring | box | ring | box | ring | |
| Small | 99 | 93 | 93 | 67 | 88 | 54 |
| Medium | 97 | 80 | 84 | 60 | 74 | 36 |
| Large | 89 | 62 | 71 | 59 | 55 | 22 |
| Kept | No noise | p=0.001 | p=0.003 | p=0.01 |
|---|---|---|---|---|
| 100% | 42.7 | 38.4 | 32.8 | 20.0 |
| 30% | 29.9 | 13.9 | 3.7 | 0.3 |
| Site | Lighting | RGB | EvRT |
|---|---|---|---|
| Tunnel | day | 13.7 / 18.8 | 5.5 / 9.5 |
| Tunnel | low sun | 18.5 / 23.7 | 8.3 / 13.6 |
| Rebar | day | 0.0 / 0.0 | 17.7 / 34.4 |
| Rebar | floodlit night | 0.0 / 0.0 | 18.5 / 35.3 |
| Sand | day | 10.4 / 18.3 | 4.1 / 11.9 |
| Sand | floodlit night | 11.1 / 16.8 | 6.8 / 24.1 |
Event-Based Object Detection
Event-based object detection localizes and classifies objects from asynchronous camera events, aiming for fast, robust perception with low latency and power.
See all 40 papers