ALIVE: Bring Foundation Features to Life with Events
Organizations: Robotics and Perception Group, University of Zurich · Tsinghua University
Abstract
Visual foundation models provide powerful representations for diverse tasks, but their features become stale as visual content changes between image observations. Refreshing them requires new frames and repeated runs of a computationally expensive image encoder. We introduce ALIVE, an event-based method that incrementally updates native foundation model features between image observations. Given an anchor image, ALIVE uses subsequent events to transport and correct existing multi-layer features extracted by a frozen DINOv3 encoder. Its lightweight updater is trained end-to-end using only synthetic data, without downstream task supervision, and transfers directly to real-world event streams without any adaptation. Across three real-world datasets and five task-dataset settings spanning object detection, semantic segmentation, and depth estimation, ALIVE outperforms all evaluated event-based methods in every metric. On DSEC, when the anchor frame is 200,ms old, ALIVE improves over anchor-image predictions by 30.9,mAP in detection, 11.6,mIoU in segmentation, and 15.9% in depth AbsRel. Compared with image re-encoding, ALIVE reduces FLOPs by 82.4% and latency by a factor of 2.5.
Figures & tables
| Method | Detection | Segmentation | Depth (0.1–30 m) | Cost per update | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DSEC [ 14 ] | DSEC [ 14 ] | DDD17 [ 5 ] | DSEC [ 14 ] | CoSEC [ 30 ] | Params (M) | GFLOPs | Latency (ms) | |||||
| mAP | AP50 | AP75 | mIoU | mIoU | AbsRel | RMSE | AbsRel | RMSE | ||||
| Stale RGB | 27.92 | 61.10 | 22.06 | 57.16 | 66.85 | 0.145 | 4.341 | 0.182 | 4.303 | – | – | – |
| E-RAFT [ 15 ] | 51.51 | 84.17 | 53.93 | 65.51 | 69.85 | 0.182 | 4.418 | 0.195 | 4.335 | 5.33 | 690.1 | 71.56 |
| EDCFlow [ 25 ] | 51.93 | 84.57 | 54.48 | 65.40 | 70.18 | 0.182 | 4.429 | 0.195 | 4.336 | 2.50 | 667.6 | 124.43 |
| E2VID [ 35 ] | 33.48 | 61.90 | 31.98 | 48.63 | 25.38 | 0.255 | 6.403 | 0.256 | 5.555 | 10.71 | 741.5 | 28.96 |
| Variant | Det. | Seg. | Depth | ||
|---|---|---|---|---|---|
| mAP | AP75 | mIoU | AbsRel | RMSE | |
| Full model | 52.55 | 56.27 | 66.30 | 0.129 | 3.741 |
| w/o events | 27.80 | 21.35 | 56.51 | 0.160 | 4.641 |
| w/o feature transport | 42.10 | 40.43 | 61.02 | 0.140 | 4.078 |
| w/o memory transport | 51.56 | 54.13 | 65.86 | 0.132 | 3.788 |
| w/o memory | 50.19 | 51.52 | 65.07 | 0.134 | 3.897 |
Event-Based Vision
Event-based vision uses asynchronous camera measurements to perceive motion and scene changes with high temporal resolution and dynamic range.
See all 252 papers