Hypnos: A Dual-Phase End-to-End Platform for Scalable Continual Learning at the Edge
Organizations: Department of Microelectronics, Delft University of Technology, 2628 CD Delft, The Netherlands
Abstract
Continual learning (CL) enables neural networks to adapt to non-stationary data streams by mitigating catastrophic forgetting, offering a path to robust, private, and adaptable edge devices. Yet, porting CL to real-world edge devices remains challenging for two reasons: (i) online operation, where sensor data arrives one sample at a time and the device must adapt on-the-fly, and (ii) the low footprint of edge devices, which makes traditional error backpropagation training prohibitive. Inspired by the brain's wake and sleep cycles, the former for rapid adaptation with minimal resources and the latter for long-term consolidation, we introduce Hypnos, an open-source dual-phase platform for efficient CL at the edge. In the wake phase (e.g., daily operation), Hypnos relies on a low-footprint system-on-chip with a RISC-V microcontroller unit (MCU) targeting efficient inference and supporting lightweight gradient-free prototype updates. In the sleep phase (e.g., night-time charging), where wall-plug energy and larger memory budgets are available, a smart charger performs rehearsal-based gradient updates for consolidation. Deployed on a ZCU104 field-programmable gate array (FPGA) and synthesized in a 28-nm process, Hypnos is benchmarked on the CORe50 NICv2-391 protocol. It sets a new state-of-the-art accuracy of 78.6%-81.6% for edge CL systems, with an on-device latency of 7.79s-10.54s per sample depending on activation precision, and requires fewer learning steps and less learning compute than quantized latent replay at higher accuracy and comparable latency.
Figures & tables
| Layer 15 | Layer 16 | |||
|---|---|---|---|---|
| Activations | w/o PQ | w/ PQ | w/o PQ | w/ PQ |
| FP32 | ||||
| INT16 | ||||
| INT8 | ||||
| Block a | LUTs | FFs | DSPs | LUTRAM b | BRAM b | URAM b |
| (design relative %) | ||||||
| MCU baseline | 22,573 | 16,132 | 5 | 169.0 kB | 756.0 kB | 3.4 MB |
| 26.0% | 29.1% | 2.2% | 40.2% | 54.5% | 100.0% | |
| Matrix co-proc. | 23,049 | 10,320 | 192 | – | – | – |
| 26.5% | 18.6% | 83.8% | ||||
| Accelerator | 16,334 | 8,877 | 32 | 225.0 kB | 612.0 kB | – |
| Hypnos A8 | Hypnos A16 | QLR-CL | QLR-CL | |
| Backbone | MobileNetV3 | MobileNetV3 | MobileNetV1 | MobileNetV1 |
| Latent split (layer) | 15 | 15 | 19 | 27 |
| Weight precision | INT8 a | INT8 a | INT8 b | INT8 b |
| Activation precision | INT8 | INT16 | INT8 b | INT8 b |
| Replay buffer precision | PQ (8-bit) | PQ (8-bit) | INT7 | INT7 |
| Learning OPs/step c | 0.343 G | 0.343 G | 7.70 G | 5 M |
Continual Learning
Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.
See all 327 papers