cs.NENov 1, 2026

HyVox: Rethinking Low-Power Voice Intelligence towards Neuromorphic-von Neumann Edge Computing

Authors: Snehasis Addy, Eduardo Calle Ortiz, Prashant Shenoy, Phuc Nguyen

Organizations: University of Massachusetts Amherst, Amherst, MA, USA

Abstract

We present HyVox, a hybrid neuromorphic--von Neumann system for low-energy open-set speaker identification under tight energy budgets. HyVox splits the workload at the speaker embedding. A neuromorphic processor embeds the speech with sparse, event-driven computation, and a von Neumann microcontroller holds the speaker gallery and makes every decision: it scores queries by cosine similarity, rejects unknown speakers, and enrolls new users by writing a prototype, with no weight update. On the neuromorphic processor, a hierarchical temporal abstraction (HTA) SNN backbone learns a synaptic time constant per leaky integrate-and-fire (LIF) neuron, allocating 832 neurons across fast, medium, and slow timescales. A 32-neuron spiking embedding extractor (SEE), pretrained on episodes that each enroll new speakers and reject outsiders, turns the backbone output into an embedding for speakers unseen in training; its 32 spike counts are the only interface between the two processors. To fit the chip's synapse and integer budgets, we compress the backbone 6.5x to 29.0k synapses, train it for integer bit-shift dynamics, and fine-tune it with chip-accurate spiking training (CAST). CAST trains through a differentiable replica of the chip's integer neuron update, derived from its cycle-accurate simulator, so that gradient descent optimizes the exact weights and thresholds the chip runs; it doubles the top-1 accuracy of the mapped backbone on unseen speakers. On 20 Common Voice households of unseen speakers, HyVox reaches 99.6% top-1 and 5.5% rejection EER with 2 members and 94.9% and 13.2% with 20, and the network deployed on the neuromorphic chip matches its trained model. Each inference on a 3-s window takes 0.96 s and 1.63 mJ of dynamic energy, 42% lower latency than the fastest embedded ANN baseline and 13.5x less dynamic and 5.8x less total energy than the most energy-efficient one.

Figures & tables

Explore similar work

CardsList
  1. Conjoint Audio-to-Spikes Encoding and Processing for Efficient Neuromorphic Speech Recognition

    Aug 31, 2026Valentin M. Meunier, Amélie Gruel, Pierre Lewden +2Automatic Speech RecognitionSpiking Neural Networks

  2. Assessing the Energy and Carbon Emissions of Neural Speaker Verification Model in Training and Inference

    Jun 6, 2026Hugo Leguillier, Driss Matrouf, Guillaume Lechien +1Speaker VerificationEnergy-Efficient ML

  3. NAVIR: Neuromorphic Audio-Visual Speech Recognition for Robust Human-Robot Interaction on Edge Hardware

    Sep 21, 2026Leonidas Delimpasis, Panagiota Moraiti, Antonis Porichis +2Visual Speaker RecognitionAutomatic Speech Recognition