OnDA: On-device Channel Pruning for Efficient Personalized Keyword Spotting

Matteo Risso; Alessio Burrello; Daniele Jahier Pagliari

eess.ASFeb 25, 2026

OnDA: On-device Channel Pruning for Efficient Personalized Keyword Spotting

Authors: Matteo Risso, Alessio Burrello, Daniele Jahier Pagliari

Organizations: Politecnico di Torino, Italy

Abstract

Always-on keyword spotting (KWS) demands on-device adaptation to cope with user- and environment-specific distribution shifts under tight latency and energy budgets. This paper proposes, for the first time, coupling weight adaptation (i.e., on-device training) with architectural adaptation, in the form of online structured channel pruning, for personalized on-device KWS. Starting from a state-of-the-art self-learning personalized KWS pipeline, we compare data-agnostic and data-aware pruning criteria applied on in-field pseudo-labelled user data. On the HeySnips and HeySnapdragon datasets, we achieve up to 9.63x model-size compression with respect to unpruned baselines at iso-task performance, measured as the accuracy at 0.5 false alarms per hour. When deploying our adaptation pipeline on a Jetson Orin Nano embedded GPU, we achieve up to 1.52x/1.57x and 1.64x/1.77x latency and energy-consumption improvements during online training/inference compared to weights-only adaptation.

Figures & tables

Figure 1: Baseline and OnDA pipelines steps.

Figure 2: Pareto fronts for personalized KWS under different OnDA pipelines compared with baseline [ 3 ] .

Figure 3: On-device latency measurements on Jetson Orin Nano for GPU and CPU deployment.

Compute unit	OnDA pipeline	Adapt. Lat./En.	Inf. Lat./En.
GPU	OnDA-1	1.52 $\times$ /1.64 $\times$	1.57 $\times$ /1.77 $\times$
GPU	OnDA-2	1.29 $\times$ /1.36 $\times$	1.91 $\times$ /2.55 $\times$
CPU	OnDA-1	1.86 $\times$ /1.94 $\times$	1.93 $\times$ /2.07 $\times$
CPU	OnDA-2	1.8 $\times$ /1.87 $\times$	2.34 $\times$ /2.53 $\times$

Table 1: Deployment relative gains on Jetson Orin Nano compute units. Improvements are reported with respect to [ 3 ] .

Neural Network Pruning

Neural network pruning removes weights, neurons, channels, heads, or layers to reduce model size and computation while retaining useful behavior.

See all 289 papers

Scalable Keyword Spotting via Modular Network Expansion
Jul 22, 2026Viktor Khaymonenko, Dzmitry Saladukha, Aliaksei Rak +1Class-Incremental Learning Efficient Neural Network Inference
Embedded Voice Input Team, Yandex, Russia · Embedded Voice Input Team, Yandex, Belarus
Sub-Model Short-Term Memory Convolutions for Keyword Spotting Systems on Device
Sep 28, 2026Paweł Warlewski, Artur Czeczko, Artur Szumaczuk +2Keyword Spotting Speech Processing
Samsung R&D Institute Poland · Samsung AI Center Warsaw
Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator
Jul 20, 2026Mateusz Piechocki, Alessandro Capotondi, Marek KraftOn-Device Training Edge ML
Poznan University of Technology, Institute of Robotics and Machine Intelligence, Poznan, Poland · University of Modena and Reggio Emilia, The High-Performance Real-Time Laboratory, Modena, Italy

Jul 22, 2026J/K move · Enter open · S save

Scalable Keyword Spotting via Modular Network Expansion

Viktor Khaymonenko, Dzmitry Saladukha, Aliaksei Rak, Alexander Rostov

Embedded Voice Input Team, Yandex, Russia · Embedded Voice Input Team, Yandex, Belarus

Class-Incremental Learning Efficient Neural Network Inference Keyword Spotting Tiny ML Modular Neural Networks

Open paper AskPDF arXiv

Keyword spotting (KWS) models on embedded devices often need to add new keywords after deployment, but updates are difficult when original training data are unavailable and regressions on existing triggers are unacceptable. At a fixed operating point, our method reduces average new-keyword false reject rate (FRR) from 6.46 to 4.37 versus a parameter-matched separate-model baseline and outperforms parameter-efficient tuning baselines (adapters, LoRA), while using fewer multiply-accumulate operations (MACs) under the same added-parameter budget (

\leq

10k): 16.34M vs 18.45M/20.52M. We achieve this via parameter-capped modular expansion: the base network, including batch-normalization statistics and the core classifier, is frozen, and only a lightweight expansion branch with a separate new-keyword head is trained, preserving core logits, shipped outputs, and thresholds for existing keywords.

Abstract

Neural Network Pruning

Explore similar work

Scalable Keyword Spotting via Modular Network Expansion

Sub-Model Short-Term Memory Convolutions for Keyword Spotting Systems on Device

Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator