Selective Attention System (SAS): Device-Addressed Speech Detection for Real-Time On-Device Voice AI

David Joohun Kim; Daniyal Anjum; Bonny Banerjee; Omar Abbasi

cs.SDApr 9, 2026

Selective Attention System (SAS): Device-Addressed Speech Detection for Real-Time On-Device Voice AI

Authors: David Joohun Kim, Daniyal Anjum, Bonny Banerjee, Omar Abbasi

Organizations: Attention Labs

Abstract

We study device-addressed speech detection under pre-ASR edge deployment constraints, where systems must decide whether to forward audio before transcription under strict latency and compute limits. We show that, in multi-speaker environments with temporally ambiguous utterances, this task is more effectively modelled as a sequential routing problem over interaction history than as an utterance-local classification task. We formalize this as Sequential Device-Addressed Routing (SDAR) and present the Selective Attention System (SAS), an on-device implementation that instantiates this formulation. On a held-out 60-hour multi-speaker English test set, the primary audio-only configuration achieves F1=0.86 (precision=0.89, recall=0.83); with an optional camera, audio+video fusion raises F1 to 0.95 (precision=0.97, recall=0.93). Removing causal interaction history (Stage~3) reduced F1 from 0.95 to 0.57+/-0.03 in the audio+video configuration under our evaluation protocol. Among the tested components, this was the largest observed ablation effect, indicating that short-horizon interaction history carries substantial decision-relevant information in the evaluated setting. SAS runs fully on-device on ARM Cortex-A class hardware (<150 ms latency, <20 MB footprint). All results are from internal evaluation on a proprietary dataset evaluated primarily in English; a 5-hour evaluation subset may be shared for independent verification (Section 8.8).

Figures & tables

Stage	Function	On-device
VAD	Speech presence detection	Yes
SAS	Addressee routing	Yes
ASR	Transcription	Optional
LLM / NLU	Intent and response	Optional
TTS / Output	Response rendering	Optional

Table 1: Voice AI pipeline stages. SAS occupies the pre-ASR routing layer, a position currently unaddressed by standard pipeline components.

Metric	No gate	With SAS	Saving
ASR calls / hr	100	9.3	90.7%
ASR CPU-time / hr	20 s	1.9 s	90.5%
LLM calls / hr	100	9.3	90.7%
LLM cost / hr	$1.00	$0.09	91.0%
False triggers / hr	92	1.9	97.9%
Missed genuine / hr	0	0.6	—

Table 2: Estimated pipeline savings at

\tau=0.70

(100 VAD segments/hr, 8% device-directed). Figures are illustrative.

Metric	Value
F1 (audio-only, primary)	0.86
Precision (audio-only)	0.89
Recall (audio-only)	0.83
F1 (audio+video)	0.95
Precision (A+V)	0.97
Recall (A+V)	0.93

Table 3: SAS evaluation summary at

\tau=0.70

. Precision, recall, and F1 are session-macro averages over VAD-positive segments. Audio-only is the primary deployment configuration; audio+video is an upper-bound configuration for devices with camera hardware.

	Pred. Sil.	Pred. Pers.	Pred. Dev.
True Silent	96.2	3.1	0.7
True Person-dir.	2.4	93.8	3.8
True Device-dir.	1.1	15.9	83.0

Table 4: Per-class confusion matrix at

\tau=0.70

(audio-only, primary deployment configuration). Rows are true labels; columns are predicted actions. Values are percentages of each true class.

Speakers present	Precision	Recall	F1
1	0.99	0.97	0.98
2	0.98	0.95	0.97
3	0.97	0.92	0.94
4	0.94	0.88	0.91
Condition mean	0.97	0.93	0.95

Table 5: Performance by number of speakers present at

\tau=0.70

( audio+video ).

Figure 1: Precision, F1, and recall as a function of number of speakers present at

\tau=0.70

. Shaded region shows the precision-recall gap, which widens with speaker count: cross-talk degrades recall faster than precision.

Figure 2: Precision-recall curve across operating thresholds

\tau\in[0.56,0.85]

. Filled square (

\blacksquare

\tau=0.70

(standard). Filled diamond (

\blacklozenge

\tau=0.82

(high-media). Shaded area: AP

\approx

0.88.

Figure 3: F1 heatmap: noise floor

\times

speakers present at

\tau=0.70

. Each cell reports macro-averaged F1 across held-out sessions under that condition.

Configuration	F1	$\Delta$ F1
Full model (SAS)	0.95	—
No Stage 1 (no beamforming)	0.81	$-$ 0.14
No Stage 2 (no classifier)	0.74 ( $\pm$ 0.02)	$-$ 0.21
No Stage 3 (no temporal ctx)	0.57 ( $\pm$ 0.03)	$-$ 0.38

Table 6: Stage ablation at

\tau=0.70

( audio+video configuration). One stage removed per row; all others held fixed.

Noise floor	1 spkr	2 spkr	3 spkr	4 spkr
Low (28–50 dBA)	+0.03	+0.05	+0.08	+0.10
Med (50–70 dBA)	+0.04	+0.07	+0.09	+0.12
High (70–85 dBA)	+0.06	+0.09	+0.11	+0.14

Table 7: Video modality contribution (

\Delta

F1 = A+V minus audio-only) at

\tau=0.70

, stratified by noise floor and speaker count. Audio-only baselines range from 0.96 (1-speaker, low-noise) to 0.74 (4-speaker, high-noise); see Figure 3 .

Figure 4: Ablation study: F1 when one stage is removed at

\tau=0.70

. Rows ordered by ascending impact. Error bars show the reported range for Stage 2 (0.74

\pm

0.02) and Stage 3 (0.57

\pm

0.03). Stage 3 (temporal context) is the dominant contributor; its removal reduces F1 by

-

0.38 (

\Delta

F1).

System	P	R	F1
VAD only (Silero)	$\approx$ 0.08	$\approx$ 0.99	$\approx$ 0.15
S1+S2 only (no ctx) a	0.55 ( $\pm$ .04)	0.55 ( $\pm$ .04)	0.57 ( $\pm$ .03)
SAS audio-only	0.89	0.83	0.86
SAS audio+video	0.97	0.93	0.95

Table 8: Comparison with routing baselines at

\tau=0.70

. The VAD-only row is a reference floor, not a competing system: VAD provides no addressee signal and is included only to illustrate the base-rate-driven precision bound. All other rows from the held-out set.

Property	Wake-word	SAS ( $\tau=0.70$ )
Trigger	Fixed phrase	Any device-directed speech
Latency	Low (processing)	$<$ 150 ms (end-to-end)
False pos.	Phonetic similarity	Tunable via $\tau$
Output	Confidence score	Confidence score
UX	Trigger required	Natural, unprompted speech
Footprint	Low	$<$ 20 MB, ARM Cortex-A

Table 9: Structural comparison with wake-word detection.

System	Position	ASR	Ctx	Params	HW
[ 11 ]	Post-ASR	Yes	No	16 M	GPU
[ 12 ]	Post-ASR	Yes	No	0.1–1.5 B	A100
[ 15 ]	Replaces	Yes	No	${\sim}$ 8 B	16 $\times$ A100
[ 16 ]	Post-VAD	No	No	${\sim}$ 5 M	On-device
SAS	Pre-ASR	No	Yes	${\sim}$ 520 K	ARM

Table 10: Structural comparison with published DDSD systems. Most listed systems exploit ASR transcripts or decoder features; SAS operates pre-ASR without transcript access. Metrics are not comparable across systems (see text).

Speech Processing

Speech processing develops computational methods for recognizing, synthesizing, analyzing, and transforming spoken audio.

See all 412 papers

Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference
Apr 16, 2026Nenad Banfic, David Fan, Kunal Vaishnavi +5Efficient Neural Network Inference Streaming ASR
CoreAI, Microsoft
ROAM-ASD: Robust Open-World Active Speaker Detection with Flexible Multimodal Fusion
Sep 22, 2026Pu Wang, Hugo Van hammeAudio-Visual Understanding Multimodal Robustness
KU Leuven, Department of Electrical Engineering, Leuven, Belgium
Efficient Streaming Audio-Visual Target Speaker Extraction for Real-World Acoustic Scenes
Oct 23, 2026Wendi Sang, Kai Li, Yifan Li +2Target Speaker Extraction Streaming Inference
School of Computer Technology and Application, Qinghai University, Xining 810016, China · Department of Computer Science and Technology, Institute for AI, BNRist, Tsinghua Laboratory of Brain and Intelligence (THBI), IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China · Intelligent Computing and Application Laboratory of Qinghai Province, Qinghai University, Xining 810016, China +1

Apr 16, 2026J/K move · Enter open · S save

Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference

Nenad Banfic, David Fan, Kunal Vaishnavi, Sam Kemp+4

CoreAI, Microsoft

Efficient Neural Network Inference Streaming ASR Post-Training Quantization On-Device Inference

Open paper AskPDF arXiv

Deploying high-quality automatic speech recognition (ASR) on edge devices requires models that jointly optimize accuracy, latency, and memory footprint while operating entirely on CPU without GPU acceleration. We conduct a systematic empirical study of state-of-the-art ASR architectures, encompassing encoder-decoder, transducer, and LLM-based paradigms, evaluated across batch, chunked, and streaming inference modes. Through a comprehensive benchmark of over 50 configurations spanning OpenAI Whisper, NVIDIA Nemotron, Parakeet TDT, Canary, Conformer Transducer, and Qwen3-ASR, we identify NVIDIA's Nemotron Speech Streaming as the strongest candidate for real-time English streaming on resource-constrained hardware. We then re-implement the complete streaming inference pipeline in ONNX Runtime and conduct a controlled evaluation of multiple post-training quantization strategies, including importance-weighted k-quant, mixed-precision schemes, and round-to-nearest quantization, combined with graph-level operator fusion. These optimizations reduce the model from 2.47 GB to as little as 0.67 GB while maintaining word error rate (WER) within 1% absolute of the full-precision PyTorch baseline. Our recommended configuration, the int4 k-quant variant, achieves 8.20% average streaming WER across eight standard benchmarks, running comfortably faster than real-time on CPU with 0.56 s algorithmic latency, establishing a new quality-efficiency Pareto point for on-device streaming ASR.

Abstract

Speech Processing

Explore similar work

Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference

ROAM-ASD: Robust Open-World Active Speaker Detection with Flexible Multimodal Fusion

Efficient Streaming Audio-Visual Target Speaker Extraction for Real-World Acoustic Scenes