Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4

Billal Boumedine

eess.ASOct 28, 2026

Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4

Authors: Billal Boumedine

Organizations: Faculty of Computer Science, University of Science and Technology Houari Boumediene (USTHB), Algiers, Algeria

Abstract

We run Kroko, a 65.7M-parameter streaming Zipformer2 transducer for English, on the ESP32-P4, a microcontroller with two 400 MHz RISC-V cores, a 128-bit SIMD extension (PIE) and 32 MB of PSRAM, but no neural accelerator. Without any training, the encoder's 290 matrices are stored with 2 or 4 bits per weight in Hadamard-rotated groups: the width of each matrix comes from its measured sensitivity, and each matrix is first corrected toward the original model's output, then rounded with GPTQ and a search over the group norms. The released 71 MB of int8 weights become a 26.65 MB file. Activations are int16 between the matrix products and int8 into them. A C engine with hand-written PIE kernels processes each 1.28 s chunk of audio in 0.90 s on both cores, beam search included, and its PC build computes the same bits as the chip, so accuracy is measured on whole test sets. LibriSpeech WER is 4.34% on test-clean and 10.26% on test-other, against 3.94% and 9.17% for the released model; on FLEURS and Common Voice the gap grows to 2.3 and 1.1 points. A GPU simulator shows that the integer arithmetic costs nothing and that the whole gap comes from the weights, and an ablation on all of test-clean shows that GPTQ does most of the work. We also report the port's speed history, from 51.3 s to 0.62 s of encoder time per chunk, each design step measured again in one build, an attempt to reach 16 MB that stopped at 19.63 MB, and what did not work. We did not measure energy. The inference engine, the firmware and the 2/4-bit weights are available at https://github.com/BoumedineBillal/esp32-pie-asr.

Figures & tables

Stack	Layers	Width	Rate (Hz)	Heads	Largest FF	Conv kernel	Matrix params	At 4 bits
0	2	192	50	4	512	31	2.02M	69.5 %
1	2	256	25	4	768	31	3.78M	40.8 %
2	3	384	12.5	4	1024	15	11.60M	18.4 %
3	4	512	6.25	8	1536	15	30.22M	13.8 %
4	3	384	12.5	4	1024	15	11.60M	100 %
5	2	256	25	4	768	31	3.78M	100 %

Table 1: Kroko’s encoder stacks. Rate: output frames per second. The last column gives the share of each stack’s matrix parameters stored at 4 bits in our 2/4-bit file (Section 4 ).

Panel of Figure 1 — Figure 1: The pipeline on the chip. The six encoder stacks run at the frame rates shown, each with 5.12 s of left context.

Part	Parameters
4 bits: 193 encoder-path matrices	25.35M
2 bits: the other 97	38.37M
float16: biases, norms, depthwise convolutions, gates	1.1M
int8 (as released): decoder and joiner output	0.9M
File on the microSD card	26.65 MB

Table 2: What the weights file holds.

Variant	4-bit matrices	Size (MB)	Error $e$	WER
Released weights, unquantized, same simulator	–	–	0	4.02 %
Round to nearest, all at 4 bits	290	36.67	0.0136	4.58 %
Round to nearest, all at 2 bits (small ones at 4)	42	20.84	0.2194	21.76 %
Round to nearest, the 25 MB plan	193	27.07	0.0445	8.83 %
+ GPTQ, 128 clips	193	27.07	0.0163	4.75 %
+ correction toward the original	193	27.07	0.0128	4.68 %

Table 3: Ablation of the rounding on LibriSpeech test-clean (2620 clips, 53,029 words), float activations, greedy search, Whisper’s English text normalizer. Size: the .kq archive the chip’s file is made from (the chip’s file is 0.42 MB smaller). Error: relative error of the encoder output on 24 held-out FLEURS clips.

Clips	Activations	WER
73 (dev-clean)	float	7.27 %
	int16 everywhere, static scales	7.36 %
	int8 everywhere, static scales	12.75 %
test-clean, 500	float	3.33 %
	the chosen activations, float products	3.33 %
	the chosen activations, integer products	3.39 %

Table 4: The cost of integer activations. Every row uses the 2/4-bit weights; greedy search, Whisper’s normalizer. Float: float activations in the GPU simulator. The chip: the integer pipeline, through the PC build. The 73-clip sample is from LibriSpeech dev-clean.

Kernel	What it does
pq_decode2 , pq_decode4	weight codes to int8 levels through a gather table
pq_tile16 , pq_tile16_n	16 rows $\times$ one group $\times$ $T$ frames; the second also applies the norms and shifts
pq_fwht	the integer Hadamard transform of an input group (radix 8)
pq_q8_shift	int32 to int8 by a power-of-two shift
pq_out_tab16 , pq_out_add16 , pq_out_act8	read-out: activation table, residual add, int8 output
pq_sumsq16 , pq_scale_bypass16	BiasNorm’s sum of squares; its scale fused with the bypass

Table 5: The PIE kernels.

First port (float glue)		Integer port
Step	ms	Step	ms
plain float C	51,300	integer products	1,257
2/4-bit products on PIE	21,400	PIE int8 rounding	1,127
int8 front end and joiner, fast math	5,720	int8 caches	1,064
two cores	4,600	softmax in int8, once	983
key-major attention	2,750	int16 FF hidden layer on PIE	888

Table 6: The history of the port: encoder time per 1.28 s chunk on the chip after each step, as measured during development (ESP32-P4 v1.3 at 360 MHz, then the P4X at 400 MHz; greedy search).

	Design	Encoder	Chunk	$\times$ real time
a	int8 products on PIE, float in between (the first port)	1043.8	1176.9	0.92
b	integer products, int8 attention caches and softmax rows	776.5	909.7	0.71
c	+ int16 feed-forward hidden layer	721.4	854.2	0.67
d	+ int16 residual stream	700.7	834.4	0.65
e	+ integer convolution modules	645.9	778.8	0.61
f	+ integer non-linear attention	631.2	763.8	0.60

Table 7: The design steps measured again in one build on the P4X (400 MHz, two cores unless stated): ms per 1.28 s chunk, mean of 9 runs (standard deviation

\leq 0.5

ms). Chunk: features, encoder and beam search (4 hypotheses, no phrases). Rows b to h add one step each; i, j and k change one thing of h.

Panel of Figure 2 — Figure 2: Where a chunk’s time goes on the P4X, in ms (rows a, i and h of Table 7 ; core 0’s wall time per part; the totals are in the row labels). Feed-forward and convolution: those modules of the 16 layers. Attention: the attention weights, the non-linear attention and the self-attention. Search: the beam search with 4 hypotheses. The 2/4-bit products are counted inside the parts that use them.

	Words	Sentences	False alarms	WER
Phrases off	0 / 60	62 / 72	0 (0)	19.5 %
Phrases on (the chip)	46 / 60	70 / 72	3 (0)	3.4 %

Table 8: Phrase biasing on 140 text-to-speech clips (beam 4). Words: the list’s words written as listed. False alarms: a listed phrase written but not said, over all clips (in the 40 clips without listed phrases).

Set	Clips	greedy	beam 4	greedy	beam 4	Gap, beam 4
		Ours (the chip)		Released
LibriSpeech test-clean	2620	4.52	4.34	4.03	3.94	+0.40
LibriSpeech test-other	2939	10.89	10.26	–	9.17	+1.09
FLEURS English (US) test	647	17.75	15.95	14.92	13.68	+2.26 [1.63, 2.95]
Common Voice 17.0 English test	1000	20.72	19.56	18.85	18.47	+1.09 [0.50, 1.69]
FLEURS, scaled to peak 0.5	647	11.42	10.97	10.28	9.99	+0.97 [0.68, 1.28]

Table 9: WER on whole test sets, Whisper’s English normalizer. Ours: the PC build, which computes what the chip computes. Released: Kroko’s int8 model in sherpa-onnx. Gap: ours minus released at beam 4, with a 95 % interval from a paired bootstrap over clips where computed. The last row is a diagnostic: every FLEURS clip scaled to a peak of 0.5.

Part	Size
weights, used in place	26.65 MB
engine state and workspace	about 4 MB
screen frame buffer ( $1024\times 600$ , RGB565)	1.2 MB
free, at the lowest	about 0.5 MB

Table 10: PSRAM use while listening (32 MB).

System	Params	Weights	Runs on	Streaming	clean	other	RT
This work, beam 4	65.7M	2/4-bit, 26.65 MB	ESP32-P4, 2 cores	yes	4.34	10.26	0.70
This work, greedy	65.7M	2/4-bit, 26.65 MB	ESP32-P4, 2 cores	yes	4.52	10.89
Kroko as released, beam 4	65.7M	int8, 71 MB	PC	yes	3.94	9.17
Oído CTC, greedy [ 29 ]	13M	int8, 14.0 MB	ESP32-S3, 1 core	no	3.70	8.23	1.97
Oído streaming, greedy [ 29 ]	13M	int8	ESP32-S3, 1 core	yes	6.27	13.06
Oído streaming, with LM [ 29 ]	14.3M	int8	ESP32-S3, 1 core	yes	4.86	11.03	1.96

Table 11: Other small speech recognizers on LibriSpeech, as published by their authors (ours: measured here). WER in %. RT: processing time over audio duration; ours is the live firmware with its screen (0.59 for the engine alone, Table 7 ); Oído’s streaming RT is for 1.28 s chunks with its language model running. ESP-SR’s MultiNet [ 11 ] , which recognizes a fixed list of up to 200 commands on the ESP32-S3, does not transcribe open speech and is left out.

Automatic Speech Recognition

Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.

See all 484 papers

Oído: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural Accelerator
Oct 9, 2026Daniel Varela Sánchez, Guillermo Martín-Coello JuárezAutomatic Speech Recognition Streaming ASR
Lokutor (Hashing Works, S.L.L.), Madrid, Spain
Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference
Apr 16, 2026Nenad Banfic, David Fan, Kunal Vaishnavi +5Efficient Neural Network Inference Streaming ASR
CoreAI, Microsoft
Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR
Jun 23, 2026Nenad BanficStreaming ASR Low-Resource Speech Recognition
CoreAI, Microsoft

Oct 9, 2026J/K move · Enter open · S save

Oído: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural Accelerator

Daniel Varela Sánchez, Guillermo Martín-Coello Juárez

Lokutor (Hashing Works, S.L.L.), Madrid, Spain

Automatic Speech Recognition Streaming ASR

Open paper AskPDF arXiv

Speech recognition on low-cost microcontrollers is, in practice, limited to keyword spotting or a fixed list of commands. We present Oído, an inference engine that runs open-vocabulary English speech recognition entirely on an ESP32-S3 (240 MHz dual-core Xtensa LX7, 512 KB of SRAM, 8 MB of PSRAM, 16 MB of flash, no neural accelerator). Oído executes a 13M-parameter Conformer-CTC model with int8 weights, per-frame dynamic int8 activations and int8 attention on the chip's SIMD instructions, with a schedule designed around flash and PSRAM bandwidth. On LibriSpeech test-clean/test-other the on-device arithmetic reaches 3.70%/8.23% word error rate (WER), within 0.12 points of full precision, and 3.33%/7.15% with an on-chip 1.3M-parameter language model; a transducer head gives 2.99%/6.67% and an int4 variant fits in 8.3 MB. This is 2.3-2.8x fewer errors than the vendor's recognizer on the same chip, and fewer than Whisper tiny.en and Moonshine tiny in full precision on a laptop, including in real noise and reverberation. Measured on a physical board, the firmware runs at 1.97x the duration of the audio on one core, with transcripts identical to an instruction-exact emulator: accurate, but not yet real time (a 4 s command is transcribed about 8.7 s after it ends). The two-core schedule gives wrong results on silicon and is disabled. A streaming variant shows partial text while the speaker talks and delivers the final text 4.1-11.1 s after clips of 3.6-10.6 s (7.2-22.4 s in utterance mode) at 4.86%/11.03% WER; a Spanish model makes 34-60% fewer errors than multilingual Whisper tiny. Code and models are released.

Abstract

Automatic Speech Recognition

Explore similar work

Oído: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural Accelerator

Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference

Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR