Bio-Inspired Auditory Processing: Coupling Parallel Resonate-and-Fire Dynamics with Ternary Recurrent Networks for Speech Recognition

Pavlos Poulos; Katelyn Sadorf; Themos Stafylakis; Costas A. Anastassiou

cs.SDOct 21, 2026

Bio-Inspired Auditory Processing: Coupling Parallel Resonate-and-Fire Dynamics with Ternary Recurrent Networks for Speech Recognition

Authors: Pavlos Poulos, Katelyn Sadorf, Themos Stafylakis, Costas A. Anastassiou

Organizations: Department of Informatics, Athens University of Economics and Business, Greece · Archimedes AI, Athena Research Center, Greece · California Institute of Technology, USA · Cedars-Sinai Medical Center, USA

Abstract

Spiking Neural Networks (SNNs) promise energy-efficient inference through sparse, event-driven computation, yet remain constrained on continuous, context-dependent tasks such as automatic speech recognition (ASR). A key bottleneck is the Leaky Integrate-and-Fire (LIF) neuron, whose membrane decay erases information across pauses and phonetic boundaries while it must both extract features and retain long-range context. Inspired by cortical separation of nonlinear filtering and somatic integration, we introduce the Spiking Ternary GRU (stGRU): a ternary-weight gated recurrent unit performs temporal filtering, while downstream Resonate-and-Fire (RF) neurons generate the binary spike stream driving the next block's event pathway. This makes binary event transmission a native training constraint rather than a separate deployment-time activation quantization step; together with ternary weights, learned projections avoid dense floating-point matrix MACs. Trained end-to-end on LibriSpeech-960, Small, Medium, and Large stGRUs reach 8.09%, 7.04%, and $6.44 \pm 0.17\%$ WER on test-clean without an external language model. To our knowledge, these are the first matrix-MAC-free spiking networks without a learned ANN sequence decoder to achieve single-digit WER on LibriSpeech. The architecture also achieves 10.88% test CER on Aishell-1 (9.28% on dev) when trained from scratch. Internally, trained networks develop a depth-graded hierarchy: shallow layers act as fast acoustic filters, while deep layers form slower semantic integrators whose latent representations segregate into part-of-speech manifolds and word-level spike events. Deep-layer firing rates follow lognormal distributions with a sparsity gradient, mirroring hierarchical processing observed in human cortex in vivo. Our biologically grounded architecture combines competitive ASR with favorable modeled compute and memory cost.

Figures & tables

Panel of Figure 1 — Figure 1: Acoustic alignment, internal population activity, and semantic manifold emergence in stGRU. (Top) Input acoustic waveform (LibriSpeech test utterance) aligned with ground-truth word boundaries (color-shaded spans) and internal spiking representations across physical time (top axis) and discrete network timesteps (bottom axis). The central heatmap displays the $Z$ -scored activity of 128 superneurons (co-active neural clusters sorted via Rastermap Stringer et al. (2025) ), with darker pixels indicating higher population firing. Red labels beneath the raster mark subword BPE token emissions predicted by the CTC readout layer (leading underscores denote new word onsets; consecutive identical tokens are collapsed into a single emission; CTC blanks omitted for clarity). (Bottom) 2D UMAP projections of latent representations across network depth (layers 0, 6, 10, and 14). Internal states progressively disentangle into distinct semantic and syntactic clusters, color-coded by part of speech: verbs (red), nouns (blue), and pronouns (green).

Panel of Figure 2 — Figure 2: Architecture and efficiency comparisons. (a) stGRU block architecture. (b) LibriSpeech test-clean WER versus estimated arithmetic energy over the 19,440 s evaluation set. (c) WER versus theoretical parameter storage in decimal MB. The Large stGRU WER is a six-seed mean; energy estimates correspond to the reported profiling run.

Architecture	Params (M)	Weight Type	Memory	Energy rate (mJ/s)	WER (%)
Architecture	Params (M)	Weight Type	(MB)	(mJ/s)	test-clean	test-other
Continuous Artificial Neural Networks (ANNs)
Moritz et al. Moritz et al. (2019)	70.0	FP32	$280.0^{\dagger}$	-	$5.4$	$16.1$
Park et al. Park et al. (2019)	360.0	FP32	$1440.0^{\dagger}$	-	$2.8$	$6.8$
Whisper-Small Radford et al. (2022)	244.0	FP16	$488.0^{\dagger}$	-	$3.1$	$7.4$
ResNet-BiGRU Reza et al. (2022)	33.0	FP32	$132.8^{\dagger}$	-	$15.6$	-

Table 1: Comprehensive Architectural Comparison on LibriSpeech-960 ASR. All reported WER results evaluate the standalone acoustic model without an external language model. The Large RF WER is mean

\pm

standard deviation over six seeds; other in-house rows and arithmetic-energy rates correspond to seed 3407.

Figure 3: Depth-graded coding emerges in trained Spiking-Ternary networks. Row 1: PSTH of word-selective neurons at shallow (L0–5), uncertainty (L11), and commitment (L14) depths. Each panel pair shows responses to a target word (“this,” blue) against acoustically similar competitors (gray), with the right sub-panel split by outcome (green: high-conf correct; orange: low-conf correct; red: incorrect). Row 2: Per-layer counts of outcome-selective neurons (matched-AUC, FDR $q<0.001$ ); correct-preferring neurons are sub-classified by high- vs. low-confidence via median CTC log-probability. Selectivity grows from L0–L5 to L11–L14 ( $<50$ vs. $>250$ neurons), and the bipartite partition reverses across depth: incorrect-preferring neurons peak at L11 ( $n=194$ ), high-confidence-correct-preferring at L14 ( $n=254$ ). Row 3: Population sparsity (per-word silent fraction) across encoder depth, with shaded IQR. Row 4: Population firing rate distributions for L9–L14 split by outcome (brackets: pairwise Mann–Whitney U-test with Bonferroni correction; ns : panels with at least one non-significant comparison). Row 5: Per-neuron firing rate histograms for L9–L14 with Gaussian fit overlaid.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Linguistic Stage	Channels	Kernel Size	Number of Layers
Linguistic Stage	Channels	Kernel Size	Small	Medium	Large
Stage 1 (Phonemes)	1024	11	1	2	3
Stage 2 (Syllables)	1024	21	1	2	3
Stage 3 (Words)	1024	39	1	2	3
Stage 4 (Sentences)	1024	51	1	2	3
Stage 5 (Extended Context)	1024	63	–	–	3

Appendix

Table S1: Detailed architectural configurations for the Small, Medium, and Large stGRU variants. The receptive field expands progressively across stages to capture hierarchical linguistic features.

Component	Hyperparameter	Value / Initialization
PRF Dynamics	Membrane Time Constant ( $\tau$ )	Target $1.0$ (with $\mathcal{N}(0,0.05)$ jitter on $\tau^{-1}$ )
	Resonant Frequency ( $\theta$ )	Linear spread $\in[0,\pi/2]$ + $\mathcal{N}(0,0.02)$ jitter
	Integration Step ( $dt$ )	Target $1.0$ + $\mathcal{N}(0,0.05)$ jitter
	Spike Threshold ( $V_{thr}$ )	Target $1.0$ + $\mathcal{N}(0,0.05)$ jitter
Optimization	SRCA Peak Learning Rate ( $\eta_{\max}$ )	$10^{-3}$
	SRCA Valley Learning Rate ( $\eta_{\min}$ )	$10^{-5}$

Appendix

Table S2: Key hyperparameters for PRF neuronal dynamics, optimization, tokenization, and CTC decoding .

Figure S1: Semi-Resonant Cosine Annealing (SRCA) learning rate schedule with initial holding period ( $S_{\text{hold}}=150$ ), followed by 2.75 cosine cycles with exponential peak decay ( $\gamma=0.75$ ) and initial phase offset ( $\phi_{\text{start}}=0.75$ ).

Continuous Artificial Neural Networks (ANNs)
Architecture	Estimated Arithmetic Energy (Joules over 19,440 s)				Estimated Arithmetic-Energy Rate (mJ/s)				Dense Matrix MACs
Architecture	Sub-Sampler	Encoder	Decoder	Total	Sub-Sampler	Encoder	Decoder	Total	Dense Matrix MACs
Transformer 112M Hou et al. (2024)	53.13	117.01	1.16	171.31	2.73	6.01	0.06	8.80	Yes
Transformer++ 112M Hou et al. (2024)	0.18	167.73	1.16	169.08	0.009	8.623	0.06	8.69	Yes
Conformer 136M Hou et al. (2024)	53.13	165.19	1.16	219.49	2.73	8.49	0.06	11.28	Yes
Hybrid Spiking Neural Networks (SNNs) & Dense ANN Decoder

Appendix

Table S3: Estimated arithmetic-energy decomposition on LibriSpeech test-clean (5.40 hours; 19,440 s). For the continuous ANN comparisons, the downstream column is a modeled FP32 CTC readout; for IML-Spikeformer it is a six-layer ANN decoder; for stGRU it is active-weight accumulation in an FP32 CTC readout driven by binary spikes; see Appendix D.1 ). Estimates omit normalization, activation functions, residual scaling, memory traffic, and decoding/search.

Dataset: LibriSpeech test-clean (5.40 Hours — 19,440 s)
Scale	Architecture Variant	Prec.	Memory (MB)	Estimated Arithmetic Energy (Joules)				Estimated Arithmetic-Energy Rate (mJ/s)
Scale	Architecture Variant	Prec.	Memory (MB)	Sub	Enc	Dec	Total	Sub	Enc	Dec	Total
Small	Linear $\rightarrow$ RF	FP32 ‡	70.79	24.62	0.22	0.82	25.66	1.26	0.01	0.04	1.31
	Conv1D $\rightarrow$ RF	FP32 ‡	71.28	26.25	4.72	0.86	31.83	1.35	0.24	0.04	1.63
	MLGRU $\rightarrow$ ReLU	FP32	513.32	338.47	75.48	11.51	425.46	17.40	3.88	0.59	21.87
	MLGRU $\rightarrow$ RF	INT2	70.98	24.02	9.20	0.87	34.09	1.23	0.47	0.04	1.74

Appendix

Table S4: Stage-by-Stage Dynamic Energy Profiling across Model Scales and Architectures for LibriSpeech and Aishell-1 .

Architecture	Params (M)	Weight Type	Memory	Energy rate	CER (%)
Architecture	Params (M)	Weight Type	(MB)	(mJ/s)	dev	test
Continuous Artificial Neural Networks (ANNs)
VGG-BiLSTM Ravanelli et al. (2022)	93.30	FP32	$373.0^{\dagger}$	–	9.7	10.7
Binary S4D ∗ Stan and Rhodes (2023)	21.00	FP32	$84.0^{\dagger}$	–	10.3	11.9
Transformer ∗ Stan and Rhodes (2023)	61.90	FP32	$247.6^{\dagger}$	–	5.1	5.6
Hybrid Spiking Neural Networks (SNNs) & Dense ANN Decoder/LM

Appendix

Table S5: Comprehensive Architectural Comparison on Aishell-1 Mandarin ASR. Evaluated on Character Error Rate (CER %). Memory sizes are theoretical footprints based on parameter counts and precision formats.

Test Set	4-gram LM	Seed 1	Seed 10	Seed 100	Seed 1000	Seed 10000	Seed 3407	Mean $\pm$ Std
test-clean	Without	6.53	6.60	6.33	6.46	6.58	6.14	$\mathbf{6.44\pm 0.17}$
test-clean	With	6.01	6.12	5.86	5.98	6.09	5.66	$5.95\pm 0.16$
test-other	Without	17.49	17.74	17.18	17.74	17.55	17.28	$\mathbf{17.49\pm 0.23}$
test-other	With	16.54	16.77	16.28	16.75	16.58	16.30	$16.53\pm 0.21$

Appendix

Table S6: Large Model WER (%) on LibriSpeech across 6 Random Seeds .

Scale	Architecture Variant	Precision	Params (M)	Memory (MB)	Error Rate (%)		Energy rate (mJ/s)
Scale	Architecture Variant	Precision	Params (M)	Memory (MB)	test-clean / Dev	test-other / Test	Energy rate (mJ/s)
Part I: LibriSpeech-960 Evaluation (WER %)
Small	Linear $\rightarrow$ RF	FP32 ‡	65.39	70.79	19.46	35.40	1.31
	Conv1D $\rightarrow$ RF	FP32 ‡	65.51	71.28	14.96	30.39	1.63
	MLGRU $\rightarrow$ ReLU	FP32	128.33	513.32	14.46	28.75	21.87
	MLGRU $\rightarrow$ RF	Ternary	128.35	70.98	15.95	30.87	1.74

Appendix

Table S7: Component-Wise Ablation: Model Scale, Architecture, and Weight Precision across LibriSpeech and Aishell-1 . Isolates individual contributions of low-precision weights, sequence gating, and discrete spiking activations. Reported numbers correspond to seed 3407.

Model	Scaling Mode	WER (%)		Sparsity (%)		Energy rate
Model	Scaling Mode	test-clean	test-other	Activation	Weight	(mJ/s)
Small	Naive ( $X(\widetilde{W}\gamma)^{\top}$ )	8.15	20.92	12.93	41.53	9.78
	Proposed ( $(X\widetilde{W}^{\top})\gamma$ )	8.09	20.79	12.84	41.76	1.95
	None ( $X\widetilde{W}^{\top}$ )	9.44	22.61	14.44	39.81	2.21
Large	Naive ( $X(\widetilde{W}\gamma)^{\top}$ )	6.44	17.43	17.36	38.57	17.74
	Proposed ( $(X\widetilde{W}^{\top})\gamma$ )	6.14	17.28	17.08	39.15	3.42

Appendix

Table S8: Placement of the weight scale in Small and Large stGRUs. All modes share input normalization. Naive folds

\gamma

into the weights; Proposed applies it after ternary accumulation; None omits it. Proposed has substantially lower modeled arithmetic-energy rate than Naive, while omitting

\gamma

worsens test-clean WER at both scales. Energy rates use the stated 45 nm operation costs and exclude an external language model.

Model Size	Neuron Type	Training Time (10 epochs)
Small	LIF	8h 53m
Small	PRF	2h 40m ( $3.3\times$ speedup)
Medium	LIF	14h 35m
Medium	PRF	4h 44m ( $3.1\times$ speedup)
Large	LIF	25h 00m
Large	PRF	7h 21m ( $3.4\times$ speedup)

Appendix

Table S9: Empirical Training Throughput for 10 Epochs on LibriSpeech-960 : PRF vs. LIF on 8 AMD Instinct MI250X GCDs.

Figure S2: Depth-dependent change in firing-rate fits (L9–L14). Top: log-axis firing-rate histograms with lognormal (red) and Gaussian (blue) fits. Bottom: $\Delta R^{2}=R^{2}_{\mathrm{ln}}-R^{2}_{\mathcal{N}}$ ; the lognormal fit has its largest advantage at L11 ( $+0.091$ ), while the Gaussian fit is slightly better at L14 ( $-0.011$ ).

Figure S3: Outcome-selective neuron counts (L9–L14) for full vs. attempted-emission-only trial pools, MLGRU vs. no-MLGRU (Large variants) . Selective neuron counts per layer, split by outcome preference (incorrect-preferring, red; correct-preferring, green). Left: Full MLGRU (Conv1D $\rightarrow$ MLGRU $\rightarrow$ RF). Empty-emission rate on incorrect words is 13.82%; restricting to attempted emissions reduces aggregate selective count by only 8.1%. Right: Ablated variant without MLGRU (Conv1D $\rightarrow$ RF). Empty-emission rate is 57.76%; restricting to attempted emissions collapses aggregate selective count by 95.5%, supporting a role for MLGRU gating for the emergent error-monitoring property.

Automatic Speech Recognition

Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.

See all 480 papers

Adaptive Speech-to-Spike Encoding for Spiking Neural Networks
Jun 17, 2026Taharim Rahman Anon, Jakaria Islam EmonKeyword Spotting Speech Processing
1PI LLC, Sapporo, Hokkaido, Japan
SpikCTU: A Compact Spiking Network with Adaptive Recurrent Computation for Speech Command Recognition
Oct 10, 2026Li Jin, Jiang Wenbin, Chen Tao +1Spiking Neural Networks Efficient Neural Network Inference
Hangzhou Dianzi University · Olympus Hedge Fund Investment Co., Ltd. · Shanghai Jiao Tong University
Spiking and Event-driven Neuromorphic Mamba Models for Efficient Speech Recognition
May 31, 2026Tauseef Ahmed, Tao Sun, Jeronimo Castrillon +2Efficient Neural Network Inference Mamba
Department of Advanced Computing Sciences, Maastricht University, Netherlands · Hardware-Efficient AI Team, IMEC, Netherlands · Chair for Compiler Construction, TU Dresden, Germany +1

Jun 17, 2026J/K move · Enter open · S save

Adaptive Speech-to-Spike Encoding for Spiking Neural Networks

Taharim Rahman Anon, Jakaria Islam Emon

1PI LLC, Sapporo, Hokkaido, Japan

Keyword Spotting Speech Processing Spiking Neural Networks Neuromorphic Computing Spike-Based Signal Encoding

Open paper AskPDF arXiv

The mismatch between continuous acoustic signals and discrete event-driven processing remains a fundamental bottleneck for neuromorphic speech processing. Current systems typically rely on fixed spike encoders, forcing downstream Spiking Neural Networks (SNNs) to compensate for non-adaptive input representations. To address this, we present a learnable residual speech-to-spike encoder jointly trained end-to-end with a Recurrent Leaky Integrate-and-Fire (R-LIF) backbone. We validate this approach on the Google Speech Commands v2 (GSC-v2) benchmark, achieving up to 94.97% accuracy. Notably, the learned encoder remains highly parameter-efficient with a compact 35k-parameter variant that reaches 89.8%, matching or exceeding prior baselines that require an order of magnitude more parameters. Our encoder-focused analysis, including linear probing and gradient-residual inspection, indicates that the encoder does not target faithful signal reconstruction but instead learns task-aligned spike representations that enhance class separability. Finally, we benchmark bio-inspired, hardware-friendly credit assignment by comparing Direct Feedback Alignment (DFA) with surrogate-gradient BPTT under identical architectures and training conditions. We find that DFA reaches 91.5% accuracy, quantifying the performance trade-off of bio-inspired learning rules for modern neuromorphic audio.

Abstract

Automatic Speech Recognition

Explore similar work

Adaptive Speech-to-Spike Encoding for Spiking Neural Networks

SpikCTU: A Compact Spiking Network with Adaptive Recurrent Computation for Speech Command Recognition

Spiking and Event-driven Neuromorphic Mamba Models for Efficient Speech Recognition