Aggregate accuracy conceals concentrated temporal vulnerability in a spiking speech classifier

İsmail Can Dikmen

cs.NEOct 2, 2026

Aggregate accuracy conceals concentrated temporal vulnerability in a spiking speech classifier

Authors: İsmail Can Dikmen

Organizations: Department of Electrical and Electronics Engineering, İstinye University, Sarıyer, İstanbul, 34396, Türkiye

Abstract

Aggregate accuracy cannot reveal which utterances are locally vulnerable or how internal activity changes when labels remain stable. We retain every prediction for 725,070 adjacent-bin, one-count changes around 100 validation utterances of a frozen SpikeSCR-based classifier. The canonical native-horizon GPU singleton path reaches 86.0836% validation accuracy. Equal-source expected accuracy under a uniformly chosen neighbor rises from 84.00% to 84.54%, although 13 of 84 initially correct sources admit adverse neighbors. Five sources carry 93.21% of adverse moves. Margin-guided and surrogate-gradient searches miss sparse cases at fixed query budgets; a post hoc gradient prefix finds all 13 at 73.45% of the census of initially correct sources. Source-matched traces and clean-state interventions distinguish internal change from harmful direction and recoverability. Two count-readout replicas have similar validation accuracies but a 5.21-fold class-change gap concentrated in four sources. Controlled query/key source isolation eliminates 531 batch-order label changes and restores bitwise order invariance across all 9,981 validation score vectors. Isolated batch predictions reproduce padding-matched singleton labels on 9,980 of 9,981 inputs. Paired CPU/GPU replay of all 25,820 class-changing neighbors gives 99.8993% label agreement and preserves all 13 vulnerable sources and their fixed witnesses. Complete source-conditioned maps distinguish vulnerability incidence, concentration, internal change, and execution dependence that aggregate accuracy leaves unresolved.

Figures & tables

Research line	Main question	Typical output	Difference from this study
Optimized SNN attacks ( Liang et al., 2023 ; Büchel et al., 2022 ; Lun et al., 2025 ; Y. Yu et al., 2026 )	Can an optimizer find a harmful perturbation?	Attack success, robust accuracy, or a selected adversarial example	Does not retain every outcome of our collision-permitting one-count SSC neighborhood.
Formal or exact search ( Banerjee et al., 2023 ; Seong et al., 2024 ; Mukhoty et al., 2024 )	Does any bounded counterexample exist?	SAT/UNSAT, certificate, witness, and runtime	Direct search may stop at a witness; our output is a full transition-aware local decision map, not a global certificate.
Speech timing studies ( Z. Yu et al., 2026 )	Does timing carry useful information, and how does aggregate accuracy change under temporal disruption?	Dataset-level accuracy curves under replacement, jitter, deletion, or reversal	Does not enumerate and retain every adjacent-bin neighbor for each audited source.
SNN explanation ( Kim & Panda, 2021 )	Where and when is internal spiking activity salient?	Observational heatmaps	Does not perform matched internal-state replacement under retiming.
Internal interventions outside SNNs ( Vig et al., 2020 ; Meng et al., 2022 ; Zhang & Nanda, 2024 )	How do interventions on internal representations affect model outputs?	Mediation effects, clean-state restoration, and sensitivity to patching choices	Provides methodological precedents for internal intervention; the present audit applies matched replacement to retimed spiking speech inputs.
This study	What is the complete local outcome map, how does activity change, and which clean boundaries are sufficient for recovery?	Every prediction and transition type, source vulnerability, informed-search coverage, internal trace, and matched replacement outcome	Retains the complete declared neighborhood and source-conditioned outcome distributions.

Table 1: Relationship to the closest research lines. "Complete" in the final row means every member of the stated finite neighborhood is retained.

Figure 1: Study workflow and separation of data roles. The frozen SpikeSCR checkpoint was evaluated once on the official SSC test split for benchmark measurement. The temporal audit used 100 fixed validation utterances. One integrated count unit was transferred to an adjacent 5 ms bin within the same transformed feature, and every valid candidate was evaluated with native-horizon CUDA singleton inference. For each candidate, one count unit was moved to either the preceding or the following bin, provided that the target remained within the source horizon. The resulting candidates were used for the decision map and the internal trace and replacement analyses. Completeness applies only to the declared finite neighborhood, frozen panel, checkpoint, and CUDA singleton implementation.

Split	Correct / $n$	Accuracy	Balanced	Macro-F1
Validation, batch 256	8,617 / 9,981	86.3340%	85.2682%	85.7325%
Validation, GPU singleton (audit path)	8,592 / 9,981	86.0836%	84.8246%	85.4290%
Validation, CPU singleton	8,591 / 9,981	86.0735%	84.8030%	85.4092%
Official test, batch 256	17,247 / 20,382	84.6188%	83.7526%	84.0844%

Table 2: Performance of the frozen SpikeSCR-based checkpoint under separate execution contracts. The test split was evaluated once on the archived batch-256 path; no singleton test evaluation was made.

	Unique		Event-weighted
Outcome	$n$	%	$n$	%
Class-preserved	699,250	96.4390	1,601,188	96.5061
Any class change	25,820	3.5610	57,970	3.4939
Adverse	5,157	0.7112	10,424	0.6283
Corrective	9,031	1.2455	20,106	1.2118
Lateral	11,632	1.6043	27,440	1.6539

Table 3: Complete outcome partition for the specified adjacent-bin neighborhood. Percentages use the corresponding unique or event-weighted denominator.

Figure 2: Complete outcome partition of the declared adjacent-bin neighborhood. Panel A shows the share of class-preserved and class-changing candidates for unique-candidate and event-weighted accounting. Panel B decomposes only the class-changing candidates into adverse, corrective, and lateral outcomes. Exact counts and percentages are given in Table 3 .

Figure 3: Source-level local fragility. Panel A relates the clean top-two margin (summed softmax probability mass across each source’s horizon, Eq. ( 5 )) to the adverse-candidate rate for the 84 initially correct sources. Red markers identify the 13 sources with at least one adverse neighbor. The low-margin-quartile membership is unchanged when the margin is divided by horizon. Panel B ranks adverse-sensitive sources by adverse count and gives cumulative load. Two sources account for 68.47% of adverse candidates, five for 93.21%, and eight for 99.20%.

Candidate queries	Uniform E	Lowest 10% E	Lowest 25% E	Lowest 50% E	50/50 E	Runner-up D	All-target gain D	All-target margin D
840	3.71	5.21	6.03	4.84	5.22	4	4	4
8,400	7.61	6.25	9.42	8.62	8.91	7	7	8
84,000	10.30	11.28	11.05	10.54	10.75	11	10	11

Table 4: Discovery among the 13 vulnerable sources within the same 84 singleton-correct sources at fixed total candidate-query budgets. E denotes exact expected discovery for uniform-within-source sampling; D denotes actual discovery by deterministic gradient rankings. Focus thresholds select the 9, 21, and 42 lowest raw-margin sources. The 50/50 allocation mixes uniform and quartile focus. Runner-up uses one backward per source; both all-target rankings use the same 34 backwards per source. Candidate-query budgets exclude this additional work.

Outcome	Candidates	Sources	Pooled median [Q1, Q3]	Equal-source mean
Class-preserved	699,250	100	0.6464 [0.6130, 0.6726]	0.5719
Adverse	5,157	13	0.7039 [0.6822, 0.7191]	0.7007
Corrective	9,031	9	0.6635 [0.6464, 0.6793]	0.6751
Lateral	11,632	8	0.6567 [0.6397, 0.6774]	0.6683

Table 5: Descriptive final-block relative

L_{2}

activation changes calculated from the stored canonical trace records. All candidates, including exact-zero changes, are retained. Brackets contain first and third quartiles, not confidence intervals. The equal-source mean includes only sources containing the specified outcome.

Boundary	Adverse	Corrective	Lateral
Stem	0.0168 (0.0018)	0.0200 (0.0230)	0.0098 (0.1749)
Attention 1	0.1815 (0.0014)	0.1562 (0.0072)	0.1535 (0.0182)
Local 1	0.1246 (0.0014)	0.1179 (0.0072)	0.1408 (0.0092)
Block 1	0.0844 (0.0014)	0.0758 (0.0072)	0.0827 (0.0113)
Attention 2	0.1528 (0.0014)	0.1583 (0.0081)	0.2164 (0.0092)
Local 2	0.1356 (0.0014)	0.1436 (0.0072)	0.1941 (0.0102)

Table 6: Source-matched transition-minus-preserved contrasts in relative

L_{2}

activation change. Each cell gives the mean contrast above its adjusted

q

value in parentheses. Source counts are 13 adverse, 9 corrective, and 8 lateral.

Candidates	Score $L_{2}$	Absolute margin	Signed margin
All neighbors	0.4410	0.3447	0.0032
Nonzero internal response	0.2212	0.0544	0.0078
Preserved, nonzero response	0.2206	0.0566	0.0138

Table 7: Median within-source Spearman correlations between final-block relative

L_{2}

change and output changes. Each row summarizes 100 sources; values are descriptive medians.

Patched boundary	Candidate-pooled rate	Equal-source mean [95% interval]
Attention 1	77.18%	92.93% [86.45, 98.24]
Local 1	71.22%	92.31% [83.60, 99.29]
Attention 2	77.64%	91.26% [83.56, 97.82]
Local 2	41.17%	78.51% [61.79, 92.76]

Table 8: Restoration of the clean, correct prediction after matched clean-activation replacement in adverse candidates. Equal-source intervals use 10,000 source-bootstrap samples.

Figure 4: Internal divergence and matched clean-activation replacement. Panel A shows source-matched transition-minus-preserved contrasts in relative

L_{2}

activation change across seven internal boundaries. The lateral contrast at the stem is the only plotted transition contrast that did not survive false-discovery-rate correction. Panel B shows restoration of the clean, correct prediction for adverse candidates after replacing the candidate activation with the matched clean activation at the four branch boundaries. Error bars show the 95% source-bootstrap intervals of the equal-source means. The candidate-pooled rates are shown separately because adverse candidates are strongly concentrated by source.

Outcome	Replica A	Replica B
T0: trace unchanged	174,198	168,422
T1: trace changed, counts preserved	97,293	77,753
T2: counts changed, class preserved	1,022,650	1,032,964
T3: class changed	3,564	18,566
Final-layer trace changed, counts preserved	89,176	72,380
Adverse neighbors (subset of T3)	790	15,773

Table 9: Complete outcomes in two separately seeded count-readout SNN replicas. Each column contains 1,297,705 unique neighbors of the same 100 sources. The final-layer row is a subset of T1, adverse is a subset of T3, and a robust source is clean-correct for every allowed neighbor.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Condition A $\to$ condition B	Accuracy A $\to$ B (%)	Changed labels	C $\to$ W	W $\to$ C	W $\neq$ W
CPU singleton $\to$ native GPU singleton	86.0735 $\to$ 86.0836	2 (0.02%)	0	1	1
Native $\to$ padding-matched GPU singleton	86.0836 $\to$ 86.0435	344 (3.45%)	110	106	128
Padding-matched GPU singleton $\to$ GPU batch 256	86.0435 $\to$ 86.3340	614 (6.15%)	191	220	203
Native GPU singleton $\to$ GPU batch 256	86.0836 $\to$ 86.3340	619 (6.20%)	199	224	196
GPU batch 256 $\to$ reversed GPU batch 256	86.3340 $\to$ 85.8631	531 (5.32%)	205	158	168
Padding-matched singleton $\to$ q/k-isolated batch 256	86.0435 $\to$ 86.0535	1 (0.01%)	0	1	0

Appendix

Table A.1: Paired execution controls on all 9,981 frozen validation sources. Each row compares condition A with condition B. C

\to

W counts correct-to-wrong predictions; W

\to

C counts wrong-to-correct predictions; W

\neq

W counts cases wrong in both conditions with different predicted classes. Changed labels equal the sum of these three counts. The q/k-isolated row uses the source-isolation intervention defined below. The last row checks the archived batch-256 labels against their fresh replay.

Added bins	Sources	Changed	Rate (%)
0	4,057	0	0.00
1 - 5	5,004	236	4.72
6 - 10	45	2	4.44
11 - 20	155	13	8.39
21 - 40	294	26	8.84
41 - 80	322	43	13.35

Appendix

Table A.2: Native versus padding-matched GPU singleton labels over all 9,981 validation sources, grouped by the number of added 5 ms bins.

Canonical type	Candidates	CPU: same type	CPU: same class
Adverse	5,157	5,152	5,151
Corrective	9,031	9,022	9,022
Lateral	11,632	11,621	11,621
Total	25,820	25,795	25,794

Appendix

Table A.3: CPU replay of every originally class-changing candidate. GPU retains all canonical types and predicted classes. “Same type” means adverse, corrective, or lateral remains the same under the device-specific clean prediction; “same class” means exact agreement with the canonical candidate label.

Adversarial Robustness

Adversarial robustness studies how well AI systems withstand deliberately crafted inputs, training-data corruption, and other attacks.

See all 481 papers

Jun 17, 2026

Adaptive Speech-to-Spike Encoding for Spiking Neural Networks

The mismatch between continuous acoustic signals and discrete event-driven processing remains a fundamental bottleneck for neuromorphic speech processing. Current systems typically rely on fixed spike encoders, forcing downstream Spiking Neural Networks (SNNs) to compensate for non-adaptive input representations. To address this, we present a learnable residual speech-to-spike encoder jointly trained end-to-end with a Recurrent Leaky Integrate-and-Fire (R-LIF) backbone. We validate this approach on the Google Speech Commands v2 (GSC-v2) benchmark, achieving up to 94.97% accuracy. Notably, the learned encoder remains highly parameter-efficient with a compact 35k-parameter variant that reaches 89.8%, matching or exceeding prior baselines that require an order of magnitude more parameters. Our encoder-focused analysis, including linear probing and gradient-residual inspection, indicates that the encoder does not target faithful signal reconstruction but instead learns task-aligned spike representations that enhance class separability. Finally, we benchmark bio-inspired, hardware-friendly credit assignment by comparing Direct Feedback Alignment (DFA) with surrogate-gradient BPTT under identical architectures and training conditions. We find that DFA reaches 91.5% accuracy, quantifying the performance trade-off of bio-inspired learning rules for modern neuromorphic audio.

Taharim Rahman Anon, Jakaria Islam Emon

1PI LLC, Sapporo, Hokkaido, Japan

Keyword Spotting Speech Processing Spiking Neural Networks Neuromorphic Computing Spike-Based Signal Encoding

Ask PDF

Jun 26, 2026

What Was That Again? Certified Robustness for Automatic Speech Recognition

Automatic Speech Recognition systems are notoriously both sensitive to adversarial and benign perturbations. While this has been repeatedly demonstrated using reference datasets, detecting such behaviors in deployed systems is incredibly challenging, due to the absence of oracle knowledge of the true transcription. We demonstrate that employing a certification-inspired mechanism can significantly decrease WER, increase recall, and decrease the Spearman correlation between confidence and WER. We achieve this through a dual-gate diagnostic pipeline: a Two-Sided Atomic Audit that accumulates statistical wealth to certify both token existence and adversarial exclusion, and a Rank-Based Tournament that selects the winning sequence. Our evaluations across four diverse architectures demonstrate up to a 55% relative reduction in Word Error Rate, while also providing granular word- and sentence-level certifications to enhance acoustic security.

Andrew C. Cullen, Neil G. Marchant, Jiani Xie +2

University of Melbourne · DST Group, Adelaide

ASR Evaluation Neural Network Robustness Automatic Speech Recognition Certified Robustness Adversarial Example Detection Robust ASR

Ask PDF

Feb 28, 2026

ACES: Accent Subspaces for Coupling, Explanations, and Stress-Testing in Automatic Speech Recognition

ASR systems exhibit persistent performance disparities across accents, but whether these gaps reflect superficial biases or deep structural vulnerabilities remains unclear. We introduce ACES, a three-stage audit that extracts accent-discriminative subspaces from ASR representations, constrains adversarial attacks to them, and tests whether removing them improves fairness. On Wav2Vec2-base with seven accents, imperceptible perturbations (~60 dB SNR) along the accent subspace amplify the WER disparity gap by nearly 50% (21.3->31.8 pp), exceeding random-subspace controls; a permuted-label test confirms specificity to genuine accent structure. Partially removing the subspace worsens both WER and disparity, revealing that accent-discriminative and recognition-critical features are deeply entangled. ACES thus positions accent subspaces as powerful fairness-auditing tools, not simple erasure levers.

Swapnil Parekh

Intuit

Demographic Bias in ASR Automatic Speech Recognition

Ask PDF

Abstract

Adversarial Robustness

Explore similar work

Adaptive Speech-to-Spike Encoding for Spiking Neural Networks

What Was That Again? Certified Robustness for Automatic Speech Recognition

ACES: Accent Subspaces for Coupling, Explanations, and Stress-Testing in Automatic Speech Recognition