Organizations: Xiangtan University, Xiangtan 411105, China · Beihang University, Beijing 100191, China · University of Alberta, Edmonton, AB T6G 2R3, Canada · Peking University, Beijing 100871, China · Macau University of Science and Technology, Macao SAR 999078, China · Swinburne University of Technology, Melbourne 3122, Australia
Existing phase-based backdoor attacks on speaker identification rely on shared absolute phase patterns for cross-sample trigger consistency, while other audio backdoor attacks have explored shared transformations that produce input-dependent triggers. Whether phase-based triggers can achieve similar consistency without imposing identical output phase patterns remains unclear. We propose GDBack, a poison-only group-delay backdoor attack using cascaded second-order analytic all-pass filters. Applying a shared transformation across utterances ideally preserves spectral magnitude while introducing a consistent frequency-dependent phase response and group-delay contribution, without forcing identical output phases. Experiments on LibriSpeech and VoxCeleb1 with ECAPA-TDNN, x-vector, and RawNet3 achieve attack success rates above 94% across all six settings (97.00% on average), while maintaining benign accuracy close to clean models. Phase and magnitude interventions, inverse compensation, and parameter-sensitivity analyses support the role of the induced phase transformation in backdoor activation. Speech quality and content evaluations indicate limited perceptual and linguistic distortion, while robustness experiments, including real over-the-air tests, reveal varying effectiveness across conditions and architectures. These findings demonstrate that cross-sample trigger consistency can arise from a shared phase transformation without requiring a common absolute phase pattern.
Figures & tables
Figure 1: Differences between phase-template backdoor attacks and GDBack . Existing phase-template attacks impose a common phase pattern on selected components of different utterances. In contrast, GDBack applies the same all-pass transformation to each utterance, introducing a shared group-delay contribution while yielding distinct output phase characteristics.
Figure 2: Overview of GDBack . The attacker applies the same analytic all-pass transformation to randomly selected training utterances and assigns them the target speaker label. The victim model is then trained using the normal training procedure. At inference time, benign utterances are classified normally, while applying the same transformation to a non-target utterance activates the learned backdoor association.
Model
Metric
Clean
General Audio Attacks
Phase-based Attacks
Ours
No Attack
PaddingBack
PBSM
SpeedMaster
ASMA
PhaseBack
StealthPhase
GDBack
ECAPA-TDNN
BA (%)
98.35
96.83
98.45
98.92
97.61
98.62
98.55
98.89
ASR (%)
–
82.97
99.97
99.93
97.83
99.12
73.87
99.70
x-vector
BA (%)
99.60
98.92
99.63
99.76
99.60
99.60
99.26
99.56
ASR (%)
–
72.17
100.00
99.63
84.53
96.75
87.71
98.71
RawNet3
BA (%)
99.09
99.12
99.46
99.26
99.53
99.66
99.36
99.33
Table I: Benign accuracy (BA) and attack success rate (ASR) of different attacks on LibriSpeech under a 5% poisoning rate.
Model
Metric
Clean
General Audio Attacks
Phase-based Attacks
Ours
No Attack
PaddingBack
PBSM
SpeedMaster
ASMA
PhaseBack
StealthPhase
GDBack
ECAPA-TDNN
BA (%)
75.09
71.51
75.76
73.65
74.69
72.86
67.19
74.24
ASR (%)
–
70.84
99.94
99.72
91.71
95.13
59.42
94.66
x-vector
BA (%)
76.73
73.74
75.79
76.70
73.30
75.38
72.20
78.27
ASR (%)
–
76.22
99.84
99.34
81.59
95.83
62.49
97.82
RawNet3
BA (%)
80.42
79.41
81.93
79.85
79.60
79.97
79.82
81.36
Table II: Benign accuracy (BA) and attack success rate (ASR) of different attacks on VoxCeleb1 under a 5% poisoning rate.
Figure 3: Mel spectrograms of the clean utterance and its poisoned versions generated by different backdoor attacks. All examples are derived from the same clean utterance for direct visual comparison.
Figure 4: Waveforms of the clean utterance and its poisoned versions generated by different backdoor attacks. All examples are derived from the same clean utterance for direct visual comparison.
Method
UTMOS
NISQA
SSL-MOS
Mean
>3 (%)
Mean
>3 (%)
Mean
>3 (%)
Clean
4.0782
99.40
4.0725
94.70
4.3083
99.50
PaddingBack
3.9869
99.20
4.0434
93.90
4.3065
99.40
PBSM
1.2867
0.00
1.8480
1.10
3.1362
44.90
SpeedMaster
3.5303
86.30
3.7297
90.50
4.1281
95.00
ASMA
3.6448
94.70
3.9672
92.70
4.2910
98.50
Table III: Objective speech quality of clean and poisoned speech. For each quality predictor, we report the mean score and the percentage of utterances with a score above 3. Higher values indicate better predicted speech quality.
Method
WER (%)
Δ WER (%)
CER (%)
Δ CER (%)
STOI
Clean
6.48
–
2.74
–
–
GDBack
6.73
+0.25
2.69
-0.05
0.9933
Table IV: Speech content preservation of GDBack .
Condition
Target Prediction Rate (%)
Clean input
0.20
Phase-only condition
99.80
Magnitude-only condition
0.30
Inverse compensation
0.10
Table V: Target-speaker prediction rates under controlled signal interventions on ECAPA-TDNN.
Center frequency
Pole radius
Cascade count
fc (Hz)
ASR (%)
r
ASR (%)
K
ASR (%)
1000
0.00
0.50
0.00
2
0.60
2000
0.00
0.70
1.90
4
16.60
3000
0.00
0.80
39.20
6
67.90
3500
0.90
0.85
85.30
8
94.40
4000
98.70
0.90
98.70
10
98.70
Table VI: Inference-time trigger parameter sensitivity of GDBack on LibriSpeech using a fixed backdoored ECAPA-TDNN model trained with r=0.90 , fc=4000 Hz, and K=10 .
Condition
BA (%)
ASR (%)
None
98.9
99.7
Volume ×0.8
98.7
99.7
Volume ×1.2
98.9
99.6
Time shift, 80 ms
98.5
99.8
Table VII: Performance of GDBack under amplitude scaling and temporal shifting.
Condition
BA (%)
ASR (%)
Reverberation
No reverberation
98.9
99.7
RT60=0.2 s
83.2
96.2
RT60=0.5 s
55.3
89.6
RT60=0.8 s
40.2
82.0
Background interference
Table VIII: Performance of GDBack under reverberation and background interference.
Figure 5: Effect of the poisoning rate on GDBack on LibriSpeech across three victim models.
Figure 6: Effect of the poisoning rate on GDBack on VoxCeleb1 across three victim models.
Target Index
Speaker ID
BA (%)
ASR (%)
1
26
98.82
95.57
5
40
98.89
99.70
10
89
99.06
99.02
20
211
99.12
98.10
30
307
99.02
98.41
Table IX: Sensitivity of GDBack to different target speakers on LibriSpeech using ECAPA-TDNN.
Figure 7: Sensitivity of GDBack to the all-pass trigger parameters on LibriSpeech using ECAPA-TDNN. For each parameter setting, a corresponding backdoored model is trained and evaluated using the matched trigger configuration. The dashed vertical lines indicate the default parameter values used in the main experiments.
Preprocessing
ECAPA-TDNN
x-vector
RawNet3
BA (%)
ASR (%)
BA (%)
ASR (%)
BA (%)
ASR (%)
None
98.89
99.70
99.56
98.71
99.33
94.89
Resample to 8 kHz
10.32
91.81
9.98
78.40
55.60
96.72
Low pass at 3.4 kHz
13.59
98.92
13.39
96.82
94.13
98.21
Phase jitter
98.85
99.66
99.56
98.65
99.22
94.85
Random APF
98.92
99.66
99.56
98.75
98.31
93.40
Table X: Effect of input preprocessing on GDBack on LibriSpeech.
Figure 8: Neural Cleanse detection results for GDBack . The actual target class obtains an anomaly index of 1.71, below the detection threshold of 2.
Figure 9: Effect of clean fine-tuning on GDBack using ECAPA-TDNN. The backdoored model is fine-tuned for 50 epochs using 5% of the clean training data.
Figure 10: Effect of model pruning on GDBack using ECAPA-TDNN. Neurons with lower average activation on a clean validation set are progressively pruned, and BA and ASR are reported under different pruning ratios.
Model
Digital ASR (%)
OTA ASR (%)
ECAPA-TDNN
100.00
96.50
x-vector
98.50
99.00
RawNet3
96.00
78.00
Table XI: Over-the-air performance of GDBack on LibriSpeech using 200 triggered utterances for each victim model.
Speech models are increasingly used as an interface to AI systems, from transcription tools to voice agents. Existing audio backdoors require an attacker to introduce triggers at inference time. We study a different threat model in which the attacker compromises the model during fine-tuning but has no access to the victim's inference-time inputs. We introduce the Speaker-Activated Backdoor (SAB) setting, in which a target speaker's natural voice serves as the trigger. We develop a two-phase fine-tuning method that first makes speaker identity more accessible and then binds the target speaker to an attacker-chosen payload. Compared with a one-phase baseline with the same training budget, the two-phase method substantially improves selectivity and clean performance, and is especially important for content-dependent attacks. SAB achieves 90.2% and 97.0% backdoor success on Whisper-medium and Parakeet, with false-positive rates of 1.7% and 0%, respectively using as little as two minutes of real audio from the target speaker. We further demonstrate content-dependent semantic rewriting and downstream tool-use attacks, where malicious instructions are inserted into 96.9% of target-speaker transcripts. When these transcripts are passed to frontier LLM agents, GPT-4o and Claude (Opus, Sonnet, and Haiku), the injected instruction drives the agent to carry out the harmful action in up to 87.5% of cases. The attack generalizes to held-out recordings, multiple target speakers, and common acoustic degradations. These results establish speaker identity as a practical backdoor trigger that requires no inference-time attacker access and is invisible at the transcript layer.
With the widespread deployment of deep-learning-based speech models in security-critical applications, backdoor attacks have emerged as a serious threat: an adversary who poisons a small fraction of training data can implant a hidden trigger that controls the model's output while preserving normal behavior on clean inputs. Existing inference-time defenses are not well suited to the audio domain, as they either rely on trigger over-robustness assumptions that fail on transformation-based and semantic triggers, or depend on properties specific to image or text modalities. In this paper, we propose STEP (Stability-based Trigger Exposure Profiling), a black-box, retraining-free backdoor detector that operates under hard-label-only access. Its core idea is to exploit a characteristic dual anomaly of backdoor triggers: anomalous label stability under semantic-breaking perturbations, and anomalous label fragility under semantic-preserving perturbations. STEP profiles each test sample with two complementary perturbation branches that target these two properties respectively, scores the resulting stability features with one-class anomaly detectors trained on benign references, and fuses the two scores via unsupervised weighting. Extensive experiments across seven backdoor attacks show that STEP achieves an average AUROC of 97.92% and EER of 4.54%, substantially outperforming state-of-the-art baselines, and generalizes across model architectures, speech tasks, an open-set verification scenario, and over-the-air physical-world settings.
Kun Wang, Meng Chen, Junhao Wang +6
Zhejiang University · Xi’an Jiaotong University · National University of Singapore
With the rapid development of deep learning, its vulnerability has gradually emerged in recent years. This work focuses on backdoor attacks on speech recognition systems. We adopt sounds that are ordinary in nature or in our daily life as triggers for natural backdoor attacks. We conduct experiments on two datasets and three models to validate the performance of natural backdoor attacks and explore the effects of poisoning rate, trigger duration and blend ratio on the performance of natural backdoor attacks. Our results show that natural backdoor attacks have a high attack success rate without compromising model performance on benign samples, even with short or low-amplitude triggers. It requires only 5% of poisoned samples to achieve a near 100% attack success rate. In addition, the backdoor will be automatically activated by the corresponding sound in nature, which is not easy to be detected and will bring severer harm.
Jinwen Xin, Xixiang Lyu, Jing Ma
School of Cyber Engineering, Xidian University, Xi’an, China