Self-supervised speech models such as WavLM achieve strong spoofing detection performance, yet which layers encode the most spoof-relevant information and what acoustic properties they capture remain opaque. We present a layer-wise interpretability study of WavLM using the ASVspoof 5 database, combining three complementary analyses: (i) per-layer linear probing with attack-level evaluation across speech synthesis, voice conversion, and adversarial attacks; (ii) a canonical correlation analysis (CCA) linking layer representations to spoofing-relevant acoustic features; (iii) an exhaustive grid search over layer subsets showing that use of five layers outperforms full thirteen-layer pooling. CCA analysis reveals that, while most acoustic features peak in alignment with mid-layers and then decay, voice quality measures jitter, shimmer, and the harmonics-to-noise ratio show the opposite trend, with correlation increasing toward later layers. Embedding visualisations reveal that deeper layers capture a continuous acoustic similarity among attacks rather than discrete categories, suggesting these layers encode fundamental synthesis artifacts rather than attack-specific signatures.
Figures & tables
Fig. 1: Overview of the analysis pipeline . Top: WavLM produces frame‑level layer representations; each layer is summarised by mean+std pooling to a 1536‑D utterance vector. Bottom left (blue): PWCCA pipeline, PCA (1536→50) and correlation between pooled layer vectors and handcrafted acoustic features, producing layer vs feature heatmaps. Bottom right (pink): per‑layer linear probing, yielding per‑layer EER and per‑attack EER heatmaps.
Fig. 2: Per-attack EER (%) across WavLM layers for the dev (A09–A16) and eval (A17–A32) splits, with overall per-layer EER for each set shown in the two rightmost columns. Green indicates low EER (easy detection), whereas red indicates high EER (hard detection). The vertical black line separates dev and eval attacks.
Fig. 3: PWCCA correlation between WavLM layers and individual acoustic features. Most features peak at mid-layers and decay, while voice quality measures (jitter, shimmer, HNR) increase with depth.
Layers
k
Dev EER
Eval EER
7–9–10–11–12
5
2.06%
6.1%
6–8–9–10–12
5
2.16%
7.16%
6–9–10–11–12
5
2.02%
7.25%
All (0–12)
13
4.59%
8.08%
TABLE I: Top layer subsets from grid search vs. all-layer pooling.
Fig. 4: t-SNE visualisation of WavLM representations at layers 1, 6, and 11. Representations transition from overlapping clusters (L1) to discrete attack-specific groupings (L6) to a continuous horseshoe manifold (L11).
Self-supervised learning (SSL) countermeasures (CMs) have shown strong performance in recent years. However, they often show degraded performance while facing unseen spoofing attacks and mismatched conditions. This study examines the Voxtral audio-language model (ALM) framework for spoofing detection, as a step toward combining CM capabilities within the ALM framework. We analyze how Voxtral captures spoofing cues through audio-text processing and propose an instruction-guided approach that uses label-sequence likelihoods to evaluate bonafide and spoofed speech. Experiments on the ASVspoof databases show that without task-specific adaptation, the LLM layers emphasize semantic representations, reducing the separability of spoof-discriminative acoustic cues compared to the Whisper-based audio encoder. Consequently, spoofing-related information becomes less separable after language-model processing. We also applied lightweight adaptation using weight-decomposed low-rank adaptation (DoRA) to the Voxtral model and propose the Spooftral model, achieving an equal error rate (EER) of 4.25% on the ASVspoof5 evaluation set.
School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Israel · Department of Electrical Engineering, Afeka the Academic College of Engineering, Israel · Avignon University, LIA, France
Recent advances in text-to-speech and voice cloning make high-quality spoofing inexpensive and scalable, threatening voice authentication systems, especially automatic speaker verification (ASV). Existing defenses mainly address this threat through binary countermeasures (CMs) for deepfake detection or spoofing-aware speaker verification (SASV), where current systems are dominated by modular ASV-CM fusion and cascaded pipelines. Although large audio language models (LALMs) have shown promise on related audio tasks, including CM and ASV, their use for SASV remains unexplored, despite their capacity to produce natural-language rationales for auditing and robustness beyond discriminative predictions. This work systematically evaluates LALMs for SASV against conventional pipelines under zero-shot prompting, supervised adaptation, reasoning-oriented training, and reinforcement-learning-based optimization. Our results show that pretrained LALMs are near chance in the zero-shot setting, confirming that they are not natively suited to SASV, but that task-specific adaptation closes this gap. We further find that competitive SASV performance can be achieved through several distinct routes. These findings position LALMs as a promising and auditable foundation for unified SASV, while clarifying where conventional cascade systems still lead.
As the accuracy of speech deepfake detection improves with the use of self-supervised representations such as wav2vec 2.0 and HuBERT, understanding why the speech is classified as bona fide or deepfake remains an open challenge. In pursuit of more trustworthy and interpretable artificial intelligence, we introduce a phoneme-level analysis framework that connects model predictions to phonetic units. Our post-hoc explainability method is generally applicable to a variety of speech deepfake detection systems based on convolutional neural networks since it leverages Gradient-weighted Class Activation Mapping in conjunction with speech recognition to generate saliency maps aligned with phonemes and pauses. This pipeline reveals statistically significant attack- and speaker-dependent phonetic cues associated with spoofed speech in terms that humans can understand. Experiments using ASVspoof 5 show comparable detection performance to similar architectures while providing linguistic interpretations across speakers and spoofing conditions.
Anna Taylor, Michele Panariello, Massimiliano Todisco +3
EURECOM, Sophia Antipolis, France · Laboratoire Informatique d’Avignon, Avignon Université, France