When and why do handcrafted cues help self-supervised anti-spoofing? A causal and faithfulness analysis

Yugwon Won

eess.ASJul 5, 2026

When and why do handcrafted cues help self-supervised anti-spoofing? A causal and faithfulness analysis

Authors: Yugwon Won

Abstract

Most spoofing countermeasures now place a light classifier on top of a self-supervised (SSL) speech encoder. A growing line of work adds handcrafted acoustic features and fuses them by cross-attention, partly because the attention weights appear to explain which cues the model relies on. Two things about such fusion remain untested: whether the handcrafted features contribute anything once a strong SSL encoder is already in place, and whether the attention map faithfully reflects what the classifier actually uses. We study both questions with MOSAIC, a single model that handles logical access (LA) and physical access (PA) attacks by projecting a 152-dimensional biophonetic vector into six query tokens attending over thirteen intermediate layers of WavLM-Large. Rather than pursuing state-of-the-art accuracy, we treat the model itself as the object of analysis and apply inference-time interventions that zero out selected components. Removing the handcrafted branch lowers the EER by 0.67 percentage points under LA, so WavLM alone is sufficient there, but raises it by 0.76 points under PA, where the branch supplies replay channel evidence the encoder does not carry. The 6x13 attention map is highly stable under resampling (bootstrap rank correlation 0.99), yet its faithfulness varies by domain: layer attention weights predict the causal importance of each layer under PA (rho = +0.62) but not under LA (rho = -0.28). The handcrafted branch and its attention-based explanation are therefore trustworthy for replay attacks and not for synthetic ones. On standard benchmarks the model is mid-range on LA and ahead of the official baselines on cross-source deepfakes (6.21% EER on ASVspoof 2021 DF). The contribution is not a new architecture but a checking procedure: verify handcrafted cues and attention maps by intervention before trusting either.

Explore similar work

Jul 16, 2026cs.SD

Large Audio Language Models for Spoofing-Aware Speaker Verification

Recent advances in text-to-speech and voice cloning make high-quality spoofing inexpensive and scalable, threatening voice authentication systems, especially automatic speaker verification (ASV). Existing defenses mainly address this threat through binary countermeasures (CMs) for deepfake detection or spoofing-aware speaker verification (SASV), where current systems are dominated by modular ASV-CM fusion and cascaded pipelines. Although large audio language models (LALMs) have shown promise on related audio tasks, including CM and ASV, their use for SASV remains unexplored, despite their capacity to produce natural-language rationales for auditing and robustness beyond discriminative predictions. This work systematically evaluates LALMs for SASV against conventional pipelines under zero-shot prompting, supervised adaptation, reasoning-oriented training, and reinforcement-learning-based optimization. Our results show that pretrained LALMs are near chance in the zero-shot setting, confirming that they are not natively suited to SASV, but that task-specific adaptation closes this gap. We further find that competitive SASV performance can be achieved through several distinct routes. These findings position LALMs as a promising and auditable foundation for unified SASV, while clarifying where conventional cascade systems still lead.

Sofya Savelyeva, Mariia Perunova, Evgeny Kushnir +3

Speaker Verification Performance Large Audio Language Models Audio

Ask PDF

Jun 7, 2026cs.SD

A Comparison of SSL-Based Feature Extractors and Back-End Classifiers for Spoofing Detection: A Multi-Corpus Training and Cross-Linguistic Analysis

Voice biometric systems face growing threats from spoofing attacks, yet the evaluation of detection models remains inconsistent across datasets. To investigate these unpredictable fluctuations, we conduct a comprehensive benchmark of four self-supervised learning feature extractors paired with four back-end classifiers. We compare the hierarchical local feature extraction of ResNet with the global sequence and relational modeling of attention and graph-based back-ends. Through multi-corpus training across three scenarios and six evaluation datasets, our empirical analysis yields two critical findings. First, we expose a domain bias within the ASVspoof 5 dataset, showing that naive data scaling actively degrades performance. Second, our cross-linguistic analysis reveals that fine-tuning with just 8 hours of target-language data enhances detection robustness. Together, these findings emphasize the critical need for domain-aware and language-specific adaptation in spoofing detection.

Anh-Tuan Dao, Driss Matrouf, Mickael Rouvier +1

Spoofed Speech Speaker Verification Performance Feature Extraction Biometrics Text Corpora

Ask PDF

Jun 2, 2026cs.SD

A Training-Efficient Transformer-Based Anti-Spoofing Network for Logical Access in ASVspoof 5

Synthetic and manipulated speech can reduce the reliability of automatic speaker verification systems, so anti-spoofing methods need to be both accurate and efficient in training and inference. This paper focuses on the ASVspoof 5 Track 1 closed condition, where standard cross-entropy training may not give enough attention to hard trials and is not directly aligned with ranking- and threshold-based evaluation metrics. We propose TFPARN, a Transformer-based focal-pairwise attentive ranking network. The system extracts log-Mel features from speech, uses a Transformer encoder to model frame-level information, applies attention pooling to obtain utterance-level representations, and is trained with a combination of focal classification loss and pairwise ranking loss. RawBoost augmentation is used during training, and test-time augmentation is applied during evaluation to improve robustness. Compared with re-implemented AASIST and RawNet2 baselines under the same protocol, TFPARN achieves the best results, with a minDCF of 0.2430 and an EER of 12.52%. Ablation experiments further show that the pairwise loss, focal loss, and attention pooling all improve performance. TFPARN also uses the lowest inference memory among the compared systems, at 1.4 GB, runs at about 0.79 ms per utterance, and reaches its best checkpoint in less training time than AASIST. These results show that TFPARN provides a good balance between detection accuracy and computational cost for logical access anti-spoofing.

Sidan Yin, Bo Zhao

Speaker Verification Performance Spoofed Speech Face Forgery Detection Transformer Encoder Synthetic Data Augmentation Reranking

Ask PDF