Graph Attention Design Choices Matter: A Controlled Study of LoRA-Adapted Audio Anti-Spoofing
Authors: Haoyu Wang, Jing Yang, Chenyu Liu, Yushan Du, Yifan Liao, Ningning Pan, Gongping Huang, Yu Zhao, +2 more
Organizations: School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics, Chengdu, China · MiLM Plus, Xiaomi Inc., Beijing, China · School of Electronic Information, Wuhan University, Wuhan, China · NERCMS, School of Computer Science, Hubei Luojia Laboratory, Wuhan University, Wuhan, China
Audio anti-spoofing systems increasingly combine self-supervised learning, parameter-efficient fine-tuning, and graph-attention-based backends. However, performance gains in such systems are often entangled with concurrent changes in the backbone, fine-tuning strategy, and training protocol, making the independent contribution of graph attention design difficult to isolate. To address this issue, we conduct a systematic controlled study of the graph attention layer under a unified experimental setting. We decompose the layer into three independently testable design dimensions: scoring symmetry, temperature learnability, and routing granularity. These are instantiated as a concat-based scoring branch, a LearnT branch with learnable temperature, and a multi-temperature routing branch, respectively. Each dimension is implemented as an independently gated residual branch, enabling the evaluation of both individual variants and their combinations under the same experimental setting. Experiments on five evaluation sets with five random seeds show that the LearnT branch achieves the best average equal error rate (EER), yielding a 16.1% relative improvement over the baseline. In contrast, the multi-temperature routing branch does not improve average performance on its own, but substantially reduces cross-seed standard deviation when combined with the concat-based scoring branch. Moreover, two individually effective branches degrade performance when used together, resulting in a 25.6% relative deterioration compared with the baseline. This finding reveals strong non-additive interactions among graph attention design dimensions. Overall, the results suggest that, under parameter-constrained fine-tuning, improvements in graph attention layers depend more on capacity allocation and branch interaction than on simply adding more learnable parameters.
Flow-matching transformers achieve strong audio separation, yet their attention dynamics are opaque. We adapt established causal-intervention principles into a deterministic, inference-time probing protocol for SAM Audio. Orthogonal probing uncovers a dual-pathway text-conditioning mechanism: additive injections control semantic identity, while cross-attention refines acoustic structure. We observe an asynchronous layerwise convergence: stable layers build temporal scaffolds early, whereas fast layers continue resolving artifacts during sampling. The model also attenuates temporal segmentation cues to maintain continuous-flow stability. Using these insights, we propose Layer-Selective Attention Caching (LSAC), a training-free acceleration method that caches attention in stable layers. Across acoustic complexities, LSAC cuts self-attention computation by about ~25% with negligible quality loss and yields up to 6.7x higher quality retention than naive step reduction.
We present SpAArSIST, a deployment-oriented refinement of the widely used AASIST graph pooling backend for self-supervised learning (SSL) based anti-spoofing. Motivated by redundant operations in public implementations, we replace learned pooling and stack-node attention with explicit, lightweight choices: separate train and inference graph pooling ratios (ktr,kinf), magnitude-based node scoring, and mean aggregation of graph nodes. The best overall configuration (rank 1) cuts backend compute by 20.7% (195.045M → 154.706M MACs) and model size by 4.1% (611.8k → 586.4k params), while improving out-of-domain robustness on In-the-Wild to 2.82% EER and 0.078 minDCF (from 4.64% and 0.133) and remaining competitive on ASVspoof5. We further provide a composite selection score that summarizes accuracy, calibration, and compute to support balanced deployment-oriented model choice.
Most spoofing countermeasures now place a light classifier on top of a self-supervised (SSL) speech encoder. A growing line of work adds handcrafted acoustic features and fuses them by cross-attention, partly because the attention weights appear to explain which cues the model relies on. Two things about such fusion remain untested: whether the handcrafted features contribute anything once a strong SSL encoder is already in place, and whether the attention map faithfully reflects what the classifier actually uses. We study both questions with MOSAIC, a single model that handles logical access (LA) and physical access (PA) attacks by projecting a 152-dimensional biophonetic vector into six query tokens attending over thirteen intermediate layers of WavLM-Large. Rather than pursuing state-of-the-art accuracy, we treat the model itself as the object of analysis and apply inference-time interventions that zero out selected components. Removing the handcrafted branch lowers the EER by 0.67 percentage points under LA, so WavLM alone is sufficient there, but raises it by 0.76 points under PA, where the branch supplies replay channel evidence the encoder does not carry. The 6x13 attention map is highly stable under resampling (bootstrap rank correlation 0.99), yet its faithfulness varies by domain: layer attention weights predict the causal importance of each layer under PA (rho = +0.62) but not under LA (rho = -0.28). The handcrafted branch and its attention-based explanation are therefore trustworthy for replay attacks and not for synthetic ones. On standard benchmarks the model is mid-range on LA and ahead of the official baselines on cross-source deepfakes (6.21% EER on ASVspoof 2021 DF). The contribution is not a new architecture but a checking procedure: verify handcrafted cues and attention maps by intervention before trusting either.