DuRe-ST: Dual-Relation Spectro-Temporal Modeling for Speech Deepfake Detection

Shaole Li; Siqing Qin; Youzhi Tu; Kong Aik Lee

eess.ASOct 7, 2026

DuRe-ST: Dual-Relation Spectro-Temporal Modeling for Speech Deepfake Detection

Authors: Shaole Li, Siqing Qin, Youzhi Tu, Kong Aik Lee

Organizations: Department of Electrical and Electronic Engineering The Hong Kong Polytechnic University, Hong Kong SAR

Abstract

Previous speech deepfake detectors can adaptively capture spectro-temporal dependencies through graph attention, yet they largely overlook the co-variation between spectral and temporal representations. To address this gap, we construct a normalized affinity graph from their joint covariance and apply polynomial graph filtering to capture higher-order covariance-induced dependencies. We first develop Cov-ST to isolate the contribution of covariance-based relational modeling. Although it improves detection performance, its sensitivity to the polynomial order suggests limited robustness when covariance relations are modeled alone. We therefore propose DuRe-ST, which jointly exploits covariance-induced and graph-attention-induced relations to capture complementary second-order co-variation and adaptive spectro-temporal dependencies. Experiments show that DuRe-ST achieves an average relative EER reduction of 25.9% over XLSR-AASIST on the ASVspoof benchmarks and 28.4% across four cross-dataset benchmarks with only 4-8k additional trainable back-end parameters. It further outperforms the strongest publicly available comparison models by 2.2-13.2% in relative EER on four benchmarks, while remaining smaller than the publicly available models considered.

Figures & tables

Figure 1: Overview of covariance-induced spectro-temporal relation modeling. Given the shared node features

\mathbf{X}=[\mathbf{X}_{\mathrm{S}};\mathbf{X}_{\mathrm{T}}]

, (a) constructs a normalized affinity graph

\widetilde{\mathbf{A}}

from their joint covariance, capturing second-order relations within and across the spectral and temporal domains; (b) applies an order-

K

polynomial graph filter over

\widetilde{\mathbf{A}}

to aggregate direct and multi-hop covariance-induced dependencies, yielding

\mathbf{Z}

; and (c) partitions

\mathbf{Z}

into the spectral and temporal node representations

\mathbf{Z}_{\mathrm{S}}

and

\mathbf{Z}_{\mathrm{T}}

, and independently summarizes each domain by max-magnitude and mean pooling over nodes. The four pooled statistics are concatenated and projected by a two-layer MLP to obtain

\mathbf{h}_{\mathrm{cov}}

. Cov-ST uses

\mathbf{h}_{\mathrm{cov}}

for classification, whereas DuRe-ST concatenates it with the attention-induced representation

\mathbf{h}_{\mathrm{att}}

to form

\mathbf{h}_{\mathrm{dual}}

for classification.

Model	$\boldsymbol{K}$	Params. (M)	2021 LA	2021 DF
XLSR-AASIST [ 18 ]	–	0.447	1.00	3.69
Cov-ST (Ours)	0	0.387	1.05	3.15
	1	0.391	1.56	2.71
	2	0.395	1.15	2.66
	3	0.400	1.08	7.79
DuRe-ST (Ours)	0	0.447	2.01	2.23

Table 1: ASVspoof benchmark performance and sensitivity to the polynomial order

K

, reported in terms of EER (%). Params. denotes the number of trainable back-end parameters in millions. The best results are highlighted in bold.

Figure 2: t-SNE projections of utterance-level embeddings on the DFADD evaluation set for (a) XLSR-AASIST, (b) DuRe-ST with $K=1$ , and (c) DuRe-ST with $K=2$ . The corresponding EER is reported above each panel.

Public models
Model	Params. (M)	In-the-Wild	Fake-or-Real	ASVspoof 5	DFADD
XLSR-Nes2Net-X [ 8 ] *	0.512	7.75	6.31	22.05	11.14
XLSR-Mamba [ 23 ]	1.937	6.70	6.71	14.40 *	11.81 $\dagger$
XLSR-Conformer [ 15 ] $\dagger$	2.383	6.68	7.55	17.21	11.51
XLSR-Conformer-TCM [ 20 ]	2.383	7.79 *	10.68 *	18.85 *	10.33 $\dagger$
XLSR-SLS [ 24 ] $\dagger$	23.399	9.65	7.55	18.50	9.02

Table 2: Cross-dataset comparison with publicly available SDD models in terms of EER (%). † and ∗ denote our reproduced results and results from Speech DF Arena [ 3 ] , respectively. Among the proposed models, the largest and second-largest EER reductions over the XLSR-AASIST baseline are shown in boldface and underlined, respectively.

Explore similar work

Jul 20, 2026cs.SD

Time-Frequency Consistency Learning for Robust Speech Deepfake Detection

Recently, speech deepfake detection (SDD) has achieved significant progress. However, its robustness evaluation remains largely confined to controlled additive noise scenarios, lacking systematic investigation of the complex distortions introduced by acoustic front-end (AFE) processing pipelines in real-world deployments. In this work, we simulate a unified AFE pipeline comprising acoustic echo cancellation, noise suppression, automatic gain control, and voice activity detection (VAD), and conduct a comprehensive evaluation of current state-of-the-art models. The results show that the nonlinear and time-frequency coupled distortions introduced by AFE significantly degrade detection performance. To address this issue, we propose a Time-Frequency Consistency Learning (TFCL) framework, which aims to learn invariant spoofing representations that remain stable before and after AFE processing. We observe that AFE not only introduces temporal misalignment (e.g., segment-level shifts caused by VAD), but also weakens or distorts critical frequency-domain cues. To this end, TFCL employs an attention-driven soft alignment mechanism to capture cross-temporal dependencies, along with frequency-domain structural consistency constraints to enforce feature invariance. As a result, the model is able to maintain stable representations under both temporal perturbations and spectral distortions. Extensive experimental results demonstrate that the proposed method effectively mitigates the performance degradation caused by AFE processing, significantly improving the robustness of SDD in real-world scenarios. The code is available at https://github.com/JunXue-tech/TFCL.

Jun Xue, Zhuolin Yi, Yanzhen Ren +6

School of Cyber Science and Engineering, Wuhan University Wuhan, China · Wuhan University Wuhan, China · Tongji University ShanghaiU, China

Audio Deepfake Detection Distortion

Ask PDF

May 5, 2026cs.SD

Deepfake Audio Detection Using Self-supervised Fusion Representations

This paper describes a submission to the Environment-Aware Speech and Sound Deepfake Detection Challenge (ESDD2) 2026, which addresses component-level deepfake detection using the CompSpoofV2 dataset, where speech and environmental sounds may be independently manipulated. To address this challenge, a dual-branch deepfake detection framework is proposed to jointly model speech and environmental contextual representations from input audio. Two pretrained models, XLS-R for speech and BEATs for environmental sound, are used to extract complementary contextual representations. A Matching Head is introduced to model representation differences through statistical normalization and representation interaction, enabling estimation of the original class. In parallel, multi-head cross-attention enables effective information exchange between speech and environmental components. The refined representations are processed with residual connections and layer normalization, and passed to an AASIST classifier to predict speech-based and environment-based spoofing probabilities. The model outputs original, speech, and environment predictions. On the test set, the proposed system achieves an F1-score of 70.20% and an environmental EER of 16.54%, outperforming the baseline system.

Khalid Zaman, Qixuan Huang, Muhammad Uzair +1

Graduate School of Advanced Science and Technology · Japan Advanced Institute of Science and Technology · 1-1 Asahidai, Nomi, Ishikawa, Japan 923-1292

Audio Understanding Self-Supervised Representations Multi-Head Attention Environment

Ask PDF

Sep 24, 2026eess.AS

WST-Graph: Topology-Preserving Wavelet Scattering Front-End for Speech Deepfake Detection

The acoustic front-end determines which forensic cues a speech deepfake detector can exploit. The wavelet scattering transform (WST) provides stable multiscale coefficients with explicit coordinates, yet direct flattening obscures the parent relation between paths. We introduce WST-Graph, reconstructing these paths as a sparse modulation-carrier grid for an AASIST graph backend. Modulation-level normalization and length-aware adaptive local attention pooling produce fixed relative-time representations while retaining the acoustic axes before learned adaptation. This yields a waveform-to-graph interface with a fixed, parameter-free WST. Our configurations remain competitive with AASIST while using approximately 60% fewer trainable parameters and show clear gains on selected out-of-domain benchmarks. These results underscore the value of preserving parent-child relations within the carrier-modulation topology when constructing a compact, physically grounded interface for graph-based speech deepfake detection. Code will be released at https://github.com/saki-ciallo/wst-graph.

Kwok-Ho Ng, Tingting Song, Bingwen Feng +1

College of Cyber Security, Jinan University, Guangzhou, China

Audio Deepfake Detection Wav2Vec Wavelets Waveforms Digital Forensics Graph Representations+1

Ask PDF