DuRe-ST: Dual-Relation Spectro-Temporal Modeling for Speech Deepfake Detection

Shaole Li; Siqing Qin; Youzhi Tu; Kong Aik Lee

eess.ASOct 7, 2026

DuRe-ST: Dual-Relation Spectro-Temporal Modeling for Speech Deepfake Detection

Authors: Shaole Li, Siqing Qin, Youzhi Tu, Kong Aik Lee

Organizations: Department of Electrical and Electronic Engineering The Hong Kong Polytechnic University, Hong Kong SAR

Abstract

Previous speech deepfake detectors can adaptively capture spectro-temporal dependencies through graph attention, yet they largely overlook the co-variation between spectral and temporal representations. To address this gap, we construct a normalized affinity graph from their joint covariance and apply polynomial graph filtering to capture higher-order covariance-induced dependencies. We first develop Cov-ST to isolate the contribution of covariance-based relational modeling. Although it improves detection performance, its sensitivity to the polynomial order suggests limited robustness when covariance relations are modeled alone. We therefore propose DuRe-ST, which jointly exploits covariance-induced and graph-attention-induced relations to capture complementary second-order co-variation and adaptive spectro-temporal dependencies. Experiments show that DuRe-ST achieves an average relative EER reduction of 25.9% over XLSR-AASIST on the ASVspoof benchmarks and 28.4% across four cross-dataset benchmarks with only 4-8k additional trainable back-end parameters. It further outperforms the strongest publicly available comparison models by 2.2-13.2% in relative EER on four benchmarks, while remaining smaller than the publicly available models considered.

Figures & tables

Figure 1: Overview of covariance-induced spectro-temporal relation modeling. Given the shared node features

\mathbf{X}=[\mathbf{X}_{\mathrm{S}};\mathbf{X}_{\mathrm{T}}]

, (a) constructs a normalized affinity graph

\widetilde{\mathbf{A}}

from their joint covariance, capturing second-order relations within and across the spectral and temporal domains; (b) applies an order-

K

polynomial graph filter over

\widetilde{\mathbf{A}}

to aggregate direct and multi-hop covariance-induced dependencies, yielding

\mathbf{Z}

; and (c) partitions

\mathbf{Z}

into the spectral and temporal node representations

\mathbf{Z}_{\mathrm{S}}

and

\mathbf{Z}_{\mathrm{T}}

, and independently summarizes each domain by max-magnitude and mean pooling over nodes. The four pooled statistics are concatenated and projected by a two-layer MLP to obtain

\mathbf{h}_{\mathrm{cov}}

. Cov-ST uses

\mathbf{h}_{\mathrm{cov}}

for classification, whereas DuRe-ST concatenates it with the attention-induced representation

\mathbf{h}_{\mathrm{att}}

to form

\mathbf{h}_{\mathrm{dual}}

for classification.

Model	$\boldsymbol{K}$	Params. (M)	2021 LA	2021 DF
XLSR-AASIST [ 18 ]	–	0.447	1.00	3.69
Cov-ST (Ours)	0	0.387	1.05	3.15
	1	0.391	1.56	2.71
	2	0.395	1.15	2.66
	3	0.400	1.08	7.79
DuRe-ST (Ours)	0	0.447	2.01	2.23

Table 1: ASVspoof benchmark performance and sensitivity to the polynomial order

K

, reported in terms of EER (%). Params. denotes the number of trainable back-end parameters in millions. The best results are highlighted in bold.

Figure 2: t-SNE projections of utterance-level embeddings on the DFADD evaluation set for (a) XLSR-AASIST, (b) DuRe-ST with $K=1$ , and (c) DuRe-ST with $K=2$ . The corresponding EER is reported above each panel.

Public models
Model	Params. (M)	In-the-Wild	Fake-or-Real	ASVspoof 5	DFADD
XLSR-Nes2Net-X [ 8 ] *	0.512	7.75	6.31	22.05	11.14
XLSR-Mamba [ 23 ]	1.937	6.70	6.71	14.40 *	11.81 $\dagger$
XLSR-Conformer [ 15 ] $\dagger$	2.383	6.68	7.55	17.21	11.51
XLSR-Conformer-TCM [ 20 ]	2.383	7.79 *	10.68 *	18.85 *	10.33 $\dagger$
XLSR-SLS [ 24 ] $\dagger$	23.399	9.65	7.55	18.50	9.02

Table 2: Cross-dataset comparison with publicly available SDD models in terms of EER (%). † and ∗ denote our reproduced results and results from Speech DF Arena [ 3 ] , respectively. Among the proposed models, the largest and second-largest EER reductions over the XLSR-AASIST baseline are shown in boldface and underlined, respectively.

Explore similar work

Time-Frequency Consistency Learning for Robust Speech Deepfake Detection
Jul 20, 2026Jun Xue, Zhuolin Yi, Yanzhen Ren +6Audio Deepfake Detection Distortion
School of Cyber Science and Engineering, Wuhan University Wuhan, China · Wuhan University Wuhan, China · Tongji University ShanghaiU, China
Deepfake Audio Detection Using Self-supervised Fusion Representations
May 5, 2026Khalid Zaman, Qixuan Huang, Muhammad Uzair +1Audio Understanding Self-Supervised Representations
Graduate School of Advanced Science and Technology · Japan Advanced Institute of Science and Technology · 1-1 Asahidai, Nomi, Ishikawa, Japan 923-1292
WST-Graph: Topology-Preserving Wavelet Scattering Front-End for Speech Deepfake Detection
Sep 24, 2026Kwok-Ho Ng, Tingting Song, Bingwen Feng +1Audio Deepfake Detection Wav2Vec
College of Cyber Security, Jinan University, Guangzhou, China

Jul 20, 2026, cs.SDJ/K move · Enter open · S save

Time-Frequency Consistency Learning for Robust Speech Deepfake Detection

Jun Xue, Zhuolin Yi, Yanzhen Ren, Yihuan Huang+5

School of Cyber Science and Engineering, Wuhan University Wuhan, China · Wuhan University Wuhan, China · Tongji University ShanghaiU, China

Audio Deepfake Detection Distortion

Open paper AskPDF arXiv

Recently, speech deepfake detection (SDD) has achieved significant progress. However, its robustness evaluation remains largely confined to controlled additive noise scenarios, lacking systematic investigation of the complex distortions introduced by acoustic front-end (AFE) processing pipelines in real-world deployments. In this work, we simulate a unified AFE pipeline comprising acoustic echo cancellation, noise suppression, automatic gain control, and voice activity detection (VAD), and conduct a comprehensive evaluation of current state-of-the-art models. The results show that the nonlinear and time-frequency coupled distortions introduced by AFE significantly degrade detection performance. To address this issue, we propose a Time-Frequency Consistency Learning (TFCL) framework, which aims to learn invariant spoofing representations that remain stable before and after AFE processing. We observe that AFE not only introduces temporal misalignment (e.g., segment-level shifts caused by VAD), but also weakens or distorts critical frequency-domain cues. To this end, TFCL employs an attention-driven soft alignment mechanism to capture cross-temporal dependencies, along with frequency-domain structural consistency constraints to enforce feature invariance. As a result, the model is able to maintain stable representations under both temporal perturbations and spectral distortions. Extensive experimental results demonstrate that the proposed method effectively mitigates the performance degradation caused by AFE processing, significantly improving the robustness of SDD in real-world scenarios. The code is available at https://github.com/JunXue-tech/TFCL.

Abstract

Explore similar work

Time-Frequency Consistency Learning for Robust Speech Deepfake Detection

Deepfake Audio Detection Using Self-supervised Fusion Representations

WST-Graph: Topology-Preserving Wavelet Scattering Front-End for Speech Deepfake Detection