Learning as Deepfakes Evolve: RF-Prompt for Continual Audio Deepfake Detection

Yuankun Xie; Xiaoxuan Guo; Xiaopeng Wang; Siqing Qin; Shaole Li; Kong Aik Lee

cs.SDSep 29, 2026

Learning as Deepfakes Evolve: RF-Prompt for Continual Audio Deepfake Detection

Authors: Yuankun Xie, Xiaoxuan Guo, Xiaopeng Wang, Siqing Qin, Shaole Li, Kong Aik Lee

Organizations: The Hong Kong Polytechnic University, Hong Kong SAR, China · Communication University of China, Beijing, China · Beijing Institute of Technology, Beijing, China

Abstract

Continual audio deepfake detection requires learning newly emerging deepfake methods while retaining discrimination of previously encountered speech. Existing dataset-incremental evaluation changes both real-speech domains and deepfake mechanisms, making their effects difficult to distinguish. We construct five task organizations over identical training, development, and evaluation pools to study these factors under a controlled sample budget. Our proposed Real-Anchored Mechanism-Incremental (RAMI) protocol reflects the practical setting in which available real speech provides a recurring mixed-domain reference while new deepfake mechanisms arrive incrementally. We further propose RF-Prompt, an asymmetric continual prompt-learning method that preserves reusable real-speech knowledge through a shared real prompt and expands mechanism-specific knowledge through inherited fake experts with orthogonal residuals. Input-adaptive soft fusion combines the accumulated experts into a fixed number of injected tokens without requiring task identity at inference. On RAMI, RF-Prompt achieves 10.110% average EER and 10.370% pooled EER, outperforming all evaluated continual-learning baselines. Across the five controlled protocols, RAMI yields the lowest common-average and pooled EER. Component ablations, limited-data experiments, and cross-backbone evaluations further validate the proposed design.

Figures & tables

Figure 1: Overview of mechanism-incremental audio deepfake detection. RF-Prompt preserves shared real-speech knowledge while expanding fake experts for emerging deepfake mechanisms. Counts below M1–M4 indicate the numbers of generators in the training, development, and evaluation splits.

Figure 2: Five task organizations from identical train, development, and evaluation pools. Columns vary real arrival, while rows vary fake organization. RAMI (Protocol 5) is the proposed protocol; recurring real domains use disjoint utterances across tasks.

Figure 3: Overview of our proposed RF-Prompt. Left: the overall audio deepfake detection pipeline. Middle: continual learning of the shared real prompt and task-specific fake experts. Right: fake expert selection, residual learning, expert construction, and soft fusion at Task 4 as an example.

Figure 4: Acquired-task EER trajectories of Sequential and RF-Prompt on RAMI (Protocol 5). Rows correspond to the checkpoint after each task and columns to acquired deepfake mechanisms; lower EER is better. Purple boxes mark the final checkpoint. Complete trajectories for all continual methods appear in Appendix C .

Method	Category	Venue	M1	M2	M3	M4	Avg EER	Pool EER	AF
Sequential	Traditional CL	–	14.50	9.12	18.78	10.12	13.130	13.445	6.433
EWC ( Kirkpatrick and others, 2017 )	General CL	PNAS’17	13.40	8.30	17.88	9.44	12.255	12.805	5.827
OWM ( Zeng et al., 2019 )	General CL	Nat. Mach. Intell.’19	13.80	8.68	17.64	9.90	12.505	13.080	5.993
RAWM ( Zhang et al., 2023 )	ADD-specific CL	ICML’23	10.22	8.02	14.96	11.98	11.295	11.730	5.753
RWM ( Zhang et al., 2024c )	ADD-specific CL	AAAI’24	13.40	8.32	17.46	9.88	12.265	12.845	5.700
RegO ( Chen and others, 2025 )	ADD-specific CL	AAAI’25	14.46	8.22	17.72	9.64	12.510	13.000	6.060

Table 1: Final results on RAMI (Protocol 5) with XLS-R 300M. EER and AF are reported in %; lower is better. Category and venue identify each method’s original scope and publication. All scores are from our matched experiments;

\dagger

marks methods adapted to continual ADD. Offline co-training is excluded when identifying the best continual method.

Protocol	real arrival	fake organization	Common Avg EER	Pool EER
1	Dataset-wise	Dataset-wise	16.400	16.430
2	Dataset-wise	Mechanism-wise	15.675	15.900
3	Source-support-matched	Mechanism-wise	13.230	13.110
4	Four-domain mixture	Dataset-wise	11.035	11.330
5 (RAMI)	Four-domain mixture	Mechanism-wise	10.110	10.370

Table 2: Controlled protocol comparison for our RF-Prompt model. The common average uses identical RAMI (Protocol 5) evaluation groups.

Configuration	Avg EER	Pool EER	AF
RF-Prompt (full)	10.110	10.370	4.560
w/o real cosine-anchoring loss	10.755	11.365	5.293
w/o residual-orthogonality loss	12.255	12.805	7.320
w/o adaptive fusion (uniform mean)	11.525	12.480	5.480
w/o orthogonal initialization projection	11.085	11.430	5.867

Table 3: Ablations on RAMI. Each control retains the other selected settings; the initialization control removes only historical-space projection.

Figure 5: Protection-depth sweeps (left) and limited-fake-data adaptation (right). The other constraint remains active across 24 layers; the right axis counts unique fake clips per task.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Protocol	Task 1	Task 2	Task 3	Task 4
1	4,900/4,505/9,832	3,200/3,535/8,525	4,800/4,800/10,000	6,300/6,360/11,643
2	4,800/4,800/10,000	4,800/4,800/10,000	4,800/4,800/10,000	4,800/4,800/10,000
3	4,704/5,526/10,600	5,819/5,019/10,380	4,800/4,800/10,000	3,877/3,855/9,020
4	4,900/4,505/9,832	3,200/3,535/8,525	4,800/4,800/10,000	6,300/6,360/11,643
5 (RAMI)	4,800/4,800/10,000	4,800/4,800/10,000	4,800/4,800/10,000	4,800/4,800/10,000

Appendix

Table 4: Total utterances per task. Each cell lists Train/Dev/Eval counts.

Mechanism	Train	Dev	Eval
M1	ASV2019: 2,400	ASV2019: 2,000; ASV5: 400	ASV2019: 3,890; ASV5: 1,110
M2	ASV2019: 100; ASV5: 800; AT-ADD: 1,500	ASV2019: 105; ASV5: 735; AT-ADD: 1,560	ASV2019: 942; ASV5: 2,030; AT-ADD: 2,028
M3	CodecFake: 2,400	CodecFake: 2,400	CodecFake: 5,000
M4	AT-ADD: 2,400	AT-ADD: 2,400	ASV5: 385; AT-ADD: 4,615

Appendix

Table 5: Source-dataset composition of the four mechanism-defined fake tasks under RAMI.

Source	Mechanism	Generator IDs in assignment catalog
ASV2019	M1	A02, A03, A04, A05, A06, A11, A13, A14, A16, A17, A18, A19
ASV2019	M2	A01, A07, A08, A09, A10, A12, A15
ASV5	M1	A12, A19, A20
ASV5	M2	A01, A02, A03, A04, A05, A06, A07, A08, A09, A10, A11, A13, A14, A15, A16, A17, A18, A21, A22, A23, A24, A25, A26, A27, A28, A30, A31, A32
ASV5	M4	A29
AT-ADD	M2	BigVGAN, DiffGANTTS, DiffSpeech_fastdiff, E2TTS, F5TTS, FastDiff, FastPitch_fastdiff, FastSpeech2_fastdiff, GlowTTS, GradTTS, HiFiGAN, Kokoro, MBMelGAN, MelGAN, MeloTTS, OpenVoice2, ParallelWaveGAN, PortaSpeech_normal_fastdiff, SeedVC1, StarGANv2VC, StyleMelGAN, StyleSpeech, StyleTTS2, Tacotron2, VITS, WaveGlow, WaveNet, prodiff_teacher_fastdiff

Appendix

Table 6: Generation-mechanism assignment by source dataset.

Setting	Value
Sample rate / input length	16 kHz / 64,600 samples
Tasks / epochs per task	4 / 50
Batch size / random seed	32 / 2026
Optimizer	Adam
Adam betas / epsilon	(0.9, 0.999) / 1e-8
Weight decay	5e-4

Appendix

Table 7: Selected XLS-R 300M configuration.

Method	After Task 1	After Task 2	After Task 3	After Task 4	Final Avg	Final Pool
Sequential	4.66	8.00/9.38	12.84/12.22/9.32	14.50/9.12/18.78/10.12	13.130	13.445
EWC	4.66	8.28/9.68	13.58/12.34/9.14	13.40/8.30/17.88/9.44	12.255	12.805
OWM	4.66	9.34/9.80	14.08/12.50/8.80	13.80/8.68/17.64/9.90	12.505	13.080
RAWM	4.66	6.18/5.10	8.54/6.62/6.18	10.22/8.02/14.96/11.98	11.295	11.730
RWM	4.66	9.24/10.08	13.80/12.38/9.10	13.40/8.32/17.46/9.88	12.265	12.845
RegO	4.66	8.82/10.16	14.92/12.88/9.34	14.46/8.22/17.72/9.64	12.510	13.000

Appendix

Table 8: Acquired-task EER trajectories on RAMI (Protocol 5). The entry under “After Task

t

” lists EERs for

\mathrm{M1}/\cdots/\mathrm{M}t

in order; the last two columns report final average and pooled EER. All entries are percentages.

Figure 6: RF-Prompt acquired-task EER trajectories on Protocols 1–4. Rows denote the checkpoint after each task, columns denote the task-specific evaluation groups, and purple boxes mark the final checkpoint. All values are EER (%); lower is better.

Figure 7: Comparison of real prompt protection and fake expert construction. Shared Prompt Distillation (SPD) aligns audio-token features produced with current and previous real prompts; Prompt cosine directly anchors their parameters. Full-Prompt orthogonality separates complete fake experts, whereas residual orthogonality separates only the newly learned residual while retaining the inherited base.

real protection	fake construction	Avg	Pool	AF
None	None	12.600	12.020	5.393
None	Inherited residual	10.755	11.365	5.293
Shared Prompt Distillation (SPD)	Inherited residual	10.910	11.180	5.467
Prompt cosine	Full-Prompt orthogonality	11.795	12.355	6.140
Prompt cosine	Inherited residual	10.110	10.370	4.560

Appendix

Table 9: Comparison of real prompt protection and fake expert construction on RAMI. All entries are EER or AF (%).

Figure 8: Geometry of the learned fake experts on RAMI. Panels (a)–(b) show a joint PCA projection of complete-expert and residual tokens; translucent points are layer-wise prompt tokens and stars are mechanism centroids. Panels (c)–(d) report normalized layer-wise subspace overlap, where lower off-diagonal values indicate stronger separation.

Unique fake/task	Method	Avg EER	Pool EER	AF
100	Sequential	17.325	17.750	6.887
	EWC	17.705	18.080	6.707
	RWM	17.075	17.645	6.067
	RegO	17.765	18.210	6.040
	RF-Prompt	12.590	12.375	3.907
500	Sequential	15.220	15.715	7.367

Appendix

Table 10: Limited-data results on RAMI. All metrics are percentages; lower is better.

Figure 9: Sensitivity to the number of real and fused fake prompt tokens per transformer layer on RAMI. The selected configuration uses five tokens for each prompt type, for ten injected tokens in total.

Figure 10: Cross-backbone comparison of Sequential and RF-Prompt. Each panel reports one final continual-learning metric after four tasks; lower is better. Values above the bars are percentages.

Explore similar work

Apr 17, 2026cs.SD

ICLAD: In-Context Learning with Comparison-Guidance for Audio Deepfake Detection

Audio deepfakes pose a significant security threat, yet current state-of-the-art (SOTA) detection systems do not generalize well to realistic in-the-wild deepfakes. We introduce a novel \textbf{I}n-\textbf{C}ontext \textbf{L}earning paradigm with comparison-guidance for \textbf{A}udio \textbf{D}eepfake detection (\textbf{ICLAD}). The framework enables the use of audio language models (ALMs) for training-free generalization to unseen deepfakes and provides textual rationales on the detection outcome. At the core of ICLAD is a pairwise comparative reasoning strategy that guides the ALM to discover and filter hallucinations and deepfake-irrelevant acoustic attributes. The ALM works alongside a specialized deepfake detector, whereby a routing mechanism feeds out-of-distribution samples to the ALM. On in-the-wild datasets, ICLAD improves macro F1 over the specialized detector, with up to

2\times

relative improvement. Further analysis demonstrates the flexibility of ICLAD and its potential for deployment on recent open-source ALMs.

Benjamin Chou, Yi Zhu, Surya Koppisetti

Purdue University, USA · Reality Defender Inc., USA

Environmental Sound Deepfake Detection Deepfake Detection Audio Understanding In-Context

Ask PDF

Sep 30, 2026cs.SD

SE-ADD: Self-Evolving Audio Deepfake Detection with Mistake-Driven Supervision

Audio deepfake detection (ADD) must remain effective when new spoofing attacks emerge after deployment. Emerging audio language model (ALM)-based ADD methods are built on predefined supervision from ground-truth labels or verified forensic rationales. However, this paradigm overlooks an ALM's own mistakes, which indicate where targeted supervision is most needed. To this end, we first introduce evolving spoofing environments for ALM-based ADD, where a new attack becomes dominant while previously observed attacks persist. Motivated by the above learning-from-mistakes perspective, we further propose SE-ADD, a self-evolving framework that iteratively adapts an ALM via low-rank adaptation (LoRA) using mistake-driven supervision built from its verdicts and self-generated forensic cues. All training samples receive direct authenticity supervision, while misclassified ones receive additional cue-augmented supervision. As verdicts and cues are regenerated by the updated ALM, the resulting supervision evolves accordingly. Experiments on two ALMs demonstrate the effectiveness of SE-ADD in generalizing to unseen attacks, reducing the equal error rate (EER) from

36.72\%

7.52\%

for Qwen2-Audio and from

19.93\%

3.97\%

for MOSS-Audio.

Rong Wan, Wei Xie, Jiaxi Li +4

University of Surrey · Guangxi University · Shenzhen University of Advanced Technology +1

Environmental Sound Deepfake Detection Audio Understanding Digital Forensics Process-Level Supervision

Ask PDF

Sep 3, 2026eess.AS

ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection

Audio deepfake detection is commonly formulated as clip-level binary classification of single-domain audio. However, real-world manipulated audio can exhibit mixed authenticity, where genuine and manipulated cues coexist across temporal transitions, overlapping sources, or both. This setting requires not only detecting manipulated audio but also localizing the components that provide evidence for the decision. We propose ToolDF, a tool-integrated reasoning framework for mixed-authenticity audio deepfake detection. ToolDF employs an audio large language model as an orchestrator trained with supervised tool-use trajectories. It adaptively analyzes the audio scene, selectively performs source separation, routes components to domain-specific experts, and aggregates their evidence into an interpretable verdict. We further introduce a mixed-authenticity ADD benchmark covering temporal transitions, acoustic overlaps, and hybrid mixtures. Experimental results show that ToolDF achieves the best overall performance on composite-type detection, achieving macro-F1 gains of 3.72 and 14.39 points over the strongest monolithic baseline and a fixed pipeline, respectively, while providing interpretable evidence localized to temporal regions and acoustic sources. Our source code and dataset are publicly available online.

Taewoo Kim, Young Han Lee, Nam In Park +1

Multi-Modal Research Center, KETI, South Korea · Digital Analysis Section, National Forensic Service, South Korea · Department of Artificial Intelligence, Korea University, South Korea

Environmental Sound Deepfake Detection Audio Understanding

Ask PDF