Exposing and Mitigating Neural Codec Vulnerabilities in Audio Deepfake Detection

Abdullah; Awais Khan; Khalid Mahmood Malik

cs.SDOct 5, 2026

Exposing and Mitigating Neural Codec Vulnerabilities in Audio Deepfake Detection

Authors: Abdullah, Awais Khan, Khalid Mahmood Malik

Organizations: College of Innovation and Technology University of Michigan-Flint Michigan, USA

Abstract

Existing audio deepfake detection (ADD) datasets and detectors are primarily built for vocoder-based synthesis, evaluated against traditional post-hoc perturbations such as MP3/AAC compression or additive noise, applied independently of generation. However, recent speech synthesizers, particularly ALM-based systems, use neural audio codecs both for compression and as the resynthesis reconstructing waveforms from generated tokens, producing artifacts distinct from post-hoc compression. Neural codecs thus play a dual role: some are designed for pure compression under low-bandwidth communication, while others serve as resynthesis components. Despite this dual role, robustness to codec-based compression, unlike post-hoc compression, remains largely unexplored. We expose this gap, showing that state-of-the-art (SOTA) ADD models degrade drastically on codec-compressed speech; in particular, systems trained on Codec Resynthesized data as a proxy for codec-based generation prove most vulnerable, with legitimately compressed bona fide speech often misclassified as fake. To investigate this, we construct the Audio Neural Codec-Spoof dataset by applying seven neural codec algorithms to existing ADD benchmarks, isolating codec-induced resynthesis artifacts as a controlled proxy for codec-based generation. As baseline mitigation, we propose PCL-NET (Pairwise Consistency Learned Network), fine-tuning a pretrained XLS-R (300M) encoder with a pairwise consistency objective that minimizes the representation distance between an utterance's uncompressed and codec-compressed versions, disentangling codec artifacts from the real-versus-fake decision. As a result, PCL-NET reduces average EER under neural codec compression from 28.67% to 12.77%, while preserving competitive CoSG-based deepfake detection performance. We will also make the dataset publicly available on Hugging Face upon acceptance.

Figures & tables

Fig. 1: Conceptual illustration of ADD performance under uncompressed, conventional, and neural codec compression. (a) Reliable detection of uncompressed fake audio (low EER, high AUC/accuracy). (b) Conventional compression (e.g., MP3/AAC/Opus) moderately degrades performance. (c) Neural audio codec compression causes more severe degradation.

Fig. 2: Overview of the proposed training and inference pipeline: SSL features extracted from original and neural-codec-compressed audio are jointly optimized via pairwise consistency and classification losses, with the resulting backend classifier used at inference to distinguish real from fake audio.

Dataset	Split	Orig.	Comp.	Total
ASV19 [ 32 ]	Training	25,380	177,660	203,040
ASV19 [ 32 ]	Validation	24,986	174,902	199,888
ASV19 [ 32 ]	Evaluation	71,933	503,531	575,464
FoR [ 33 ]	Evaluation	4,634	32,438	37,072
Wild [ 34 ]	Evaluation	31,780	222,460	254,240
Total		158,713	1,110,991	1,269,704

TABLE I: Configurational Details of the Proposed ANC-Spoof Dataset

Model	Dataset	Original	MP3/AAC/Opus				BigCodec	DAC	HiggsV2	Mimi	SNAC	Speech Tokenizer	Wav Tokenizer	Average
Model	Dataset	Original	12K	24K	48K	96K	BigCodec	DAC	HiggsV2	Mimi	SNAC	Speech Tokenizer	Wav Tokenizer	Average
AASIST ∗ [ 35 ]	ASV19	0.83	13.13	2.30	1.01	0.91	8.35	1.62	4.53	3.02	12.03	5.10	18.15	7.54
	Wild	43.02	51.51	45.79	43.73	43.26	34.91	39.55	34.99	36.39	32.88	42.28	47.28	38.33
	FoR	44.13	26.80	16.03	15.49	16.12	21.34	21.06	25.83	20.22	26.37	22.96	31.1	24.13
XLS-R-SLS ∗ [ 36 ]	ASV19	0.24	7.33	3.59	0.72	0.24	4.80	0.52	1.93	1.14	13.30	3.21	18.59	6.21
	Wild	7.46	27.43	13.32	9.91	9.22	25.43	10.17	17.44	13.51	37.03	27.81	46.62	25.43

TABLE II: Vulnerability of SOTA ADD Systems to Conventional and Neural Codec Compression (EER %). ∗ Standard, † Manipulation-robust, ‡ CoRS-based training strategies. Results obtained using author-provided pretrained weights.

Codec Method(s)	Dataset	Original	MP3/AAC/Opus				BigCodec	DAC	HiggsV2	Mimi	SNAC	Speech Tokenizer	Wav Tokenizer	Average
Codec Method(s)	Dataset	Original	12K	24K	48K	96K	BigCodec	DAC	HiggsV2	Mimi	SNAC	Speech Tokenizer	Wav Tokenizer	Average
Baseline	ASV19	0.34	7.97	5.65	0.83	0.38	5.11	0.75	2.35	1.74	13.39	4.02	21.10	6.92
	Wild	8.28	25.13	10.94	8.02	7.75	23.68	8.65	15.31	12.05	36.87	24.84	43.31	23.53
	FoR	11.35	35.78	24.47	14.48	9.58	31.77	22.49	49.50	32.41	61.31	42.58	53.45	41.93
BigCodec [ 25 ]	ASV19	0.14	5.98	1.28	0.23	0.19	2.55	0.41	1.44	1.11	12.67	2.88	15.25	5.19
	Wild	9.29	27.01	12.67	10.45	10.10	17.56	9.49	14.54	12.91	26.63	20.86	32.46	19.21

TABLE III: Robustness of PCL-NET Trained with Different Neural Codecs for Consistency Learning (EER %).

Model	CodecFake+ [ 14 ]	CDD [ 23 ]
AASIST [ 35 ]	42.23	37.72
XLS-R-SLS [ 36 ]	24.26	31.32
CLAD [ 2 ]	44.57	41.95
RawNet2 [ 21 ]	44.9	40.60
W2W2-AASIST [ 13 ]	9.9	16.89
MMS-300M [ 37 ]	5.5	6.51

TABLE IV: Generalization to CoSG Detection (EER %).

Explore similar work

Sep 30, 2026cs.SD

Neural Audio Codec for Robust Audio Deepfake Detection

Audio deepfake detectors are typically evaluated on uncompressed audio, although real-world audio often undergoes low-bitrate coding. In this work, we investigate how audio coding affects deepfake detection across codecs, bitrates, and detectors, finding higher errors at lower rates. A mixed-pair protocol isolates codec-induced changes in bona fide and spoof audio, revealing asymmetric, codec-dependent failures: low-rate DAC and EnCodec mainly degrade bona fide detection, whereas X-Codec shows a stronger spoof-side limitation. Motivated by these, we propose a forensic-preserving neural audio codec (FP-NAC), which fine-tunes a pretrained codec using a detector-guided objective while preserving its native hard quantization path and bitrate. On ASVspoof 2019 LA, FP-NAC reduces EER by up to 49.8~~pp compared with the original DAC at 0.5~~kbps while maintaining comparable reconstruction quality. Although supervised by only one detector, FP-NAC improves performance across multiple detectors, highlighting forensic transparency as a codec design objective alongside perceptual quality. Our codes are available at https://github.com/kjungwoo03/FP-NAC.

Jungwoo Kim, Joonyong Park, Junyoung Koh +1

Yonsei University, Seoul, Republic of Korea · MAAP Lab, Republic of Korea · The University of Tokyo, Tokyo, Japan +1

Neural Audio Codecs Affectcodec Spoofing

Ask PDF

Sep 27, 2026cs.SD

What Survives the Codec Shift: Pooled No-Vocals Residuals for Speech Deepfake Detection

The transition from vocoder-based to neural-codec speech synthesis makes generalization more difficult for speech deepfake detectors, particularly those relying on speech-oriented representations. It remains unclear which acoustic representations retain discriminative information when the generation mechanism changes. We therefore compare 12 acoustic representations using a shared low-capacity linear classifier to identify effective evidence under codec shift. The analysis shows that hierarchical XLS-R leads on the pooled test set, while pooled no-vocals residual statistics perform best on the unseen-codec condition, revealing complementary behavior across generation conditions. Building on this finding, we propose MN-P, a dual-view detector that integrates an utterance-level pooled no-vocals representation with token-level XLS-R features through adaptive gating. The proposed MN-P reduces EER by 54.2% overall and by 60.9% on the codec-unseen condition relative to the best-performing retrained state-of-the-art system, with consistent gains across different detector backends. These results indicate that pooled no-vocals residual statistics provide effective complementary evidence for cross-generation speech deepfake detection.

Jiajun Xu, Menglu Li, Xiao-Ping Zhang

Department of Engineering Physics Tsinghua University Beijing, China · Tsinghua Shenzhen International Graduate School Tsinghua University Shenzhen, China

Audio Deepfake Detection

Ask PDF

Jun 5, 2026cs.SD

Mitigating Proxy-to-Wild Domain Gap in Deepfake Speech

Recent neural audio codec-based speech generation (CodecFake) produces highly realistic audio, posing a challenge to existing deepfake countermeasure models. While using codec resynthesized speech (CoRS) as proxy data improves performance, it often suffers from limited generalization. We propose Domain-Shift Feature Augmentation (DSFA), which simulates "in-the-wild" variations by transforming deterministic feature statistics into stochastic distributions during fine-tuning. To evaluate generalization, we further introduce Codec-based Speech Generation Extension Evaluation (CoSG ExtEval) dataset, a more challenging extension of the CoSG Eval (from CodecFake+) dataset, featuring 40 unseen generative models and long-form audio. Experimental results demonstrate that combining a post-trained SSL backbone with DSFA effectively narrows the proxy-to-wild domain gap. This approach achieves state-of-the-art performance across diverse CodecFake attacks in both CoSG Eval and CoSG ExtEval.

Xuanjun Chen, Yun-Shing Wu, Wei-Chung Lu +4

Graduate Institute of Communication Engineering, National Taiwan University · Graduate Institute of Networking and Multimedia, National Taiwan University · Department of Information Management, National Taiwan University +2

Topic pending

Ask PDF