Neural Audio Codec for Robust Audio Deepfake Detection
Authors: Jungwoo Kim, Joonyong Park, Junyoung Koh, Jong-Seok Lee
Organizations: Yonsei University, Seoul, Republic of Korea · MAAP Lab, Republic of Korea · The University of Tokyo, Tokyo, Japan · University of Michigan, Ann Arbor, MI, USA
Audio deepfake detectors are typically evaluated on uncompressed audio, although real-world audio often undergoes low-bitrate coding. In this work, we investigate how audio coding affects deepfake detection across codecs, bitrates, and detectors, finding higher errors at lower rates. A mixed-pair protocol isolates codec-induced changes in bona fide and spoof audio, revealing asymmetric, codec-dependent failures: low-rate DAC and EnCodec mainly degrade bona fide detection, whereas X-Codec shows a stronger spoof-side limitation. Motivated by these, we propose a forensic-preserving neural audio codec (FP-NAC), which fine-tunes a pretrained codec using a detector-guided objective while preserving its native hard quantization path and bitrate. On ASVspoof 2019 LA, FP-NAC reduces EER by up to 49.8pp compared with the original DAC at 0.5kbps while maintaining comparable reconstruction quality. Although supervised by only one detector, FP-NAC improves performance across multiple detectors, highlighting forensic transparency as a codec design objective alongside perceptual quality. Our codes are available at https://github.com/kjungwoo03/FP-NAC.
Figures & tables
Figure 1 : Low-bitrate Coding Degrades Audio Deepfake Detection. Full EER (%) on the ASVspoof 2019 LA evaluation set [ 24 ] is reported with its measured bitrate across three different detectors. The gray horizontal lines indicate the EER on uncompressed audio. EER generally increases as bitrate decreases, while FP-NAC substantially reduces the EER compared with other NACs.
Figure 2 : Codec-Dependent Asymmetric Failure Modes. H1 and H2 across bitrates under AASIST [ 9 ] . Acoustic codecs (DAC, EnCodec) show large H1 that shrinks with bitrate, while X-Codec shows a persistently elevated H2 .
Figure 3 : Class-Directed Representation Shifts in AASIST. Detector embeddings at 0.5 kbps are projected onto the clean bona fide–spoof centroid axis. DAC strongly shifts bona fide representations toward spoof, while FP-NAC substantially mitigates this shift.
Figure 4 : Overall Architecture of FP-NAC. The left panel shows frozen DAC backbones with trainable residual adapters conditioned on the active RVQ depth K . The right panel illustrates the adapter architecture, consisting of bottleneck projection, depthwise temporal convolutions, K -conditioned FiLM modulation, and residual addition.
Figure 5 : Multi-Rate Sampling Distribution. Sampling probability over the active RVQ depth K .
Figure 6 : Reconstruction Quality across Bitrates. PESQ-WB [ 8 ] and STOI [ 21 ] on a subset of ASVspoof 2019 LA [ 24 ] . Both reconstruction-only adaptation and FP-NAC improve low-rate reconstruction quality over Plain DAC, with closely matched quality across most operating points.
Configuration
Trainable Params.
Overhead
Plain DAC
74.14M
0%
Encoder only
+0.72M
+0.97%
Decoder only
+0.58M
+0.78%
Both (FP-NAC)
+1.30M
+1.75%
Table 1 : Parameter Overhead of Adapters. Additional trainable parameters relative to the 74.14M-parameter original DAC [ 12 ] .
Figure 7 : Encoder–Decoder Adaptation Ablation at 0.5 kbps. Full EER (%) across AASIST, AASIST-L, and RawNet2 for encoder-only, decoder-only, and joint adaptation (ours).
The transition from vocoder-based to neural-codec speech synthesis makes generalization more difficult for speech deepfake detectors, particularly those relying on speech-oriented representations. It remains unclear which acoustic representations retain discriminative information when the generation mechanism changes. We therefore compare 12 acoustic representations using a shared low-capacity linear classifier to identify effective evidence under codec shift. The analysis shows that hierarchical XLS-R leads on the pooled test set, while pooled no-vocals residual statistics perform best on the unseen-codec condition, revealing complementary behavior across generation conditions. Building on this finding, we propose MN-P, a dual-view detector that integrates an utterance-level pooled no-vocals representation with token-level XLS-R features through adaptive gating. The proposed MN-P reduces EER by 54.2% overall and by 60.9% on the codec-unseen condition relative to the best-performing retrained state-of-the-art system, with consistent gains across different detector backends. These results indicate that pooled no-vocals residual statistics provide effective complementary evidence for cross-generation speech deepfake detection.
Jiajun Xu, Menglu Li, Xiao-Ping Zhang
Department of Engineering Physics Tsinghua University Beijing, China · Tsinghua Shenzhen International Graduate School Tsinghua University Shenzhen, China
Audio deepfake detectors often fail to generalize across speakers, as they learn speaker-identity features rather than synthesis artifacts, known as implicit identity leakage. Existing methods address this but incur architectural complexity or training instability. This paper proposes a dual-granularity orthogonal disentanglement framework enforcing feature independence at two levels: sample-level cosine orthogonality captures directional decorrelation, while batch-level cross-covariance regularization eliminates linear correlations across embedding dimensions. A curriculum disentanglement schedule progressively strengthens the orthogonality constraint without auxiliary networks or adversarial dynamics. Experiments on ASVspoof 2019 LA, ASVspoof 2021 DF, and In-the-Wild datasets demonstrate that the proposed method achieves 1.35%, 7.88%, and 21.58% equal error rates (EER), respectively, surpassing gradient reversal disentanglement by 2.60% absolute on cross-dataset transfer.
Zhuodong Liu, Hugen Lv, Xiangyu Li +1
Beijing Jiaotong University, China · Shanghai Jiao Tong University, China · ITMO University, Russia
Audio deepfakes generated by neural text-to-speech and voice-cloning systems threaten speaker verification and public discourse at scale. The core challenge is cross-dataset generalization: detectors trained on one synthesis pipeline collapse on unseen forgeries. We argue that this failure is primarily because of structural synthetic speech artifacts which are multi-timescale trajectory anomalies. Though every existing detector aggregates a fixed-window frame statistics, this misaligns the architecture with the signal. We propose FlowFake, a Liquid Time-Constant (LTC) architecture whose hidden state evolves via a learned ODE, with per-neuron adaptive time constants simultaneously resolving spectral (10ms) and prosodic (2s) cues. At only 34K parameters FlowFake achieves formal BIBO stability and O(dt^4) integration error. On a four-dataset cross domain benchmark (ASVspoof2019-LA, FakeOrReal, InTheWild, MLAAD), FlowFake reaches 75.29% on ASVspoof2019 trained only on FakeOrReal and 79.97% trained only on MLAAD. It outperforms RawGAT-ST and Whisper-DF on every evaluated pair and matching SSL Wav2vec2 (300x larger) at 0.01% of its parameter count. The source code is available on : https://github.com/GhostRider2023/FlowFake