Quantum Vision Theory Applied to Audio Classification for Deepfake Speech Detection
Authors: Khalid Zaman, Melike Sah, Anuwat Chaiwongyenc, Cem Direkoglu
Organizations: Graduate School of Advanced Science and Technology, Japan Advanced Institute of Science and Technology, Nomi, 923-1292, Ishikawa, Japan · Computer Engineering Department, Cyprus International University, Nicosia, 99258, North Cyprus via Mersin 10, Turkiye · Department of Management Information Systems, Thammasat University, Khlong Luang, Pathum Thani, 12121, Thailand · Electrical and Electronics Engineering Department, Middle East Technical University - Northern Cyprus Campus, Kalkanlı, Güzelyurt, 99738, North Cyprus via Mersin 10, Turkiye
We propose Quantum Vision (QV) theory as a new perspective for deep learning-based audio classification, applied to deepfake speech detection. Inspired by particle-wave duality in quantum physics, QV theory is based on the idea that data can be represented not only in its observable, collapsed form, but also as information waves. In conventional deep learning, models are trained directly on these collapsed representations, such as images. In QV theory, inputs are first transformed into information waves using a QV block, and then fed into deep learning models for classification. QV-based models improve performance in image classification compared to their non-QV counterparts. What if QV theory is applied speech spectrograms for audio classification tasks? This is the motivation and novelty of the proposed approach. In this work, Short-Time Fourier Transform (STFT), Mel-spectrograms, and Mel-Frequency Cepstral Coefficients (MFCC) of speech signals are converted into information waves using the proposed QV block and used to train QV-based Convolutional Neural Networks (QV-CNN) and QV-based Vision Transformers (QV-ViT). Extensive experiments are conducted on the ASVSpoof dataset for deepfake speech classification. The results show that QV-CNN and QV-ViT consistently outperform standard CNN and ViT models, achieving higher classification accuracy and improved robustness in distinguishing genuine and spoofed speech. Moreover, the QV-CNN model using MFCC features achieves the best overall performance on the ASVspoof dataset, with an accuracy of 94.20% and an EER of 9.04%, while the QV-CNN with Mel-spectrograms attains the highest accuracy of 94.57%. These findings demonstrate that QV theory is an effective and promising approach for audio deepfake detection and opens new directions for quantum-inspired learning in audio perception tasks.
Figures & tables
Figure 1: Block diagram of the propose study.
Figure 2: QV block architecture.
Figure 3: Sample information waves. (a) Bonafide spectrogram and its corresponding information waves. (b) Spoof spectrogram and its corresponding information waves.
Figure 4: Workflow of QV block and deep learning of the propose study.
Dataset
Bonafide
Spoof
Total
Duration (sec)
Training
2,580
24,072
26,652
0.65 / 3.42 / 13.19
Evaluation
7,355
63,882
71,237
0.47 / 3.14 / 16.55
Table 1: Statistics of the ASVspoof 2019 Datasets (Durations with Three Values Denoted with Minimum/Average/Maximum).
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
STFT
CNN
8
100
92.30
13.69
STFT
CNN
16
100
91.73
18.68
STFT
CNN
32
100
90.72
16.87
STFT
CNN
64
100
92.38
14.95
STFT
QV-CNN
8
100
91.72
15.35
Table 2: Performance comparison of CNN and QV-CNN classifiers using STFT features with different batch sizes
Figure 5: Confusion matrices of the STFT-based (a) CNN and (b) QV-CNN evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
STFT
ViT
8
100
88.45
13.96
STFT
ViT
16
100
86.34
12.41
STFT
ViT
32
100
85.44
14.56
STFT
ViT
64
100
85.60
12.91
STFT
QV-ViT
8
100
89.67
34.14 (bias)
Table 3: Performance comparison of ViT and QV-ViT classifiers using STFT features with different batch sizes
Figure 6: Confusion matrices of the STFT-based (a) ViT and (b) QV-ViT evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
Mel
CNN
8
100
92.67
11.52
Mel
CNN
16
100
92.87
10.71
Mel
CNN
32
100
92.27
9.00
Mel
CNN
64
100
92.23
14.92
Mel
QV-CNN
8
100
92.11
10.74
Table 4: Performance comparison of CNN and QV-CNN classifiers using Mel spectrogram features with different batch sizes
Figure 7: Confusion matrices of the Mel-based (a) CNN and (b) QV-CNN evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
Mel
ViT
8
100
89.65
18.73
Mel
ViT
16
100
90.26
13.24
Mel
ViT
32
100
–
–
Mel
ViT
64
100
88.29
14.14
Mel
QV-ViT
8
100
92.62
9.80
Table 5: Performance comparison of ViT and QV-ViT classifiers using Mel spectrogram features with different batch sizes
Figure 8: Confusion matrices of the Mel-based (a) ViT and (b) QV-ViT evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
MFCC
CNN
8
100
91.72
15.35
MFCC
CNN
16
100
91.52
16.45
MFCC
CNN
32
100
91.51
14.85
MFCC
CNN
64
100
91.67
14.62
MFCC
QV-CNN
8
100
92.63
9.80
Table 6: Performance comparison of CNN and QV-CNN classifiers using MFCC features with different batch sizes
Figure 9: Confusion matrices of the MFCC-based (a) CNN and (b) QV-CNN evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
MFCC
ViT
8
100
89.67
24.39
MFCC
ViT
16
100
89.06
20.10
MFCC
ViT
32
100
89.15
15.67
MFCC
ViT
64
100
87.07
17.59
MFCC
QV-ViT
8
100
93.49
10.60
Table 7: Performance comparison of ViT and QV-ViT classifiers using MFCC features with different batch sizes
Figure 10: Confusion matrices of the MFCC-based (a) CNN and (b) QV-CNN evaluated on the ASVspoof evaluation set.
Figure 11: Accuracy (a) and ERR (b) for each model per feature
Figure 12: Accuracy a) and ERR b) of each feature per model.
Figure 13: Accuracy of each feature per model for batch size .
Figure 14: ERR of each feature per model for batch size
Study
Approach
Evaluation Metrics
Accuracy (%)
EER (%)
[ 25 ]
MFCC + LSTM-CNN
88.00
–
[ 28 ]
STM (Mel FB) + LCNN
–
9.79
[ 29 ]
Spectrogram + Vision Transformer
–
11.02
[ 30 ]
Compact Convolutional Transformer (CCT)
92.13
–
[ 32 ]
CQCC + DNN
–
13.74
Table 8: Performance comparison with existing studies
Quantum machine learning has emerged as a promising tool for pattern recognition, yet many audio-focused approaches still treat spectrograms as generic images and do not explicitly exploit their time-frequency structure. We propose Q-Patch, a quantum feature map tailored to audio that encodes local time-frequency patches from mel-spectrograms into quantum states using shallow, hardware-efficient circuits with adjacency-aware entanglement. Each selected patch is summarized by a compact four-dimensional acoustic descriptor and mapped to a four-qubit circuit with depth at most three, enabling practical quantum kernel construction under near-term constraints. We evaluate Q-Patch on an audio spoofing detection task using a controlled, balanced protocol and compare it with size-matched classical baselines. Q-Patch improves discrimination between bona fide and spoofed samples, achieving an area under the receiver operating characteristic curve (AUROC) of 0.87, compared with 0.82 for a radial basis function support vector machine (RBF-SVM) trained on the same patch-level features. Kernel-space analysis further reveals a clear class structure, with cross-class similarity around 0.615 and within-class self-similarity of 1.00. Overall, Q-Patch provides a practical framework for incorporating time-frequency-aware representations into quantum kernel learning for audio authenticity assessment in low-resource settings.
Lisan Al Amin, Rakib Hossain, Mahbubul Islam +2
Potomac Quantum, USA · United International University, Dhaka, Bangladesh · University of Maryland, College Park, MD, USA +2
Synthetic speech detection is critical for audio security, but performance can degrade when labeled data are scarce and evaluation conditions differ from training. This study examines quantum kernel methods and lightweight neural models for cross-corpus audio deepfake detection under limited training data. We compare a Quantum Support Vector Machine (QSVM), a classical support vector machine (SVM), and a multilayer perceptron (MLP), all trained on frozen wav2vec 2.0 embeddings using a strict budget of 200 training samples. To match the qubit budget of near-term quantum hardware, embeddings are reduced to four dimensions using principal component analysis, and all models use the same reduced features. Experiments on ASVspoof 2019, ASVspoof 5, the ADD 2023 Challenge, and the In-the-Wild dataset show that under severe domain shift from ASVspoof 2019 to ADD 2023, the MLP degrades to near-random performance, with an area under the curve of approximately 50% and an equal error rate of 50.0%. In contrast, the QSVM maintains meaningful discrimination, achieving an area under the curve of 76.0% and an equal error rate of 27.0%. This advantage is not consistent across transfer directions. When trained on ADD 2023, the QSVM falls below chance on two of three transfers, while the MLP performs better. These results suggest that quantum kernel methods can be competitive under severe cross-corpus shifts and strict low-resource constraints, but do not provide a consistent advantage under near-domain transfer. We interpret these findings as an empirical characterization of quantum kernel inductive bias under distribution shift, rather than evidence of quantum advantage, since the four-qubit kernel can be simulated exactly on classical hardware.
Lisan Al Amin, Lei Zhang, Vandana P. Janeja
University of Maryland, Baltimore County, Baltimore, Maryland, USA
Neural audio codecs discretize speech via residual vector quantization (RVQ), forming a coarse-to-fine hierarchy across quantizers. While codec models have been explored for representation learning, their discrete structure remains underutilized in speech deepfake detection. In particular, different quantization levels capture complementary acoustic cues, where early quantizers encode coarse structure and later quantizers refine residual details that reveal synthesis artifacts. Existing systems either rely on continuous encoder features or ignore this quantizer-level hierarchy. We propose a hierarchy-aware representation learning framework that models quantizer-level contributions through learnable global weighting, enabling structured codec representations aligned with forensic cues. Keeping the speech encoder backbone frozen and updating only 4.4% additional parameters, our method achieves relative EER reductions of 46.2% on ASVspoof 2019 and 13.9% on ASVspoof5 over strong baselines.
Jinyang Wu, Zihan Pan, Qiquan Zhang +2
Agency for Science, Technology and Research (A*STAR), Singapore · The University of New South Wales, Australia