Quantum Vision Theory Applied to Audio Classification for Deepfake Speech Detection
Authors: Khalid Zaman, Melike Sah, Anuwat Chaiwongyenc, Cem Direkoglu
Organizations: Graduate School of Advanced Science and Technology, Japan Advanced Institute of Science and Technology, Nomi, 923-1292, Ishikawa, Japan · Computer Engineering Department, Cyprus International University, Nicosia, 99258, North Cyprus via Mersin 10, Turkiye · Department of Management Information Systems, Thammasat University, Khlong Luang, Pathum Thani, 12121, Thailand · Electrical and Electronics Engineering Department, Middle East Technical University - Northern Cyprus Campus, Kalkanlı, Güzelyurt, 99738, North Cyprus via Mersin 10, Turkiye
We propose Quantum Vision (QV) theory as a new perspective for deep learning-based audio classification, applied to deepfake speech detection. Inspired by particle-wave duality in quantum physics, QV theory is based on the idea that data can be represented not only in its observable, collapsed form, but also as information waves. In conventional deep learning, models are trained directly on these collapsed representations, such as images. In QV theory, inputs are first transformed into information waves using a QV block, and then fed into deep learning models for classification. QV-based models improve performance in image classification compared to their non-QV counterparts. What if QV theory is applied speech spectrograms for audio classification tasks? This is the motivation and novelty of the proposed approach. In this work, Short-Time Fourier Transform (STFT), Mel-spectrograms, and Mel-Frequency Cepstral Coefficients (MFCC) of speech signals are converted into information waves using the proposed QV block and used to train QV-based Convolutional Neural Networks (QV-CNN) and QV-based Vision Transformers (QV-ViT). Extensive experiments are conducted on the ASVSpoof dataset for deepfake speech classification. The results show that QV-CNN and QV-ViT consistently outperform standard CNN and ViT models, achieving higher classification accuracy and improved robustness in distinguishing genuine and spoofed speech. Moreover, the QV-CNN model using MFCC features achieves the best overall performance on the ASVspoof dataset, with an accuracy of 94.20% and an EER of 9.04%, while the QV-CNN with Mel-spectrograms attains the highest accuracy of 94.57%. These findings demonstrate that QV theory is an effective and promising approach for audio deepfake detection and opens new directions for quantum-inspired learning in audio perception tasks.
Figures & tables
Figure 1: Block diagram of the propose study.
Figure 2: QV block architecture.
Figure 3: Sample information waves. (a) Bonafide spectrogram and its corresponding information waves. (b) Spoof spectrogram and its corresponding information waves.
Figure 4: Workflow of QV block and deep learning of the propose study.
Dataset
Bonafide
Spoof
Total
Duration (sec)
Training
2,580
24,072
26,652
0.65 / 3.42 / 13.19
Evaluation
7,355
63,882
71,237
0.47 / 3.14 / 16.55
Table 1: Statistics of the ASVspoof 2019 Datasets (Durations with Three Values Denoted with Minimum/Average/Maximum).
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
STFT
CNN
8
100
92.30
13.69
STFT
CNN
16
100
91.73
18.68
STFT
CNN
32
100
90.72
16.87
STFT
CNN
64
100
92.38
14.95
STFT
QV-CNN
8
100
91.72
15.35
Table 2: Performance comparison of CNN and QV-CNN classifiers using STFT features with different batch sizes
Figure 5: Confusion matrices of the STFT-based (a) CNN and (b) QV-CNN evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
STFT
ViT
8
100
88.45
13.96
STFT
ViT
16
100
86.34
12.41
STFT
ViT
32
100
85.44
14.56
STFT
ViT
64
100
85.60
12.91
STFT
QV-ViT
8
100
89.67
34.14 (bias)
Table 3: Performance comparison of ViT and QV-ViT classifiers using STFT features with different batch sizes
Figure 6: Confusion matrices of the STFT-based (a) ViT and (b) QV-ViT evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
Mel
CNN
8
100
92.67
11.52
Mel
CNN
16
100
92.87
10.71
Mel
CNN
32
100
92.27
9.00
Mel
CNN
64
100
92.23
14.92
Mel
QV-CNN
8
100
92.11
10.74
Table 4: Performance comparison of CNN and QV-CNN classifiers using Mel spectrogram features with different batch sizes
Figure 7: Confusion matrices of the Mel-based (a) CNN and (b) QV-CNN evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
Mel
ViT
8
100
89.65
18.73
Mel
ViT
16
100
90.26
13.24
Mel
ViT
32
100
–
–
Mel
ViT
64
100
88.29
14.14
Mel
QV-ViT
8
100
92.62
9.80
Table 5: Performance comparison of ViT and QV-ViT classifiers using Mel spectrogram features with different batch sizes
Figure 8: Confusion matrices of the Mel-based (a) ViT and (b) QV-ViT evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
MFCC
CNN
8
100
91.72
15.35
MFCC
CNN
16
100
91.52
16.45
MFCC
CNN
32
100
91.51
14.85
MFCC
CNN
64
100
91.67
14.62
MFCC
QV-CNN
8
100
92.63
9.80
Table 6: Performance comparison of CNN and QV-CNN classifiers using MFCC features with different batch sizes
Figure 9: Confusion matrices of the MFCC-based (a) CNN and (b) QV-CNN evaluated on the ASVspoof evaluation set.
Features
Classifier
Batch Size
Epochs
Evaluation Metrics
Accuracy (%)
EER (%)
MFCC
ViT
8
100
89.67
24.39
MFCC
ViT
16
100
89.06
20.10
MFCC
ViT
32
100
89.15
15.67
MFCC
ViT
64
100
87.07
17.59
MFCC
QV-ViT
8
100
93.49
10.60
Table 7: Performance comparison of ViT and QV-ViT classifiers using MFCC features with different batch sizes
Figure 10: Confusion matrices of the MFCC-based (a) CNN and (b) QV-CNN evaluated on the ASVspoof evaluation set.
Figure 11: Accuracy (a) and ERR (b) for each model per feature
Figure 12: Accuracy a) and ERR b) of each feature per model.
Figure 13: Accuracy of each feature per model for batch size .
Figure 14: ERR of each feature per model for batch size
Study
Approach
Evaluation Metrics
Accuracy (%)
EER (%)
[ 25 ]
MFCC + LSTM-CNN
88.00
–
[ 28 ]
STM (Mel FB) + LCNN
–
9.79
[ 29 ]
Spectrogram + Vision Transformer
–
11.02
[ 30 ]
Compact Convolutional Transformer (CCT)
92.13
–
[ 32 ]
CQCC + DNN
–
13.74
Table 8: Performance comparison with existing studies