The performance of speech classification systems such as speech emotion recognition or pathological speech detection can degrade substantially when there is a domain shift between training and test data. This shift can arise due to e.g., differences in recording setup, acoustic environment, speaker, language, or speaking style. Adversarial domain adaptation addresses this problem by encouraging representations that are less sensitive to domain-specific differences. However, many adversarial approaches rely on training a separate domain discriminator, which can be unstable and difficult to train. In this paper, we propose to use a discriminator-free adversarial learning network (DALN) for unsupervised domain adaptation in speech classification tasks. DALN reuses the task classifier as an implicit discriminator and aligns domains through the nuclear-norm Wasserstein discrepancy on classifier outputs. We extensively evaluate the performance of DALN on intra-lingual, cross-lingual, and noisy adaptations for speech emotion recognition and pathological speech detection. Extensive experimental results show that DALN generally achieves the best performance across the considered tasks and domain shifts, outperforming discriminator-based domain adaptation approaches.
Figures & tables
Figure 1: DALN architecture for robust speech classification. Source and target utterances are jointly processed by a frozen SSL model, statistical pooling, and a trainable MLP encoder f . The task classifier c is supervised by the source classification loss Lcls and reused as an implicit domain discriminator. The nuclear-norm Wasserstein loss Lnwd is computed from the source and target prediction matrices Cs and Ct . Through gradient reversal, the classifier maximizes this loss while the encoder minimizes it, aligning the domains without an additional domain discriminator. Solid lines denote the forward pass and dashed lines the corresponding backward paths.
Corpus
Lang.
Style
Spk.
Utt.
Cl.
CREMA-D [ 21 ]
EN
acted, scripted
91
7442
6
IEMOCAP [ 22 ]
EN
acted, scripted/impro.
10
5531
4
MSP-PODCAST [ 23 ]
EN
natural, podcast
102
12053
4
EmoFilm-EN [ 28 ]
EN
acted, film
77
343
5
CaFE [ 24 ]
FR
acted, scripted
12
936
7
EMOVO [ 25 ]
IT
acted, scripted
6
588
7
Table 1: Characteristics of the considered datasets in terms of language (Lang.), speaking style, number of speakers (Spk.), number of utterances (Utt.), and number of classes (Cl.).
Adaptation
Source
Target
DANN
CDAN
Proposed
CREMA-D → IEM
86.4
50.1
+ 5.3
+ 6.8
+ 7.6
CREMA-D → MSP
86.4
39.2
− 2.3
− 0.5
− 0.3
IEM → CREMA-D
66.4
57.6
+ 0.8
+ 7.3
+ 16.7
IEM → MSP
66.4
38.5
− 1.1
+ 0.3
+ 1.2
MSP → CREMA-D
37.4
36.8
− 6.5
− 8.1
+ 4.0
MSP → IEM
37.4
38.0
+ 3.4
+ 6.8
+ 14.4
Table 2: Change in target UAR (%) for the six intra-lingual adaptations using DANN, CDAN, and the proposed approach. The UAR of the baseline unadapted model on the source test set (Source) and on the target test set (Target) is also presented.
Direction
Source
Target
DANN
CDAN
Proposed
CaFE → CREMA-D
73.9
29.4
+ 5.2
+ 22.3
+ 11.1
CaFE → EmoDB
72.0
51.3
− 25.6
− 0.1
+ 13.2
CaFE → EMOVO
72.5
34.5
− 2.6
+ 2.8
+ 8.5
CREMA-D → CaFE
75.2
42.6
− 12.1
− 9.6
− 3.5
CREMA-D → EmoDB
75.2
58.0
− 6.7
− 6.9
+ 6.5
CREMA-D → EMOVO
75.2
42.0
− 1.8
− 6.9
− 7.9
Table 3: Change in target UAR (%) for the 12 cross-lingual adaptations using DANN, CDAN, and the proposed approach. The UAR of the baseline unadapted model on the source test set (Source) and on the target test set (Target) is also presented.
Corpus
Source
Target
DANN
CDAN
Proposed
CaFE
72.6
37.1
+ 0.0
+ 12.3
+ 15.5
CREMA-D
75.2
54.4
+ 5.4
+ 1.9
+ 0.2
EmoDB
91.7
76.9
− 19.6
− 11.9
− 4.4
EmoFilm-EN
63.0
48.5
+ 3.1
+ 4.2
+ 4.8
EMOVO
60.5
49.0
− 3.9
− 0.4
+ 3.0
IEMOCAP
66.4
56.4
+ 4.9
+ 5.3
+ 5.4
Table 4: Change in target UAR (%) for the noisy adaptations on SER datasets using DANN, CDAN, and the proposed approach. The UAR of the baseline unadapted model on the source test set (Source) and on the target test set (Target) is also presented.
Corpus
Source
Target
DANN
CDAN
Proposed
PC-GITA
86.3
62.1
+ 10.2
+ 10.7
+ 14.6
Std
1.6
1.1
0.7
Table 5: Change in target speaker-level accuracy (%) for the noisy adaptations on PC-GITA using DANN, CDAN, and the proposed approach. The accuracy of the baseline unadapted model on the source test set (Source) and on the target test set (Target) is also presented.
As Alzheimer's disease (AD) has increasingly become a major global public health issue, speech-based AD detection has attracted widespread attention. However, most existing methods are trained and evaluated on a single dataset, often leading to severe cross-domain performance degradation due to reliance on dataset-specific artifacts rather than disease-related speech cues. In real-world applications, reliable Alzheimer's disease detection requires models that are robust to variations in recording environments, speakers and data collection conditions. To address this challenge, this paper adopts unsupervised domain adaptation to learn robust, domain-invariant feature representations in the absence of target-domain diagnosis labels. On this basis, a novel unsupervised domain adaptation method, Iterative Adversarial Self-Training (IAST), is proposed. Results demonstrate that IAST significantly improves the generalization ability and robustness under various cross-domain settings.
This paper explores the effectiveness of domain adaptation techniques when using convolutional neural network (CNN)-based and transformer-based feature representations for acoustic scene classification. Two well-known domain adaptation techniques, namely domain adversarial neural network (also called DANN) and conditional domain adversarial network (also called CDAN) are evaluated under various domain shifts. Our study indicates that DANN provides effective domain adaptation fairly consistently for both feature extractors. On the other hand, CDAN provides effective domain adaptation only for CNN-based feature extractors. The study gives insights into how domain adaptation methods may need to be tailored to the underlying feature representation. Experimental evaluation with multiple devices on the DCASE 2020 dataset supports the observations.
Abhishek dileep, Shubham Sharma, Padmanabhan Rajan
School of Computing and Electrical Engineering Indian Institute of Technology Mandi, Himachal Pradesh, India
Domain Generalization (DG) aims to develop models that remain robust to conditions unseen during training. While DG has been systematically studied in computer vision through controlled benchmarks and diverse distribution shifts, its evaluation in spoken language recognition remains less structured. Existing speech datasets provide valuable benchmarks for robustness to real-world acoustic conditions, but are primarily designed around specific scenarios and large scale rather than as general-purpose tools for systematically constructing and evaluating domain shifts. In this work, we introduce a smallscale speech dataset and evaluation protocol for controlled studies of acoustic domain shifts. It enables the evaluation of spoken language recognition models under a variety of acousitc domain shifts. We introduce the speech modality into the DomainBed Domain Generalization framework and evaluate three domain generalization algorithms. We show that in-domain performance is not a reliable predictor of cross-domain robustness and verify that explicit domain invariant algorithms such as MMD or DANN algorithms do not outperform ERM. We further validate the generality of these findings on MMS-LID-126, a state-of-the-art spoken language identification system. We release the code.
Francois Derrida, Raphaël Duroselle, Thomas Courtat +1
X · THALES, CortAIx-Labs · AMIAD Pˆole Recherche - Institut Polytechnique de Paris Palaiseau, France