Parameter isolation with domain-specific experts for incremental audio classification
Authors: Jongyeon Park, Do-Hyeon Lim, Sang-won Park, Hong Kook Kim, Kyungdeuk Ko, Hyeongcheol Geum, Jeong Eun Lim
Organizations: Dept. of AI Convergence, Gwangju Institute of Science and Technology (GIST) · Dept. of EECS, Gwangju Institute of Science and Technology (GIST) · AunionAI Co., Ltd., Gwangju, Korea · AI Lab., R&D Center, Hanwha Vision, Seongnam, Korea
To successfully deploy a model in time-varying environments such as streaming data prediction and sensing control, domain-incremental learning (DIL) has attracted attention since it aims to adapt a previously trained model to newly arriving domains, while reserving knowledge from earlier domains without accessing their data. Incremental learning across domains can be regarded as a recurrent update, in which the current model is obtained by updating the model carried over from previous domains. Conventional DIL approaches that rely on domain-invariant feature learning and weight regularization gradually overwrite or constrain parameters learned in previous domains, leading to catastrophic forgetting. Instead, this paper proposes a new domain-specific parameter-isolation architecture that retains all past domains. The proposed architecture mitigates catastrophic forgetting through a full-order recurrent update, constructing a new expert using domain-specific data conditioned on all previously frozen models. To achieve this, we incorporate data-free generative replay to reconstruct previous-domain data and cross-domain feature generation to recover later expert features missing from earlier domain samples. Finally, we apply the proposed model architecture to domain-agnostic incremental learning for audio classification, as defined in the DCASE 2026 Challenge Task 7. Consequently, we achieve micro and macro accuracies of 78.4% and 78.9%, respectively, representing increases of 33 and 25 percentage points over the Challenge baseline. Ablation studies are conducted to examine the effectiveness of each processing component in terms of classification accuracy.
Figures & tables
Fig. 1: Conceptual diagram for representing three different realizations of domain incremental update: (a) first-order, (b) anchored, and (c) our full-order update.
Fig. 2: Domain-agnostic inference pipeline for T=3 . Frozen experts E1,E2,E3 emit features ϕ1,ϕ2,ϕ3 , concatenated into z and mapped to class probabilities p by a single classifier G .
System
E1 ( D1 )
E2 ( D2 )
E3 ( D3 )
FoIL-FineTune
CNN14
CNN14 inv
CNN14 inv
FoIL
CNN14
CNN14 inv , CRNN
CNN14 inv , CNN14 s
FoIL-FDY
CNN14
CNN14 inv , FDY-CNN14
CNN14, CNN14 s
FoIL-NoReplay
CNN14
CNN14, FDY-CNN14
CNN14, CNN14 s
Table 1: List of four different systems depending on the configuration of domain experts. Each expert is a CNN14, optionally paired with one additional model except for FoIL-FineTune; a subscript of ‘inv’ denotes a DeepInversion-based replay variant and ‘s’ means a model trained from scratch.
D2
D3
Dev-test Avg.
System
micro
macro
micro
macro
micro
macro
Challenge Baseline
54.77
58.95
36.23
47.34
45.50
53.15
OR-KDL
–
79.73
–
66.55
–
73.14
FoIL-FineTune
68.86
71.94
70.10
70.15
69.48
71.05
FoIL
79.97
81.88
75.19
73.97
77.58
77.92
FoIL-Ensemble
81.69
83.62
75.06
74.22
78.38
78.92
Table 2: Comparison of micro and macro average classification accuracy (%) between our proposed systems and comparison systems, including the Challenge baseline, for D2 and D3 domains.
Components
Macro Acc. (%)
Variant
Replay
MLP-Gen
Feat-Mean
D2
D3
Avg.
FoIL-None
72.88
72.05
72.47
FoIL-R
✓
74.98
74.19
74.59
FoIL-M
✓
79.22
74.42
76.82
FoIL-RM
✓
✓
81.88
73.97
77.92
FoIL-RF
✓
✓
65.33
75.50
70.42
Table 3: Ablation results of the macro accuracy according to different combinations of the processing components in the proposed FoIL framework.
Fig. 3: Plot of the distribution of cosine similarity scores calculated between the ground truth and generated feature.
This technical report presents submission systems for Task 7(domain-incremental audio classification) of the DCASE 2026 Challenge. The main obstacle is that, the system is unable to access to past or future domain's data at once. We approached domain-incremental learning (DIL) as a frozen-feature replay problem. At each incremental stage, one or two compact experts are trained and then kept fixed; at the final stage, the penultimate features from all frozen experts are concatenated and used to train a lightweight per-class prototype classifier solely on cached features. This design prevents catastrophic forgetting by preserving each expert models at inference. To retain earlier-domain knowledge without storing raw audio, some experts were trained with DeepInversion-based generative replay. A cross-stage regression imputer was trained to fill the expert feature slots that did not yet exist at an ealier stage. We submit four fully DIL-compliant systems: three systems based on diverse frozen five-expert backbones and their cross-stack ensemble achieving 78.15% micro / 77.03% macro on the development set, outperforming every individual backbone on both evaluations.
Jongyeon Park, Do-Hyeon Lim, Sang-won Park +4
Dept. of AI Convergence, Gwangju Institute of Science and Technology, Gwangju 61005, Korea · Dept. of EECS, Gwangju Institute of Science and Technology, Gwangju 61005, Korea · AI Lab., R&D Center, Hanwha Vision, Seongnam-si, Gyeonggi-do 13488, Korea
We propose a domain-incremental learning framework for generative speech enhancement (SE) that learns from a sequence of datasets or domains recorded under diverse acoustic conditions. Fine-tuning a pretrained model on continuously evolving domains leads to catastrophic forgetting of previously acquired knowledge, while zero-shot generalization often fails to adequately adapt to unseen domains. To address these challenges, we first develop a novel language model-based generative SE model that we then use as a pretrained backbone and incrementally adapt it to acoustically mismatched domains using lightweight domain-specific Low-Rank Adaptation. The proposed framework enables the model to acquire enhancement capabilities for new domains while preserving performance on previously learned domains. Evaluated on four heterogeneous speech datasets, our approach effectively adapts to new domains without forgetting previously learned domains.
Manjunath Mulimani, Annamaria Mesaros, Minje Kim +1
Aalborg University, Department of Electronic Systems, Denmark · Tampere University, Signal Processing Research Centre, Finland · University of Illinois Urbana-Champaign, Siebel School of Computing and Data Science, USA
In this work we introduce a novel approach to domain incremental learning, adapting models over time to evolving, non-stationary data. In contrast to other works, we do not attempt to avoid catastrophic forgetting, but rather allow it and exploit it. Our model combines a main task head with a self-supervised masked autoencoder (MAE) head. We then learn domain-specific LoRA adapters during incremental training. Each adapter specializes to its domain, naturally inducing forgetting on other domains in both heads. At inference, we perform online test-time training on the self-supervised MAE head to identify which LoRAs best matches the current input, so the model can `remember' the domain again. Our scheme is especially well-suited to real-world streaming data, such as video, where consecutive samples are highly correlated and domain shifts are gradual. We demonstrate our method on domain-incremental action recognition and semantic segmentation tasks.