SPECTRA: Subspace-Preserving Embedding Calibration, Transport, and Replay for Fully Few-Shot Class-Incremental Audio Classification

Giries Abu Ayoub; Loay Mualem; Simon Korman

cs.SDAug 25, 2026

SPECTRA: Subspace-Preserving Embedding Calibration, Transport, and Replay for Fully Few-Shot Class-Incremental Audio Classification

Authors: Giries Abu Ayoub, Loay Mualem, Simon Korman

Organizations: Department of Computer Science, University of Haifa · Institute for AI, University of Stuttgart · IMPRS-IS

Abstract

Fully few-shot class-incremental audio classification (FFCAC) requires recognizing new sound classes from only a handful of labeled examples per session, without forgetting previously learned classes and without any large base dataset. Existing methods typically freeze a pre-trained audio--language encoder and classify with point prototypes, but they suffer from significant performance degradation throughout the sessions due to generic feature representations. We propose SPECTRA, a framework built on a frozen encoder which adds three components. (i) a lightweight trainable adapter that calibrates the generic embeddings to the task; (ii) subspace feature replay, an exemplar-free anti-forgetting scheme that replays old classes by sampling from the low-rank subspace of their stored features; and (iii) a transductive optimal-transport refinement of prototypes at test time. Our central finding is that the subspace structure of the replay diminishes forgetting and outperforms naive Gaussian replay of equal variance. On three FFCAC benchmarks (NSynth-100, FSC-89, LS-100), SPECTRA improves average accuracy and reduces forgetting over current state-of-the-art methods, and our ablations statistically validate each component.

Figures & tables

Figure 1: (a) The FFCAC setting: sessions arrive over time, each introducing $5$ new classes with $K{=}5$ shots and no base session; the model is evaluated on all classes seen so far (cumulative test set growing $5\!\to\!25$ ). (b) SPECTRA. An audio clip is embedded by a frozen audio–language encoder (PENGI $f$ ), calibrated by a trainable residual-MLP adapter $g$ , and mapped by TAPE’s closed-form orthogonalising transform $\mathbf{P}_{t}$ before a prototypical classifier. Green (training only): each class’s stored support features $\mathcal{M}_{c}$ give a low-rank SVD subspace from which synthetic old-class features $\tilde{\mathbf{x}}_{c}$ are sampled and passed through the same head, a rehearsal that counters forgetting without storing audio. Purple, dashed (inference only): a transductive Sinkhorn step refines the prototypes from the unlabelled query batch.

Figure 2: Replay geometry across sessions. As more classes are added, low-rank subspace replay remains class-aligned, whereas Gaussian replay produces overlapping clouds.

Method	S0	S1	S2	S3	S4	AA $\uparrow$	PD $\downarrow$
$\text{iCaRL}_{\text{CVPR'20}}$ [ 15 ]	71.70	53.51	53.66	49.07	49.48	55.48	40.08
$\text{PODNET}_{\text{ECCV'20}}$ [ 5 ]	71.87	44.89	43.58	42.93	40.97	48.85	35.57
$\text{DER}_{\text{CVPR'21}}$ [ 22 ]	74.40	61.42	60.06	53.73	44.42	58.81	38.95
$\text{CEC}_{\text{CVPR'21}}$ [ 23 ]	76.13	58.15	53.80	48.26	44.34	56.14	37.49
$\text{FACT}_{\text{CVPR'22}}$ [ 24 ]	74.97	51.94	51.43	46.54	43.45	53.27	38.65
$\text{PAN}_{\text{TMM'23}}$ [ 13 ]	76.71	58.38	53.92	48.44	44.48	56.39	37.72

Table 1: Comparison of different methods on NSynth-100.

Method	S0	S1	S2	S3	S4	AA $\uparrow$	PD $\downarrow$
$\text{PAN}_{\text{TMM'23}}$ [ 13 ]	41.48	23.72	18.08	15.27	12.25	22.16	37.72
$\text{EDE}_{\text{Interspeech'24}}$ [ 17 ]	53.25	37.65	35.59	32.69	27.60	37.36	30.05
$\text{AISP}_{\text{TASLP'25}}$ [ 18 ]	62.66	48.92	43.81	37.73	33.43	45.31	29.23
$\text{CLAP}_{\text{ICASSP'23}}$ [ 6 ]	49.64	37.73	28.55	25.42	23.70	33.01	25.94
$\text{COOP}_{\text{IJCV'22}}$ [ 26 ]	63.32	37.75	27.50	22.07	18.64	33.86	44.69
$\text{COCOOP}_{\text{CVPR'22}}$ [ 25 ]	64.65	37.62	26.63	21.22	16.85	33.39	47.80

Table 2: Comparison of different methods on FSC-89.

NSynth-100 ( $10$ -way $\times\,10$ -session)
Method	S0	S1	S2	S3	S4	S5	S6	S7	S8	S9	AA $\uparrow$	PD $\downarrow$
TAPE	95.6	92.2	90.8	89.6	88.4	86.5	85.1	84.0	82.9	81.5	87.66	14.11
SPECTRA	97.6	95.6	94.3	93.4	92.7	91.5	90.8	90.1	89.6	88.9	92.45	8.77
LS-100 ( $10$ -way $\times\,10$ -session)
TAPE	90.1	85.4	80.7	77.9	75.2	73.4	70.7	69.0	67.2	65.7	75.53	24.43
SPECTRA	95.1	91.2	88.1	85.7	83.7	82.3	80.4	79.7	78.6	77.8	84.26	17.31

Table 3: Per-session accuracy (%) under harder protocols:

10

-way

\times\,10

-session on NSynth-100 and LS-100, and

10

-way

\times\,8

-session on FSC-89 (FSC-89 has only

89

classes, capping it at

8

sessions);

30

seeds.

Method	S0	S1	S2	S3	S4	AA $\uparrow$	PD $\downarrow$
$\text{PAN}_{\text{TMM'23}}$ [ 13 ]	85.70	52.20	39.17	32.95	29.88	47.98	37.72
$\text{EDE}_{\text{Interspeech'24}}$ [ 17 ]	91.90	70.23	54.21	46.97	45.94	61.85	45.96
$\text{AISP}_{\text{TASLP'25}}$ [ 18 ]	91.20	70.26	53.71	47.08	45.15	61.48	46.05
$\text{CLAP}_{\text{ICASSP'23}}$ [ 6 ]	18.96	9.06	5.91	4.75	3.37	8.41	15.59
$\text{COOP}_{\text{IJCV'22}}$ [ 26 ]	49.14	27.64	18.48	14.48	11.01	24.15	38.14
$\text{COCOOP}_{\text{CVPR'22}}$ [ 25 ]	46.90	24.32	15.75	10.86	7.84	21.14	39.06

Table 4: Comparison of different methods on LS-100.

Configuration	NSynth-100		FSC-89
Configuration	AA $\uparrow$	PD $\downarrow$	AA $\uparrow$	PD $\downarrow$
TAPE (baseline)	93.5	5.6	69.2	20.0
+ Adapter	95.5	4.3	70.0	19.0
+ Transport (OT)	95.8	4.7	70.5	19.9
w/ Gaussian replay	95.8	4.6	70.8	19.3
w/ subspace replay	96.5	3.2	71.1	18.8

Table 5: Component ablation (AA / PD, 50 seeds).

Few-Shot Audio Classification

Few-shot audio classification learns to recognize sound classes from only a small number of labeled examples.

See all 18 papers

Few-shot Class-variable Incremental Audio Classification via Prototype Adaptation and Pseudo Class-variable Training
Jun 8, 2026Yanxiong Li, Guoqing Chen, Qianqian Li +1Class-Incremental Learning Few-Shot Audio Classification
School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China
Few-Shot Open-Set Audio Classification Using Attention Information-Fused Prototypes
Jul 1, 2026Yanxiong Li, Jiaxin Tan, Qianqian Li +3Few-Shot Audio Classification Prototype-Based Classification
School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510640, China · Faculty of Information Technology and Communication Sciences, Tampere University, FI-33720 Tampere, Finland
DF-PALM: Dynamic Fusion Prompt Learning for Audio--Language Models
Oct 18, 2026Yulin Sun, Shenghua Li, Yong Dou +3Few-Shot Audio Classification Prompt Learning
College of Computer Science and Technology, National University of Defense Technology, Changsha, China. · State Key Laboratory of Complex & Critical Software Environment

Jun 8, 2026J/K move · Enter open · S save

Few-shot Class-variable Incremental Audio Classification via Prototype Adaptation and Pseudo Class-variable Training

Yanxiong Li, Guoqing Chen, Qianqian Li, Sen Huang

School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China

Class-Incremental Learning Few-Shot Audio Classification Prototype-Based Classification Few-Shot Learning Audio Classification

Open paper AskPDF arXiv

In the task of few-shot class-incremental audio classification, the number of classes is assumed to always increase without considering the possibility of decrease. However, the number of classes generally increases or decreases in practice. In this paper, we investigate a problem of Few-shot Class-variable Incremental Audio Classification (FCIAC), in which the number of classes increases or decreases. We propose a FCIAC method using prototype adaptation and pseudo class-variable training. The model in our method consists of an encoder and a classifier. The classifier is initialized by a class-variable prototype adaptation network, whose structure dynamically changes with the change of classes. In addition, we design a pseudo class-variable training strategy to enhance the model's adaptability to changing classes. Experiments on three public datasets show that our method exceeds previous methods in average accuracy. The code is at: https://github.com/cgq2971-afk/FCIAC.

Abstract

Few-Shot Audio Classification

Explore similar work

Few-shot Class-variable Incremental Audio Classification via Prototype Adaptation and Pseudo Class-variable Training

Few-Shot Open-Set Audio Classification Using Attention Information-Fused Prototypes

DF-PALM: Dynamic Fusion Prompt Learning for Audio--Language Models