Metadata-Conditioned Audio Transformers for Adaptive Respiratory Sound Classification
Organizations: Computer Engineering and Informatics Department, University of Patras, Patras, Greece
Abstract
Respiratory sound classification from auscultation recordings is confounded by heterogeneous acquisition conditions, including recording device, auscultation site, and patient characteristics, that introduce systematic variability unrelated to the underlying pathology. Although such metadata is often available, existing approaches either ignore it or rely on naïve fusion strategies that do not model how distinct metadata sources influence different components of the acoustic representation. We propose a family of metadata-conditioning mechanisms for the Audio Spectrogram Transformer (AST), progressing from gated residual fusion to Feature-wise Linear Modulation (FiLM) adapted to the Transformer architecture. We further introduce two novel variants: Token-Aware FiLM (TAFiLM), which applies distinct modulation to classification and patch tokens, and Soft-Factorized FiLM (SoftFiLM), which learns soft masks to align metadata factors with disentangled feature subspaces. Experiments on the ICBHI benchmark demonstrate consistent gains across all conditioning strategies. SoftFiLM achieves 64.11% on the official 4-class task (+3.94% over the baseline) and 72.40% on the 2-class task, establishing a new state-of-the-art on the 2-class task and competitive performance on the 4-class benchmark.
Figures & tables
| Method | Backbone | Pretrain | Venue | Meta | (%) | (%) | Score (%) |
| 4-class evaluation | |||||||
| Kim et al. (SG-SCL) [ 12 ] | AST | IN + AS | ICASSP’24 | ✓ | 79.87 | 43.55 | 61.71 |
| Kim et al. (RepAugment) [ 17 ] | AST | IN + AS | EMBC’24 | ✗ | 82.47 | 40.55 | 61.51 |
| Daisuke et al. (M2D) [ 18 ] | M2D ViT | AS | TASLP’24 | ✗ | 81.51 | 45.08 | 63.29 |
| Kim et al. (BTS) [ 10 ] | CLAP | LA | INTERSPEECH’24 | ✓ | 81.40 | 45.67 | 63.54 |
| Wang et al. (HTS-AT) [ 19 ] | HTS-AT | IN + AS | ICSIP’24 | ✗ | 79.61 | 48.77 | 64.19 |
| Method | Last 2 | Last 6 | All 12 |
|---|---|---|---|
| Baseline AST | 60.17 (no conditioning) | ||
| FiLM | 61.02 | 61.68 | 62.37 |
| TAFiLM | 61.89 | 62.74 | 63.52 |
| SoftFiLM | 62.45 | 63.31 | 64.11 |
Respiratory Sound Classification
Respiratory sound classification uses audio and machine learning to identify breathing events and support respiratory disease assessment.
See all 29 papers