RespCCT and FreqCCT: Compact, Explainable and Cost-Aware Convolutional Transformers for Multi-Class Respiratory Disease Classification
Authors: Auntor Chakma, Mst. Farhana Akter, Abu Henaf Rashid Ahmad Mufti, Asmana Yasmin
Organizations: ELITE Research Lab, New York, NY, USA · Department of Computer Science and Engineering, East Delta University, Abdullah Al Noman Road, Noman Society, East Nasirabad, Khulshi, Chattogram, 4209, Bangladesh · Department of Computer Science and Engineering, United International University, United City, Madani Avenue, Badda, Dhaka, 1212, Bangladesh
Chest auscultation depends on the listener, and software that classifies lung sounds could give a second reading where trained listeners are scarce. The most accurate published classifiers fine-tune large pretrained networks, and their accuracy is rarely reported with running cost or paired statistics. This study makes three contributions. (i) It proposes RespCCT and FreqCCT, two compact convolutional transformers of 173,187 and 124,999 parameters that learn from random initialization on mel, constant-Q and cepstral images of 5 s frames from three public databases, ICBHI, KAUH and RD@TR, with 1,665 audio files in seven classes. (ii) It compares both models with five ImageNet-pretrained networks and larger distilled variants on one test set, using paired bootstrap intervals. (iii) It measures the cost of every pipeline stage on one CPU thread, tests whether attribution maps depend on the trained weights, and probes whether the learned features reveal the recording source. On the test fold, RespCCT reaches an accuracy of 98.13% and a macro F1 of 0.970, and FreqCCT reaches 97.58% and 0.951. Over five folds, RespCCT with three seeds reaches 97.65% and 0.959, and FreqCCT with one seed reaches 96.95% and 0.941. The two strongest pretrained networks reach a macro F1 of 0.984, whereas RespCCT needs 2.0 to 64.6 times less weight storage. Computing the three input images takes longer than running either model, and a linear probe identifies the source database with 0.958 accuracy. A team can weigh accuracy against storage and time before device testing.
Figures & tables
Figure 1: Overview of the study, covering data pooling and framing, three time–frequency images per frame, the compact and reference models and the evaluation steps.
RespireNet, ResNet34 with ImageNet weights, device-specific fine-tuning
ICBHI Score 56.20%
Needs ImageNet weights
Training from random initialization
2022
Nguyen and Pernkopf Nguyen and Pernkopf [2022]
Fine-tuned ResNets, co-tuning, device correction
ICBHI Score 58.29%
Needs ImageNet weights, source information not measured
Source probe on learned features
2023
Bae et al. Bae et al. [2023]
Pretrained AST, patch-mix contrastive learning
ICBHI Score 62.37%
Large pretrained model
Training from random initialization
2024
He et al. He et al. [2024]
Multi-view patches of one mel spectrogram, gated fusion
ICBHI Score 66.55%
One transform, cost not reported
Three transforms, removed one at a time
2024
Zhang et al. Zhang et al. [2024a]
Dual-channel CNN–LSTM on MFCC
Accuracy 99.01% (ICBHI, random split)
One input representation
Paired comparison on one test set
Table 1: Thirteen studies on respiratory sound classification. Results are quoted as reported and are not comparable across class sets, metrics and splits.
Class
Audio files
Patients
Frames
Test frames
Frames by source
COPD
1,324
115
9,837
1,967
ICBHI 6,055, RD@TR 3,629, KAUH 153
Healthy
140
61
828
139
KAUH 588, ICBHI 240
Asthma
97
33
520
101
KAUH 513, ICBHI 7
Pneumonia
52
11
355
70
ICBHI 259, KAUH 96
URTI
23
14
159
35
ICBHI 159
Bronchiectasis
16
7
112
29
ICBHI 112
Table 2: Composition of the pooled data. Frames are 5 s windows before augmentation, and test frames are those of fold 0.
Figure 2: Preprocessing and image generation. A cleaned 5 s frame is converted into mel, CQT and MFCC images, which are scaled, resized and stacked.
Figure 3: One real frame per class, with its waveform and the three images before resizing.
Stage
Item
Value
Reason
Signal
Sampling rate, channels
4 kHz, mono
keeps the 0 to 2 kHz band analyzed here and gives 20,000 samples per 5 s frame
Frame length, hop
5 s (20,000 samples), 2.5 s (10,000 samples)
a 5 s frame holds on average 1.9 breathing cycles of ICBHI with a mean cycle of 2.7 s Nguyen and Pernkopf [2022] , and 50% overlap doubles the number of frames without new recordings
Drift removal
DFT bins 0 to 1 Hz set to zero
removes offset and slow drift below the band of respiratory sounds
Scaling
Division by the maximum absolute value
removes loudness differences between devices, so that loudness cannot decide the class
Augmentation
Stretch, pitch, noise
0.85 to 1.15, ± 3 semitones, factor 0.001 to 0.01
each copy changes tempo, pitch or background noise within these ranges and keeps the label of the frame it was made from
64 rows match the 64 CQT bins, 1024 points give a resolution of 3.9 Hz, the hop gives 39 columns close to the 40 of the other views, and the band is the full range available at 4 kHz
Table 3: Preprocessing, augmentation and image parameters, with the reason for each setting.
Figure 4: RespCCT architecture with the parameter count of each stage, and the lower panel expands one transformer block.
Figure 5: FreqCCT architecture, in which frequency rows are kept and time is averaged before the classifier.
Model
Group
Initialization
Parameters
MFLOPs
Width / blocks / heads
RespCCT
Proposed
Random
173,187
124.2
64 / 4 / 4
FreqCCT
Proposed
Random
124,999
72.5
64 / 4 / 4
CCT-M
Larger variant
Random
967,619
536.8
128 / 6 / 4
CCT-M + KD
Distilled student
Random
967,619
536.8
128 / 6 / 4
CCT-L
Larger variant
Random
2,151,811
1,135.5
192 / 6 / 6
CCT-L + KD
Distilled student
Random
2,151,811
1,135.5
192 / 6 / 6
Table 5: Size of every model, given as parameters, MFLOPs per 5 s frame and, for the CCT family, embedding width, number of blocks and number of heads. Distilled students have the size of their plain versions, and the teachers are MobileViT XXS and MobileNetV3 Small.
Model group
Learning rate
Weight decay
Batch
Label smoothing
Mixup
RespCCT, FreqCCT
0.002946
0
128
0.15
0
Pretrained baselines
10−3 (grid for two)
0
128
0.15
0
CCT-M plain
0.001414
0.03
256
0.15
0
CCT-M with distillation
0.001414
0.03
256
0
0.3
CCT-L plain
0.001061
0.05
256
0.15
0
CCT-L with distillation
0.001061
0.05
256
0
0.3
Table 6: Training settings. All runs use the balanced pool, two time masks of up to 8 columns, weight averaging and a cosine schedule, and CCT-M and CCT-L train for up to 150 epochs with a 10-epoch linear warm-up, all other models for up to 100.
Model
Params
MFLOPs
Acc.
F1
MCC
Bal. acc.
AUC
ECE
Vote
Δ F1 [95% CI]
RespCCT
173,187
124.2
.9813
.9695
.9410
.9907
.9994
.117
.985
FreqCCT
124,999
72.5
.9758
.9514
.9244
.9867
.9992
.125
.983
− .018 [ − .036, − .005]
ShuffleNetV2 × 0.5
348,967
17.2
.9784
.9659
.9310
.9842
.9994
.121
.983
− .004 [ − .014, +.006]
MobileNetV3 Small
931,047
24.5
.9881
.9777
.9611
.9892
.9983
.122
.989
+.008 [ − .003, +.020]
MobileViT XXS
953,271
109.6
.9911
.9843
.9707
.9920
.9999
.123
.994
+.015 [+.006, +.026]
EfficientNet-B0
4,016,515
168.2
.9915
.9841
.9718
.9901
.9997
.123
.992
+.015 [+.005, +.025]
Table 7: Fold-0 test results ( n=2,359 frames). Δ F1 is the model’s macro F1 minus that of RespCCT with a 95% paired bootstrap interval, and Vote is file-level majority-vote accuracy. Bold rows are the proposed models.
RespCCT
FreqCCT
Test fold
Acc.
F1
MCC
Best epoch
Acc.
F1
MCC
Best epoch
0
.9813
.9695
.9410
68 of 80
.9758
.9514
.9244
47 of 59
1
.9766
.9625
.9283
61 of 73
.9675
.9415
.8994
61 of 73
2
.9674
.9411
.8981
64 of 76
.9586
.9191
.8757
49 of 61
3
.9780
.9612
.9299
46 of 58
.9709
.9358
.9077
56 of 68
4
.9769
.9584
.9297
84 of 96
.9749
.9562
.9213
74 of 86
Table 8: Five-fold results of RespCCT and FreqCCT (seed 0).
Figure 6: Training and validation loss (top) and accuracy (bottom) on fold 0 for all twelve runs. For the distilled CCT-M and CCT-L, training accuracy is measured on mixup batches and training loss includes the distillation term, so neither is comparable with the other runs.
Figure 7: Confusion matrices (counts) of RespCCT (left) and FreqCCT (right) on the 2,359 fold-0 test frames, with true classes in rows and predicted classes in columns.
RespCCT
FreqCCT
Class
Frames
P
R
F1
P
R
F1
COPD
1,967
.999
.981
.990
.997
.975
.986
Healthy
139
.865
.964
.912
.882
.971
.925
Asthma
101
.847
.990
.913
.851
.960
.902
Pneumonia
70
.972
1.000
.986
.854
1.000
.921
URTI
35
.972
1.000
.986
.921
1.000
.959
Table 9: Per-class precision (P), recall (R) and F1 on fold 0.
Variant
Params
Acc.
F1
MCC
Δ F1 [95% CI]
Full model (sequence pooling, three views, gate)
173,187
.9813
.9695
.941
Views, mel only
172,001
.9780
.9601
.929
− .009 [ − .025, +.004]
Views, mel and CQT
172,594
.9775
.9597
.929
− .010 [ − .022, +.001]
Views, mel and MFCC
172,594
.9780
.9650
.931
− .005 [ − .016, +.007]
Views, CQT and MFCC (no mel)
172,594
.9665
.9355
.896
− .034 [ − .059, − .015]
Encoder, no transformer blocks
39,299
.8533
.6779
.581
− .292 [ − .343, − .250]
Table 10: Ablations of RespCCT on fold 0 (one run each). Δ F1 is the variant minus the full model with a 95% paired bootstrap interval.
Figure 8: Change in fold-0 macro F1 of each ablation relative to the full RespCCT, with 95% paired bootstrap intervals, and filled markers exclude zero. The variant without transformer blocks ( − 0.292) lies outside the axis.
Model
Params
MFLOPs
Acc.
F1
MCC
Errors
Vote
RespCCT
173,187
124.2
.9813
.9695
.941
44
.985
CCT-M
967,619
536.8
.9843
.9681
.950
37
.987
CCT-M + KD
967,619
536.8
.9864
.9748
.956
32
.988
CCT-L
2,151,811
1,135.5
.9792
.9637
.933
49
.982
CCT-L + KD
2,151,811
1,135.5
.9852
.9680
.952
35
.987
Ensemble of four CCTs
6.2 M
3,345
.9903
.9813
.968
23
.991
Table 11: Larger CCTs, distillation and ensembling on fold 0. KD means distilled from MobileViT XXS and MobileNetV3 Small with mixup and no label smoothing, and the ensemble averages CCT-M and CCT-L with and without KD.
Figure 9: Per-row contribution waterfalls of RespCCT for each class, built from the occlusion maps summed over time tokens. Red rows lower the class probability when occluded and blue rows raise it. Row 5 dominates Healthy and Pneumonia, Asthma and URTI spread over several rows, and Bronchiolitis ends near zero because its red and blue rows cancel.
Figure 10: Shapley attribution of RespCCT per class, grouped by view (left) and by frequency band from low to high (right).
Model
Params
MFLOPs
Weights (kB)
Peak act. (kB)
Model latency (ms)
Total (ms)
RespCCT (without gate)
173,128
124.2
692.5
655.4
6.43 / 7.21
36.0
FreqCCT
124,999
72.5
500.0
655.4
5.81 / 6.57
35.4
ShuffleNetV2 × 0.5
348,967
17.2
1,395.9
245.8
9.11 / 11.57
38.7
MobileNetV3 Small
931,047
24.5
3,724.2
184.3
6.84 / 7.71
36.4
MobileViT XXS
953,271
109.6
3,813.1
327.7
13.96 / 16.14
43.6
EfficientNet-B0
4,016,515
168.2
16,066.1
983.0
16.06 / 17.95
45.7
Table 12: Cost per 5 s frame in PyTorch on one Colab CPU thread. Latency is the median / 90th percentile, and Total adds the 29.6 ms of the three images (sum of separately measured stages). RespCCT is timed without its view gate.
Figure 11: ONNX Runtime latency on one CPU thread against fold-0 macro F1 for the ten models of Table 13 , with RespCCT shown with and without its gate. Bubble area shows file size, and red marks models that no faster and smaller model exceeds in macro F1.
Model
File (MB)
Median (ms)
Ratio
FreqCCT
0.559
3.11
1.00
RespCCT (without gate)
0.742
3.73
1.20
RespCCT (with gate)
0.743
6.46
2.08
EfficientNet-B0
15.314
6.85
2.20
MobileViT XXS
3.840
7.77
2.50
CCT-M
3.813
12.60
4.05
Table 13: ONNX Runtime 1.31.1 on one CPU thread (Intel Xeon 2.20 GHz, one Colab session, batch size one), showing file size, median latency and latency relative to FreqCCT.
Figure 12: Cost per 5 s frame on one CPU thread, with (a) the three image transforms and the RespCCT model and (b) model and image time (hatched) for each network.
Table 14: Results reported by earlier studies on diagnostic classification of lung sounds and by the present models. The data, classes and protocols differ, so the values are not ranked, and n/a means that the value is not given in the source text checked.
Respiratory sound classification from auscultation recordings is confounded by heterogeneous acquisition conditions, including recording device, auscultation site, and patient characteristics, that introduce systematic variability unrelated to the underlying pathology. Although such metadata is often available, existing approaches either ignore it or rely on naïve fusion strategies that do not model how distinct metadata sources influence different components of the acoustic representation. We propose a family of metadata-conditioning mechanisms for the Audio Spectrogram Transformer (AST), progressing from gated residual fusion to Feature-wise Linear Modulation (FiLM) adapted to the Transformer architecture. We further introduce two novel variants: Token-Aware FiLM (TAFiLM), which applies distinct modulation to classification and patch tokens, and Soft-Factorized FiLM (SoftFiLM), which learns soft masks to align metadata factors with disentangled feature subspaces. Experiments on the ICBHI benchmark demonstrate consistent gains across all conditioning strategies. SoftFiLM achieves 64.11% on the official 4-class task (+3.94% over the baseline) and 72.40% on the 2-class task, establishing a new state-of-the-art on the 2-class task and competitive performance on the 4-class benchmark.
George Kontogiannis, Pantelis Tzamalis, Sotiris Nikoletseas
Computer Engineering and Informatics Department, University of Patras, Patras, Greece
Respiratory auscultation provides a low-cost means of assessing pulmonary function, yet transient crackles and sustained wheezes can be difficult to recognize in noisy clinical recordings. Conventional deep learning classifiers often operate as black boxes, offering limited insight into the temporal acoustic evidence supporting their predictions. We present RespFocus, a weakly supervised framework that uses multiple instance learning (MIL) to learn inspectable temporal evidence from cycle-level labels and integrate it into classification. We also introduce PedResp2026, a multicenter pediatric respiratory sound cohort collected in routine clinical environments with adjudicated cycle-level annotations. In classification experiments, RespFocus achieves four-class Scores of 69.55% on public ICBHI 2017 and 72.91% on PedResp2026. Evidence diagnostics and matched temporal interventions support the functional contribution of the learned evidence to model predictions. Together, these findings highlight RespFocus's potential as a tool for automated respiratory sound diagnosis.
Yi Tian, Xiaohuan Chen, Yufei tang +5
Fuyao University of Science and Technology (fyust.edu.cn) · Xiamen University (xmu.edu.cn)
Recent respiratory sound classification (RSC) studies largely rely on CLS-token driven self-attention architectures such as the Audio Spectrogram Transformer (AST). While effective at modeling global context, recent analyses suggest a low-pass filtering behavior that may reduce sensitivity to localized abnormal patterns. In this work, we investigate State Space Models (SSMs) as an alternative backbone for RSC. Using the Distilled Audio State Space model, we analyze intermediate representations through spectral response curves and observe stronger preservation of mid-to-high spatial-frequency components. Based on these observations, we introduce spectral-aware layer regularization using Gaussian convolution applied to selected layers. We further propose Dual-Axis Patch-Mix contrastive learning tailored to SSM-based audio models for robust representation learning. Experiments on the ICBHI benchmark show that our approach achieves 64.48% score, outperforming the AST baseline by 5%. Code is available at https://github.com/RSC-Toolkit/Lung-SRAD.
Hemansh Shridhar, Miika Toikkanen, June-Woo Kim
RSC LAB, MODULABS, Republic of Korea · Department of Electronic Engineering, Wonkwang University, Republic of Korea · AI Convergence Research Institute, Wonkwang University, Republic of Korea