How well do routinely collected demographic and clinical variables aid point-of-care lung ultrasound TB classification
Authors: Joshua M. Jansen van Vüren, Christiaan M. Geldenhuys, Devendra S. Parihar, Véronique Suttels, Trevor Brokowski, Ablo P. Wachinou, Mary-Anne Hartley, Rensu P. Theart, +2 more
Organizations: Department of Electrical and Electronic Engineering, University of Stellenbosch, Stellenbosch, South Africa · Laboratory for intelligent Global Health and Humanitarian Response Technologies (LiGHT) Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland · National Teaching Hospital for Tuberculosis and Pulmonary Diseases (CNHU-PPC), Cotonou, Benin · South African Medical Research Council Centre for Tuberculosis Research (CTR) Division of Molecular Biology and Human Genetics, Faculty of Medicine and Health Sciences Stellenbosch University, Cape Town, South Africa
We consider the fusion of lung ultrasound images with routinely-collected clinical and demographic data for the purpose of automated tuberculosis (TB) screening using deep-learning. Such deep-learning based screening tools for TB could meaningfully support the health care system in Africa, where the burden of disease is severe and resources are constrained. Beginning with an established ResNet baseline for classification of lung ultrasound images, which achieves an area under the receiver operating characteristic (AUROC) curve of 0.91 [0.86,0.96] (95% CI), we consider the incorporation of the clinical and demographic data using three fusion approaches. We find that a simple average-based fusion of the output scores of separately-trained image and clinical data classifiers consistently matches or outperforms a more complex approach where the data is fused earlier and a combined classifier is trained. Fusing the image and the clinical classifiers in this way leads to a classifier with an overall AUROC of 0.95 [0.91,0.99] (specificity of 0.76 at sensitivity 0.93) which is an improvement of 4% absolute over the image-only baseline. We also find that greedy feature selection can be used to reduce the number of clinical and demographic inputs without sacrificing classification performance. Finally, when we differentiate between clinical and demographic data that are self-reported, that require some basic measurement or calculation, and that require a point-of-care (POC) test, we find the inclusion of the POC tests included in this study to be of minimal benefit to classification performance. We conclude that the incorporation of routinely-collected clinical and demographic data is a promising way to improve the performance of lung ultrasound based automatic classification.
Figures & tables
Fold
TB Pos
# Patients
Num scans [min,max]
1
32
81
27 [13, 31]
2
36
80
27 [18, 31]
3
37
80
26 [10, 33]
4
40
80
27 [15, 33]
5
36
80
27 [14, 31]
Test
43
101
27 [19, 33]
TABLE I : Cross validation fold statistics. Num scans: mean [min,max].
Class
Kind
Variable
A
Numeric
Age
Binary
Asthenia
Binary
Anorexia
One-hot
Chief complaint ( cough, fever, etc. )
Binary
Cardiopathy
Binary
Chronic lung disease
TABLE II : Clinical, demographic and POC values collected for each of the participants in the Benin dataset [ 23 ] . Variables are partitioned into one of three classes (A: 28,B:13,C:3) in order of their ease-of-acquisition, A being the most accessible and C the least.
Model
Hyperparameter
Range
Optimal
Learning rate
[0.1, 0.01, 0.001]
0.001
LR
Batch size
[8, 16, 32]
16
Epochs
[1, … , 128]
100
Learning rate
[0.01, 0.001, 0.0001]
0.001
MLP
Batch size
[8, 16, 32]
16
Hidden size
[[16],[32], [64, 32],
[16]
TABLE III : Model and optimisation hyperparameters for clinical (LR,MLP) and LUS TB-classification models. Parameters are presented for the top performing configuration. For the clinical classifiers these values are for the second stage sweep after feature selection has occurred.
Fig. 1 : Fusion of clinical data into ultrasound classifier through the concatenation of the encoded feature vector before the final classification layers (classification head).
Fig. 2 : Alternative strategies for classifier fusion using either (top) a simple average of the posterior TB probabilities produced by the clinical and LUS-based classifiers or (bottom) LR trained directly on the model logits. The sigmoid function is denoted as σ .
Feature
Model
SFS
#
Dev AUROC
Test AUROC
subset
Var
Baseline LUS
0.87 [0.84,0.90]
0.91 [0.86,0.96]
A
LR
28
0.77 [0.72, 0.82]
0.86 [0.78, 0.93]
MLP
28
0.78 [0.74, 0.82]
0.84 [0.77, 0.92]
LR
✓
10
0.82 [0.77, 0.85]
0.86 [0.79, 0.94]
MLP
✓
13
0.82 [0.78, 0.86]
0.80 [0.72, 0.89]
TABLE IV : Development (95% bootstrap CI [ 27 ] ) and test set (95% CI DeLong [ 28 ] ) results for baseline LUS classifier and the various clinical classifiers trained on subsets of availability classes. A: self-reported, B: simple measurement, C: POC test.
Feature subset
SFS
Fusion type
Dev AUROC
Test AUROC
Baseline LUS
0.87 [0.84,0.90]
0.91 [0.86,0.96]
A
Concat
0.87 [0.83,0.91]
0.91 [0.85,0.97]
LR
0.88 [0.84,0.91]
0.94 [0.89,0.98]
Mean
0.87 [0.83,0.90]
0.93 [0.88,0.98]
✓
Concat
0.88 [0.85,0.91]
0.92 [0.86,0.97]
✓
LR
0.89 [0.86,0.92]
0.93 [0.87,0.98]
TABLE V : Development (95% bootstrap CI) and test set (95% CI DeLong) results for baseline LUS classifier and the various fused classifiers trained on subsets of availability classes. A: self-reported, B: some simple measurement, C: POC test.
Fig. 3 : Test set receiver-operating characteristic (ROC) curve for the best of each considered architecture based on the development sets. All curves are smoothed over 10,000 bootstrap resamples and a bootstrap percentile-based 95% CI is displayed for the fused classifier. AUROCs are presented with DeLong 95% CI. Fusion: Mean of output probabilities from best clinical classifier MLP ABC (SFS) and ResNet-18. All three classifiers are those which afforded the lowest average development set loss across the respective configurations.
Department of Computer Science and Engineering, Daffodil International University, Dhaka, Bangladesh. · Department of Computer Science and Engineering, Daffodil International University, Savar, Dhaka, Bangladesh. · Department of Public Health, Daffodil International University, Dhaka, Bangladesh. +1