Cross-dataset transportability of pediatric chest X-ray deep learning across three countries: discrimination, calibration, operating-point failure, and limited-label recovery

Nazim-E-Alam

eess.IVSep 4, 2026

Cross-dataset transportability of pediatric chest X-ray deep learning across three countries: discrimination, calibration, operating-point failure, and limited-label recovery

Authors: Nazim-E-Alam

Organizations: Department of Computer Science, American International University-Bangladesh (AIUB), 408/1 (Old KA 66/1), Kuratoli, Khilkhet, Dhaka 1229, Bangladesh

Abstract

Background and Objective: External evaluation of medical-imaging AI is often collapsed into discrimination. We evaluated a computational protocol that separately tests discrimination, probability calibration, fixed operatingpoint transport, shortcut-associated signal, and limited-label recoverability for pediatric pneumonia classification across datasets from three countries. Methods: After exact-duplicate removal, 5,824 Guangzhou radiographs supported leakage-controlled source development and internal testing. A frozen three-seed DenseNet121 dual-view ensemble was evaluated zero-shot on BDCXR-3257 from Bangladesh (n = 3, 257) and an untouched harmonized VinDr-PCXR/PediCXR test cohort from Vietnam (n = 1, 077). Matched seed-42 variants tested architectural robustness. Secondary BDCXR analyses used a fixed 651-image adaptation pool and 2,606-image hold-out; 163, 326, and 651 labels represented 5%, 10%, and 20% of complete BDCXR. Results: Internal AUROC was 0.976 with 95.1% sensitivity. BDCXR and VinDr-PCXR AUROC were 0.798 and 0.742, while frozen-threshold sensitivity fell to 6.2% and 0%. Source-to-BDCXR AUROC degradation occurred for a full-image baseline (0.961 to 0.749), ungated dual-view model (0.977 to 0.766), and gated MixStyle model (0.966 to 0.789). With 163 BDCXR labels, Platt recalibration preserved AUROC while increasing held-out sensitivity to 88.3%, but specificity was 47.9% and the alert rate was 78.5%. Two hundred repeated 163-label fits confirmed sensitivity recovery but substantial specificity variability. Conclusions: Cross-dataset shifts across countries affected ranking, probability alignment, and source-defined decision behavior differently. Transport studies should evaluate these components separately and quantify the operational burden of apparent recovery.

Figures & tables

Figure 1: Study design and analysis hierarchy. Guangzhou data were used for model development and internal testing. BDCXR was first evaluated as a complete zero-shot external cohort; only afterward was a secondary target-label recoverability analysis performed on a fixed 651-image adaptation pool and disjoint 2,606-image hold-out. Label budgets of 163, 326, and 651 images correspond to 5%, 10%, and 20% of complete BDCXR. VinDr-PCXR/PediCXR remained untouched and served as a second independent cross-dataset external test.

Cohort	Country	Analytical role	Final $N$	Key design restriction
Guangzhou/ Kermany	China	Development, tuning, internal test	5,824 unique	Exact duplicates removed; filename-derived leakage-control groups
BDCXR-3257	Bangladesh	Primary external test; secondary adaptation	3,257	No verified patient/study identifier
VinDr-PCXR/ PediCXR	Vietnam	Untouched external test	1,077 harmonized	No target fitting; global-diagnosis harmonization frozen before inference

Table 1: Cohorts, endpoints, and analytical roles.

Architecture	Source threshold	Source AUROC	BDCXR AUROC	Source sensitivity	BDCXR sensitivity	BDCXR specificity
Full-image DenseNet121	0.938630	0.9606	0.7492	94.8%	16.2%	98.1%
Dual view, no gate/MixStyle	0.999934	0.9772	0.7664	96.1%	4.7%	100%
Gated dual view + MixStyle	0.999828	0.9655	0.7888	95.9%	4.4%	100%

Table 2: Matched seed-42 architecture robustness from Guangzhou source testing to complete-cohort BDCXR.

Figure 2: Discrimination versus frozen operating-point transport. AUROC remained above chance at both external sites while sensitivity at the frozen source threshold fell from 95.1% internally to 6.2% in BDCXR and 0% in VinDr-PCXR.

Cohort	$N$	Pneumonia $n$ (%)	AUROC	AUPRC	Sensitivity	Specificity	Balanced acc.	ECE
Guangzhou internal test	618	387 (62.6)	0.976	0.982	95.1%	90.9%	93.0%	0.152
BDCXR external	3,257	2,377 (73.0)	0.798	0.912	6.2%	99.9%	53.0%	0.258
VinDr-PCXR external	1,077	170 (15.8)	0.742	0.396	0%	100%	50.0%	0.139

Table 3: Internal and external performance at the frozen source operating point.

Figure 3: Calibration-only recovery on the locked BDCXR hold-out. Platt recalibration used 163, 326, or 651 labeled adaptation images (5%, 10%, or 20% of complete BDCXR). Because the transform is monotonic, AUROC was preserved while sensitivity, specificity, and calibration changed.

Method	Labels	AUROC	Sens.	Spec.	ECE	PPV	NPV	Alert rate	False alerts/100
Source locked	0	0.7987	0.064	1.000	0.256	1.000	0.283	0.047	0.0
Platt	163 (5%)	0.7987	0.883	0.479	0.044	0.821	0.603	0.785	14.1
Platt	326 (10%)	0.7987	0.919	0.386	0.029	0.802	0.638	0.837	16.6
Platt	651 (20%)	0.7987	0.907	0.425	0.030	0.810	0.629	0.818	15.5

Table 4: Secondary BDCXR recovery and operational burden on the locked 2,606-image hold-out.

Figure 4: Reliability before and after target recalibration. Displayed points summarize equal-count probability bins for locked BDCXR source predictions, BDCXR after the primary 163-label Platt fit, and untouched VinDr-PCXR predictions. Recalibration substantially improved BDCXR probability alignment but did not establish an acceptable clinical operating policy.

Figure 5: Post-hoc source-threshold robustness. A substantially more liberal operating point derived only from Guangzhou tuning data achieved 100% internal sensitivity but 69.0% sensitivity on BDCXR and 15.9% on VinDr-PCXR.

Operating point	Guangzhou internal test	BDCXR	VinDr-PCXR
Primary source threshold (0.9999728)	95.1%	6.2%	0%
Liberal source-only threshold (0.314311)	100%	69.0%	15.9%

Table 5: Sensitivity under the primary and liberal source-defined operating points.

Figure 6: Cross-dataset displacement of calibrated scores among pneumonia-positive examinations. Empirical cumulative distributions are shown for Guangzhou internal-test, BDCXR, and VinDr-PCXR pneumonia-positive examinations on the calibrated log-odds scale. Vertical lines mark source-derived operating thresholds. The figure characterizes score transport and does not attribute the displacement causally to country.

Explore similar work

Aug 31, 2026

Beyond Accuracy: Quantifying Pulmonary Attribution in Anatomy-Guided Chest X-Ray Classification Under Domain Shift

Deep-learning models can achieve strong chest X-ray (CXR) classification performance without establishing whether their predictions predominantly rely on pulmonary image content. This study evaluates pulmonary attribution containment as an anatomy-related reliability property distinct from diagnostic performance. We propose DBCA-SegNet-MGAP, a multi-task anatomy-guided CNN-Transformer framework that combines complementary feature representations through bidirectional cross-backbone attention, predicts a soft lung mask, and incorporates this anatomical prior directly into classification through Mask-Guided Adaptive Global Average Pooling (MGAP). Pulmonary attribution containment is quantified using the Anatomical Local Energy Ratio (ALR) and high-intensity cumulative ALR ([email protected]). Experiments were repeated across three training seeds using the COVID-19 Radiography Database for four-class internal testing and a locked Shenzhen-to-Montgomery protocol for zero-shot external tuberculosis testing. On COVID-19, the proposed model achieved a weighted F1 of

0.9615 \pm 0.0015

and macro ROC-AUC of

0.9906 \pm 0.0007

. In an architecture-matched dual-bridge comparison, replacing conventional GAP with MGAP increased ALR from

0.3878 \pm 0.0098

0.7086 \pm 0.0104

and [email protected] from

0.5265 \pm 0.0101

0.9905 \pm 0.0018

, while weighted F1 remained essentially unchanged (

0.9618 \pm 0.0015

vs.

0.9615 \pm 0.0015

). Under locked external transfer to Montgomery, ROC-AUC remained

0.9080 \pm 0.0043

and pulmonary ALR remained

0.6466 \pm 0.0081

, whereas weighted F1 decreased to

0.7528 \pm 0.0080

and ECE increased to

0.1683 \pm 0.0055

. These findings show that diagnostic discrimination, calibration, and pulmonary attribution containment are distinct model properties and support their joint evaluation under internal testing and external domain shift.

Abdullah Al Mamun, Md. Nasif Osman Khansur, Md Ashraful Hossen Akash +2

Department of Computer Science and Engineering, Rajshahi University of Engineering & Technology (RUET), Kazla, Rajshahi 6204, Bangladesh · Elite Research Lab LLC, New York, USA · Faculty of Information Science and Technology, Multimedia University, Melaka, Malaysia +1

Chest X-Ray Classification

Ask PDF

Apr 14, 2026

CBAM-Enhanced DenseNet121 for Multi-Class Chest X-Ray Classification with Grad-CAM Explainability

Pneumonia remains a leading cause of childhood mortality worldwide, with a heavy burden in low-resource settings such as Bangladesh where radiologist availability is limited. Most existing deep learning approaches treat pneumonia detection as a binary problem, overlooking the clinically critical distinction between bacterial and viral aetiology. This paper proposes CBAM-DenseNet121, a transfer-learning framework that integrates the Convolutional Block Attention Module (CBAM) into DenseNet121 for three-class chest X-ray classification: Normal, Bacterial Pneumonia, and Viral Pneumonia. We also conduct a systematic binary-task baseline study revealing that EfficientNetB3 (73.88%) underperforms even the custom CNN baseline (78.53%) -- a practically important negative finding for medical imaging model selection. To ensure statistical reliability, all experiments were repeated three times with independent random seeds (42, 7, 123), and results are reported as mean +/- standard deviation. CBAM-DenseNet121 achieves 84.29% +/- 1.14% test accuracy with per-class AUC scores of 0.9565 +/- 0.0010, 0.9610 +/- 0.0014, and 0.9187 +/- 0.0037 for bacterial pneumonia, normal, and viral pneumonia respectively. Grad-CAM visualizations confirm that the model attends to anatomically plausible pulmonary regions for each class, supporting interpretable deployment in resource-constrained clinical environments.

Utsho Kumar Dey

Department of Computer Science and Engineering, Northern University of Business and Technology Khulna, Bangladesh

Chest X-Ray Classification Convolutional Neural Networks

Ask PDF

Sep 29, 2026

Evaluation Choices Shape Biomedical ML Claims: A Pediatric Pneumonia Benchmark Case Study

Biomedical machine learning papers often compress model performance into one headline number. That number can look like a property of the model even when it depends strongly on how the benchmark was evaluated. We study this problem on the widely used Kermany pediatric chest radiograph dataset using nine image classifiers and a controlled evaluation protocol. Under the same protocol, the eight pretrained backbones differ by only 0.026 AUROC. In contrast, changing whether the backbone is frozen or fine-tuned changes AUROC by 0.044 on average, and changing the decision threshold changes balanced accuracy by 0.090 on average. The official test split is also measurably different from the training pool: a partition classifier distinguishes them at AUC 0.697, rising to 0.898 for normal radiographs. Most strikingly, a classifier using only file properties, with no image anatomy, reaches 0.992 balanced accuracy within the training pool but falls to 0.496 on the official test split. Validation-fitted thresholds and calibration also transfer imperfectly. These results show that a high benchmark score can support different conclusions when the split, training policy, threshold, metric, calibration, and uncertainty are not communicated with it. We end with a seven-item reporting recommendation in which each item is tied to an effect measured in the study

Bhanu Prakash Vangala, Sowmya Guda, Latha Peddi +1

University of Missouri · Government Degree College · Amar Bio Tech Pvt Ltd

Data Leakage Chest X-Ray Classification Shortcut Learning Medical Image Benchmarks

Ask PDF