Cross-dataset transportability of pediatric chest X-ray deep learning across three countries: discrimination, calibration, operating-point failure, and limited-label recovery

Nazim-E-Alam

eess.IVSep 4, 2026

Cross-dataset transportability of pediatric chest X-ray deep learning across three countries: discrimination, calibration, operating-point failure, and limited-label recovery

Authors: Nazim-E-Alam

Organizations: Department of Computer Science, American International University-Bangladesh (AIUB), 408/1 (Old KA 66/1), Kuratoli, Khilkhet, Dhaka 1229, Bangladesh

Abstract

Background and Objective: External evaluation of medical-imaging AI is often collapsed into discrimination. We evaluated a computational protocol that separately tests discrimination, probability calibration, fixed operatingpoint transport, shortcut-associated signal, and limited-label recoverability for pediatric pneumonia classification across datasets from three countries. Methods: After exact-duplicate removal, 5,824 Guangzhou radiographs supported leakage-controlled source development and internal testing. A frozen three-seed DenseNet121 dual-view ensemble was evaluated zero-shot on BDCXR-3257 from Bangladesh (n = 3, 257) and an untouched harmonized VinDr-PCXR/PediCXR test cohort from Vietnam (n = 1, 077). Matched seed-42 variants tested architectural robustness. Secondary BDCXR analyses used a fixed 651-image adaptation pool and 2,606-image hold-out; 163, 326, and 651 labels represented 5%, 10%, and 20% of complete BDCXR. Results: Internal AUROC was 0.976 with 95.1% sensitivity. BDCXR and VinDr-PCXR AUROC were 0.798 and 0.742, while frozen-threshold sensitivity fell to 6.2% and 0%. Source-to-BDCXR AUROC degradation occurred for a full-image baseline (0.961 to 0.749), ungated dual-view model (0.977 to 0.766), and gated MixStyle model (0.966 to 0.789). With 163 BDCXR labels, Platt recalibration preserved AUROC while increasing held-out sensitivity to 88.3%, but specificity was 47.9% and the alert rate was 78.5%. Two hundred repeated 163-label fits confirmed sensitivity recovery but substantial specificity variability. Conclusions: Cross-dataset shifts across countries affected ranking, probability alignment, and source-defined decision behavior differently. Transport studies should evaluate these components separately and quantify the operational burden of apparent recovery.

Figures & tables

Figure 1: Study design and analysis hierarchy. Guangzhou data were used for model development and internal testing. BDCXR was first evaluated as a complete zero-shot external cohort; only afterward was a secondary target-label recoverability analysis performed on a fixed 651-image adaptation pool and disjoint 2,606-image hold-out. Label budgets of 163, 326, and 651 images correspond to 5%, 10%, and 20% of complete BDCXR. VinDr-PCXR/PediCXR remained untouched and served as a second independent cross-dataset external test.

Cohort	Country	Analytical role	Final $N$	Key design restriction
Guangzhou/ Kermany	China	Development, tuning, internal test	5,824 unique	Exact duplicates removed; filename-derived leakage-control groups
BDCXR-3257	Bangladesh	Primary external test; secondary adaptation	3,257	No verified patient/study identifier
VinDr-PCXR/ PediCXR	Vietnam	Untouched external test	1,077 harmonized	No target fitting; global-diagnosis harmonization frozen before inference

Table 1: Cohorts, endpoints, and analytical roles.

Architecture	Source threshold	Source AUROC	BDCXR AUROC	Source sensitivity	BDCXR sensitivity	BDCXR specificity
Full-image DenseNet121	0.938630	0.9606	0.7492	94.8%	16.2%	98.1%
Dual view, no gate/MixStyle	0.999934	0.9772	0.7664	96.1%	4.7%	100%
Gated dual view + MixStyle	0.999828	0.9655	0.7888	95.9%	4.4%	100%

Table 2: Matched seed-42 architecture robustness from Guangzhou source testing to complete-cohort BDCXR.

Figure 2: Discrimination versus frozen operating-point transport. AUROC remained above chance at both external sites while sensitivity at the frozen source threshold fell from 95.1% internally to 6.2% in BDCXR and 0% in VinDr-PCXR.

Cohort	$N$	Pneumonia $n$ (%)	AUROC	AUPRC	Sensitivity	Specificity	Balanced acc.	ECE
Guangzhou internal test	618	387 (62.6)	0.976	0.982	95.1%	90.9%	93.0%	0.152
BDCXR external	3,257	2,377 (73.0)	0.798	0.912	6.2%	99.9%	53.0%	0.258
VinDr-PCXR external	1,077	170 (15.8)	0.742	0.396	0%	100%	50.0%	0.139

Table 3: Internal and external performance at the frozen source operating point.

Figure 3: Calibration-only recovery on the locked BDCXR hold-out. Platt recalibration used 163, 326, or 651 labeled adaptation images (5%, 10%, or 20% of complete BDCXR). Because the transform is monotonic, AUROC was preserved while sensitivity, specificity, and calibration changed.

Method	Labels	AUROC	Sens.	Spec.	ECE	PPV	NPV	Alert rate	False alerts/100
Source locked	0	0.7987	0.064	1.000	0.256	1.000	0.283	0.047	0.0
Platt	163 (5%)	0.7987	0.883	0.479	0.044	0.821	0.603	0.785	14.1
Platt	326 (10%)	0.7987	0.919	0.386	0.029	0.802	0.638	0.837	16.6
Platt	651 (20%)	0.7987	0.907	0.425	0.030	0.810	0.629	0.818	15.5

Table 4: Secondary BDCXR recovery and operational burden on the locked 2,606-image hold-out.

Figure 4: Reliability before and after target recalibration. Displayed points summarize equal-count probability bins for locked BDCXR source predictions, BDCXR after the primary 163-label Platt fit, and untouched VinDr-PCXR predictions. Recalibration substantially improved BDCXR probability alignment but did not establish an acceptable clinical operating policy.

Figure 5: Post-hoc source-threshold robustness. A substantially more liberal operating point derived only from Guangzhou tuning data achieved 100% internal sensitivity but 69.0% sensitivity on BDCXR and 15.9% on VinDr-PCXR.

Operating point	Guangzhou internal test	BDCXR	VinDr-PCXR
Primary source threshold (0.9999728)	95.1%	6.2%	0%
Liberal source-only threshold (0.314311)	100%	69.0%	15.9%

Table 5: Sensitivity under the primary and liberal source-defined operating points.

Figure 6: Cross-dataset displacement of calibrated scores among pneumonia-positive examinations. Empirical cumulative distributions are shown for Guangzhou internal-test, BDCXR, and VinDr-PCXR pneumonia-positive examinations on the calibrated log-odds scale. Vertical lines mark source-derived operating thresholds. The figure characterizes score transport and does not attribute the displacement causally to country.

Explore similar work

Beyond Accuracy: Quantifying Pulmonary Attribution in Anatomy-Guided Chest X-Ray Classification Under Domain Shift
Aug 31, 2026Abdullah Al Mamun, Md. Nasif Osman Khansur, Md Ashraful Hossen Akash +2Chest X-Ray Classification
Department of Computer Science and Engineering, Rajshahi University of Engineering & Technology (RUET), Kazla, Rajshahi 6204, Bangladesh · Elite Research Lab LLC, New York, USA · Faculty of Information Science and Technology, Multimedia University, Melaka, Malaysia +1
CBAM-Enhanced DenseNet121 for Multi-Class Chest X-Ray Classification with Grad-CAM Explainability
Apr 14, 2026Utsho Kumar DeyChest X-Ray Classification Convolutional Neural Networks
Department of Computer Science and Engineering, Northern University of Business and Technology Khulna, Bangladesh
Evaluation Choices Shape Biomedical ML Claims: A Pediatric Pneumonia Benchmark Case Study
Sep 29, 2026Bhanu Prakash Vangala, Sowmya Guda, Latha Peddi +1Data Leakage Chest X-Ray Classification
University of Missouri · Government Degree College · Amar Bio Tech Pvt Ltd

Aug 31, 2026J/K move · Enter open · S save

Beyond Accuracy: Quantifying Pulmonary Attribution in Anatomy-Guided Chest X-Ray Classification Under Domain Shift

Abdullah Al Mamun, Md. Nasif Osman Khansur, Md Ashraful Hossen Akash, Md. Kishor Morol+1

Department of Computer Science and Engineering, Rajshahi University of Engineering & Technology (RUET), Kazla, Rajshahi 6204, Bangladesh · Elite Research Lab LLC, New York, USA · Faculty of Information Science and Technology, Multimedia University, Melaka, Malaysia+1

Chest X-Ray Classification

Open paper AskPDF arXiv

Deep-learning models can achieve strong chest X-ray (CXR) classification performance without establishing whether their predictions predominantly rely on pulmonary image content. This study evaluates pulmonary attribution containment as an anatomy-related reliability property distinct from diagnostic performance. We propose DBCA-SegNet-MGAP, a multi-task anatomy-guided CNN-Transformer framework that combines complementary feature representations through bidirectional cross-backbone attention, predicts a soft lung mask, and incorporates this anatomical prior directly into classification through Mask-Guided Adaptive Global Average Pooling (MGAP). Pulmonary attribution containment is quantified using the Anatomical Local Energy Ratio (ALR) and high-intensity cumulative ALR ([email protected]). Experiments were repeated across three training seeds using the COVID-19 Radiography Database for four-class internal testing and a locked Shenzhen-to-Montgomery protocol for zero-shot external tuberculosis testing. On COVID-19, the proposed model achieved a weighted F1 of

0.9615 \pm 0.0015

and macro ROC-AUC of

0.9906 \pm 0.0007

. In an architecture-matched dual-bridge comparison, replacing conventional GAP with MGAP increased ALR from

0.3878 \pm 0.0098

0.7086 \pm 0.0104

and [email protected] from

0.5265 \pm 0.0101

0.9905 \pm 0.0018

, while weighted F1 remained essentially unchanged (

0.9618 \pm 0.0015

vs.

0.9615 \pm 0.0015

). Under locked external transfer to Montgomery, ROC-AUC remained

0.9080 \pm 0.0043

and pulmonary ALR remained

0.6466 \pm 0.0081

, whereas weighted F1 decreased to

0.7528 \pm 0.0080

and ECE increased to

0.1683 \pm 0.0055

. These findings show that diagnostic discrimination, calibration, and pulmonary attribution containment are distinct model properties and support their joint evaluation under internal testing and external domain shift.

Abstract

Explore similar work

Beyond Accuracy: Quantifying Pulmonary Attribution in Anatomy-Guided Chest X-Ray Classification Under Domain Shift

CBAM-Enhanced DenseNet121 for Multi-Class Chest X-Ray Classification with Grad-CAM Explainability

Evaluation Choices Shape Biomedical ML Claims: A Pediatric Pneumonia Benchmark Case Study