CXR-CMFA: A Cross-Modal Feature Aggregation Network for Chest X-Ray Pathology Classification
Organizations: American International University–Bangladesh (AIUB), Dhaka, Bangladesh · University of Sorocaba (UNISO), S˜ao Paulo, Brazil
Abstract
Chest X-ray (CXR) interpretation remains challenging because subtle pathology can be obscured by overlapping anatomy, image-quality variation, and limited visual evidence in isolated radiographs. Although image-only deep learning models provide useful baselines, they discard the complementary diagnostic context available in radiology reports, while large vision-language models can be computationally expensive for practical deployment. This paper presents CXR-CMFA, a lightweight cross-modal feature aggregation network for binary CXR pathology classification. The proposed framework combines a dual-stream visual encoder, using EfficientNet-B0 for global anatomical representation and DenseNet-121 for local pathological detail, with a TF-IDF-MLP report encoder. Two fusion modules integrate these modalities: the Dual-Modal Spatial Attention Module (DMSAM) uses the report embedding to gate global and local visual features, and the Cross-Modal Gated Feature Fusion (CGFF) module learns per-instance weights for visual and textual contributions. Experiments on the MIMIC-CXR benchmark, using the provided train, validation, and test splits, show that CXR-CMFA achieves 93.45% accuracy, 99.88% precision, 91.8% recall, 95.7% F1-score, and 99.19% AUC on the held-out test set. The model produces only four false positives among 984 normal test cases, supporting its suitability for high-specificity screening workflows. An ablation study further demonstrates a 22.47 percentage-point AUC improvement over the global-image-only variant, confirming the value of adaptive cross-modal fusion. These results indicate that compact report-conditioned fusion can provide strong discriminative performance without relying on large pretrained language models.
Figures & tables
| Parameter | Selected Value | Search Range |
|---|---|---|
| Learning rate | ||
| Batch size | 16 | |
| Dropout rate (projection) | 0.4 | |
| Weight decay | ||
| Number of epochs | 10 | |
| Early stopping patience | 4 |
| Model | Acc | Prec | Rec | F1 | AUC |
|---|---|---|---|---|---|
| CheXNet [ 7 ] | 0.707 | 0.898 | 0.707 | 0.791 | 0.759 |
| MedViT-S [ 11 ] | 0.707 | 0.883 | 0.722 | 0.794 | 0.739 |
| TieNet [ 15 ] | 0.960 | 0.995 | 0.954 | 0.974 | 0.990 |
| BioViL [ 2 ] | 0.962 | 0.997 | 0.955 | 0.976 | 0.994 |
| CXR-CMFA (Ours) | 0.935 | 0.999 | 0.918 | 0.957 | 0.992 |
| Model | Params. (M) | Inference (ms/img) | GPU Memory (MB) |
|---|---|---|---|
| CheXNet | 7.0 | 5.2 | 850 |
| TieNet | 5.8 | 6.1 | 920 |
| BioViL | 86.4 | 28.7 | 2,850 |
| MedViT-S | 94.2 | 31.4 | 3,100 |
| CXR-CMFA (Ours) | 16.7 | 9.8 | 1,350 |
| Variant | Acc. | Prec. | Rec. | F1 | AUC |
|---|---|---|---|---|---|
| Global Only | 0.6899 | 0.9013 | 0.6800 | 0.7751 | 0.7658 |
| Local Only | 0.6632 | 0.8859 | 0.6559 | 0.7537 | 0.7285 |
| Global + Report Conc. | 0.9450 | 0.9956 | 0.9341 | 0.9639 | 0.9905 |
| Dual Concatenation | 0.9360 | 0.9976 | 0.9208 | 0.9577 | 0.9915 |
| Dual + DMSAM (w/o CGFF) | 0.9545 | 0.9945 | 0.9474 | 0.9704 | 0.9917 |
| CXR-CMFA (Proposed) | 0.9406 | 0.9979 | 0.9264 | 0.9608 | 0.9905 |
| Model | Mean Acc. | Std. | Min | Max | Range |
|---|---|---|---|---|---|
| CheXNet | 0.6876 | 0.0345 | 0.6534 | 0.7475 | 0.0941 |
| TieNet | 0.9486 | 0.0058 | 0.9422 | 0.9591 | 0.0169 |
| BioViL | 0.9591 | 0.0038 | 0.9522 | 0.9629 | 0.0107 |
| MedViT-S | 0.6500 | 0.0303 | 0.6128 | 0.7024 | 0.0896 |
| CXR-CMFA (Ours) | 0.9250 | 0.0083 | 0.9143 | 0.9368 | 0.0225 |
Chest X-Ray Classification
Chest X-ray classification uses machine learning to detect and categorize diseases and findings in radiographs for screening, triage, and clinical decision support.
See all 108 papers