CLARiTy: A Vision Transformer for Multi-Label Classification and Weakly-Supervised Localization of Chest X-ray Pathologies

John M. Statheros; Hairong Wang; Richard Klein

cs.LGDec 18, 2025

CLARiTy: A Vision Transformer for Multi-Label Classification and Weakly-Supervised Localization of Chest X-ray Pathologies

Authors: John M. Statheros, Hairong Wang, Richard Klein

Organizations: School of Computer Science and Applied Mathematics, University of the Witwatersrand, Johannesburg, South Africa

Abstract

The interpretation of chest X-rays poses significant challenges in achieving accurate multi-label pathology classification and spatial localization. These tasks demand different annotation granularities but are constrained by the scarcity of region-level (dense) annotations. We introduce CLARiTy (Class Localizing and Attention Refining Image Transformer), a vision transformer-based model for joint multi-label classification and weakly-supervised localization of thoracic pathologies. CLARiTy employs multiple class-specific tokens to generate discriminative attention maps and a SegmentCAM module for foreground segmentation and background suppression using explicit anatomical priors. Trained on image-level labels from the NIH ChestXray14 dataset, it leverages distillation from a ConvNeXtV2 teacher for efficiency. Localization heatmaps are converted to bounding boxes with OneBox, a hysteresis-gated extractor whose parameters are calibrated per finding on held-out validation data. Evaluated on the official NIH split, CLARiTy achieves competitive classification across 14 pathologies and strong weakly-supervised localization on 8 pathologies, outperforming prior methods by up to 50.7% relative Macro Intersection over Union (IoU) Accuracy at stringent thresholds. Gains are most pronounced for small pathologies: at T(IoU) = 0.5, the 224 variant reaches an IoU Accuracy of 0.154 on nodules, where the compared methods remain at or below 0.013. The lower-resolution CLARiTy-S-16-224 offers high efficiency while surpassing baselines, and could potentially be applied in low-resource settings. An ablation study confirms the contributions of SegmentCAM, DINO pretraining, and attention pooling. CLARiTy advances beyond CNN-ViT hybrids by harnessing ViT self-attention for global context and class-specific localization, refined through convolutional background suppression for precise, noise-reduced heatmaps.

Figures & tables

Figure 1: Illustration of the proposed CLARiTy model. An input chest X-ray image is split into $N^{2}$ patches and embedded into $P$ patch tokens, which are concatenated with $C$ class tokens. Learned positional embeddings are added to produce $C+P$ input tokens to the transformer. A series of $d$ transformer blocks extracts relevant information for classification and weakly-supervised localization. At the output, the $C$ class tokens are passed through an attention pooling module to produce class token logits. The output $P$ patch tokens are passed through the SegmentCAM module, where foreground logits and a segmentation map are produced. The self-attention maps from the final $p$ transformer blocks are fused together to produce class-specific attention maps.

Figure 2: Transformer self-attention matrix with multiple class tokens. Each class token is denoted

c_{1},\allowbreak c_{2},\allowbreak\dots,\allowbreak c_{C}

, and each patch token is denoted

n_{1},\allowbreak n_{2},\allowbreak\dots,\allowbreak n_{\negthinspace P}

. The class-to-patch attention in the upper-right sub-matrix is extracted for the class-specific attention maps.

Figure 3: Weakly-supervised localization method of CLARiTy. During inference, a foreground mask and attention map are produced for each class. Element-wise multiplication yields a class-specific localization heatmap that is both highly precise and confined to the class’s ground-truth region. In this chest X-ray, a mass (round opacity) is found in the upper-middle left lung of the patient. The attention map has high intensity directly over the mass, and the heatmap intensity is confined to the lung lobes.

Figure 4: Illustration of the CLARiTy losses. Following the attention pooling layer, the class token logits are used to calculate the class token classification loss $\mathcal{L}_{\mathrm{CLS}}^{C}$ . The outputs of the SegmentCAM module are used to calculate the combined CAM loss $\mathcal{L}_{\mathrm{CAM}}$ .

Figure 5: Illustration of the attention pooling module. Row-wise softmax activation is applied to a set of learned weights, which are invariant to the input image. This matrix is element-wise multiplied with the output class tokens after reshaping to a

C\times D

matrix. Thereafter, a row-wise sum produces class-specific logits. Each dimension in

D

is able to attend to particular class features.

Figure 6: Illustration of the progressive fusion block used in the heatmap branch of SegmentCAM. The

N\times N\times D

patch tokens are progressively refined and upsampled by three stages of bilinear

\times 2

interpolation, each followed by a

3\times 3

convolution,

\mathrm{SiLU}

activation, and batch normalization. The channel width tapers from

\max\left(4C,64\right)

through

\max\left(3C,48\right)

C

, so all learned computation runs at or below half the image resolution. A

1\times 1

convolution projects the raw patch tokens into

C

class maps, which are bilinearly enlarged by a factor of

8

and added to the main path, restoring the global context that the narrowed stages would otherwise discard. A final bilinear

\times 2

and sigmoid activation produce the image-resolution heatmap

\boldsymbol{H}\in\left(0,1\right)^{M\times M\times C}

Figure 7: Fused inverse bottleneck block, as used in Xiong et al. (2021) . In SegmentCAM, the block is used in the feature head and in the classification head of the background activation suppression stage. An input tensor of shape

H\times W\times K_{1}

is passed through a

3\times 3

convolution with an expansion ratio of

r\in\mathbb{R}_{>1}

. This increases the number of channels to

rK_{1}

. Batch normalization and

\mathrm{SiLU}

activation are then applied. Finally, a

1\times 1

convolution with batch normalization reduces the number of channels to

K_{2}

. If

K_{1}=K_{2}

, then a residual connection is added to the output.

Figure 8: Illustration of the SegmentCAM module’s feature and heatmap extraction. The output patch tokens are first reshaped to an

N\times N\times D

tensor. Then, in the heatmap branch, the progressive fusion block ( Fig. 6 ) with sigmoid activation produces a heatmap tensor

\boldsymbol{H}

of shape

M\times M\times C

, where

M

is the input image resolution. A threshold

t

is applied to

\boldsymbol{H}

to produce foreground and background segmentation maps

\boldsymbol{S}_{\negthinspace f}

and

\boldsymbol{S}_{b}

respectively. The heatmap

\boldsymbol{H}

is used to calculate the soft Dice segmentation loss

\mathcal{L}_{\mathrm{Dice}}

. In the feature branch, a convolutional head produces feature tensor

\boldsymbol{F}

of shape

N\times N\times C

, which is used in the background activation suppression methodology.

Figure 9: Illustration of the SegmentCAM module’s background activation suppression methodology. The segmentation maps

\boldsymbol{S}_{\negthinspace f}

and

\boldsymbol{S}_{b}

are derived from the heatmap at the feature resolution. The feature tensor

\boldsymbol{F}

is passed through the convolutional classification head twice: with unmasked features, and with foreground features suppressed (after fusing with

\boldsymbol{S}_{b}

). In the top branch, the output of the unmasked features has its background features suppressed with

\boldsymbol{S}_{\negthinspace f}

, and global average pooling (GAP) is applied to produce foreground logits

\boldsymbol{\ell}_{\negthinspace f}

. In the middle branch, the output unmasked features are used to produce patch token logits

\boldsymbol{\ell}_{\hskip-0.65556ptP}

using GAP. In the bottom branch, the output of the foreground-suppressed features produces background logits

\boldsymbol{\ell}_{b}

using GAP. Foreground and patch token losses

\mathcal{L}_{\mathrm{CLS}}^{f}

and

\mathcal{L}_{\mathrm{CLS}}^{P}

are calculated using

\boldsymbol{\ell}_{\negthinspace f}

and

\boldsymbol{\ell}_{\hskip-0.65556ptP}

respectively. Finally, the background activation suppression loss

\mathcal{L}_{\mathrm{BAS}}

is calculated using

\boldsymbol{\ell}_{\hskip-0.65556ptP}

and

\boldsymbol{\ell}_{b}

. Using these losses, the classification head learns to classify

\boldsymbol{F}

while ignoring background features.

Pathology	Anatomical Regions
Atelectasis	Left Lung, Right Lung
Cardiomegaly	Heart
Consolidation	Left Lung, Right Lung
Edema	Left Lung, Right Lung
Effusion	Left Lung, Right Lung
Emphysema	Left Lung, Right Lung

Table 1: Anatomical regions of pathologies in the NIH dataset, as identified by a diagnostic radiologist.

Loss Weight	Value
$\alpha_{C}$	2.2674
$\alpha_{\hskip-0.65958ptP}$	2.4678
$\alpha_{\negthinspace f}$	3.9670
$\beta$	0.4377
$\gamma$	0.1427

Table 2: Final optimal scalar loss weights selected to maximize validation classification and weakly-supervised localization. These values are used for all experiments.

Model	Image Size	NIH Training Set Size	Macro AUC	Macro IoU Accuracy at T(IoU)
				0.1	0.2	0.3	0.4	0.5
RGT 1 ‡	224	70%	$0.839$ †	0.591	0.424	0.281	0.173	0.090
ThoraX-PriorNet-224 2 ‡	224	70%	$0.844$	0.666	0.509	0.398	0.296	0.201
ThoraX-PriorNet-512 2 ‡	512	70%	0.847	0.803	0.626	0.488	0.334	0.217
Wang et al. (2024) ‡	224	–	–	0.450	–	0.230	–	–
Wang et al. (2017)	1024	70%	$0.745$	0.568	0.373	0.221	0.116	0.062

Table 3: Classification and weakly-supervised localization performance of different models evaluated on the NIH test set. Classification results are macro averaged across all 14 pathologies, and the localization results are macro averaged across the 8 pathologies with bounding box labels. Localization results for the CLARiTy models are obtained with the calibrated OneBox extractor. The highest metric values are shown in bold. The results of cited models are taken from their publications.

Model	Pathology AUC
	Atel.	Card.	Cons.	Edem.	Effu.	Emph.	Fibr.	Hern.	Infi.	Mass	Nodu.	Pleu.	Pn. 1	Pn. 2	Mean
Yao et al. (2017) ‡	$0.772$	$0.904$	$0.788$	$0.882$	$0.859$	$0.829$	$0.767$	$0.914$	$0.695$	$0.792$	$0.717$	$0.765$	$0.713$	$0.841$	$0.798$
Li et al. (2018) ‡	$0.800$	$0.870$	$0.800$	$0.880$	$0.870$	$0.910$	$0.780$	$0.770$	$0.700$	$0.830$	$0.750$	$0.790$	$0.670$	$0.870$	$0.806$
Ho and Gwak (2019) ‡	$0.795$	$0.887$	$0.786$	$0.892$	$0.875$	$0.875$	$0.756$	$0.836$	$0.703$	$0.835$	$0.716$	$0.774$	$0.742$	$0.863$	$0.810$
Ho and Gwak (2019) ¶	$0.726$	$0.865$	$0.717$	$0.812$	$0.810$	$0.740$	$0.748$	$0.803$	$0.682$	$0.773$	$0.686$	$0.730$	$0.660$	$0.796$	$0.753$
RGT 1 ‡	$0.800$ †	$0.920$ †	–	–	$0.780$ †	–	–	–	$0.860$ †	$0.880$ †	$0.880$ †	–	$0.790$ †	$0.810$ †	$0.839$ †

Table 4: Classification performance comparison across different models evaluated on the NIH test set. The highest values are shown in bold for models that classify all 14 pathologies. The results of cited models are taken from their publications.

T(IoU)	Model	Pathology IoU Accuracy
		Atel.	Card.	Effu.	Infi.	Mass	Nodu.	Pn. 1	Pn. 2	Mean
0.1	RGT 1 ‡	$0.610$	$0.950$	$0.650$	$0.820$	$0.500$	$0.130$	$0.790$	$0.280$	$0.591$
	ThoraX-PriorNet-224 2 ‡	$0.628$	1.000	$0.791$	$0.862$	$0.518$	$0.127$	$0.825$	$0.577$	$0.666$
	ThoraX-PriorNet-512 2 ‡	$0.761$	1.000	$0.837$	$0.870$	$0.741$	$0.582$	$0.867$	0.763	$0.803$
	Wang et al. (2024) ‡	$0.250$	$0.990$	$0.490$	$0.630$	$0.270$	$0.030$	$0.680$	$0.300$	$0.450$
[1pt/1.5pt]	Wang et al. (2017)	$0.689$	$0.938$	$0.660$	$0.707$	$0.400$	$0.139$	$0.633$	$0.378$	$0.568$

Table 5: Weakly-supervised localization performance comparison across different models on the bounding-box-annotated images of the NIH test set. For the CLARiTy models, localization is computed on all images of the localization test split, irrespective of whether the pathology was correctly classified. Localization results for the CLARiTy models are obtained with the calibrated OneBox extractor. The highest values for each T(IoU) are shown in bold. The results of cited models are taken from their publications.

SegmentCAM	Dice	Pretrained Weights		Classification Loss		Class Token Pooling		Validation Metric
		ImageNet	DINO	MLSM	WBCE	GAP	Attention	Macro AUC	MSLI	UDP
✓	✓	–	✓	–	✓	–	✓	0.836	0.391	0.614
–	–	–	✓	–	✓	–	✓	0.841	0.339	0.590
✓	–	–	✓	–	✓	–	✓	0.833	0.302	0.567
✓	✓	✓	–	–	✓	–	✓	0.826	0.363	0.595
✓	✓	–	✓	✓	–	–	✓	0.841	0.382	0.611

Table 6: Component ablations on the CLARiTy-S-16-224. Classification and weakly-supervised localization performance was evaluated on the validation set, with localization evaluated using the calibrated OneBox extractor. The highest values are shown in bold.

Figure 10: Selected localization outputs from the CLARiTy-S-16-224. For each pathology, two chest X-rays at $224\times 224$ are shown. All chest X-rays belong to the localization-test set. The model correctly predicted the presence of the pathology in each image. Class-specific outputs: foreground mask, attention map, localization heatmap, and bounding boxes. The predicted bounding boxes are obtained with the calibrated OneBox extractor. Each output is overlaid on the input chest X-ray. Ground-truth bounding boxes: green. Predicted bounding boxes: red.

Image Size	Component	Parameters	FLOPS
224	Vision Transformer Backbone	21.68 M	4.95 G
	Attention Pooling Module	5.38 K	5.38 K
	SegmentCAM Module	2.94 M	0.87 G
	CLARiTy-S-16-224	24.62 M	5.83 G
512	Vision Transformer Backbone	21.99 M	32.41 G
	Attention Pooling Module	5.38 K	5.38 K

Table 7: Computing resources of the CLARiTy-S-16-224, CLARiTy-S-16-512, and their components.

Figure 11: Validation UDP as a function of the number of fused attention layers

p

for the CLARiTy-S-16-224 and CLARiTy-S-16-512 variants. Each point is the highest validation UDP achieved during training for that layer count, and the star marks the maximum of each curve. The maximum validation UDP is reached at

p=5

for CLARiTy-S-16-224 and at

p=3

for CLARiTy-S-16-512.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

GT box area	Instances	224	512
$<0.5\%$	36	0.056	0.167
$0.5$ – $1\%$	38	0.053	0.105
$1$ – $2\%$	75	0.173	0.173
$2$ – $5\%$	108	0.157	0.111
$5$ – $10\%$	87	0.287	0.195
$\geq 10\%$	149	0.638	0.644

Appendix

Table 1: Detection accuracy at

\text{T(IoU)}=0.5

by ground-truth box area (fraction of the image), pooled over the 493 localization-test instances.

Finding	Instances	Accuracy	Missed (small/mirr.)	Extra boxes
Atelectasis	91	0.121	80 (12/2)	1.58
Cardiomegaly	73	1.000	0 (0/0)	0.00
Effusion	77	0.078	71 (7/4)	1.77
Infiltration	62	0.274	45 (2/2)	1.35
Mass	43	0.326	29 (10/0)	1.09
Nodule	39	0.154	33 (31/0)	1.79

Appendix

Table 2: Localization failures at

\text{T(IoU)}=0.5

on all 493 localization-test instances, per variant. “Missed” counts instances without a matching prediction and is followed in parentheses by the number of misses with a small ground-truth box (less than 1% of the image area) and with a mirrored prediction (matching the left–right mirrored ground-truth box). “Extra boxes” is the mean number of predicted boxes per instance that do not match the instance’s ground-truth box.

Group	Images	224	512
All	25,595	0.808	0.825
Male	14,881	0.809	0.825
Female	10,714	0.806	0.824
PA view	11,096	0.803	0.822
AP view	14,499	0.786	0.803
Age $<40$	8,411	0.749	0.770

Appendix

Table 1: Classification Macro AUC by subgroup on the official NIH test set.

Finding	$\sigma$	$t_{\mathrm{lo}}$	$t_{\mathrm{hi}}$	$s$	$\alpha$
Atelectasis	0.00	0.75	0.80	0.25	1.10
Cardiomegaly	4.00	0.05	0.40	1.25	1.00
Effusion	0.00	0.70	0.75	1.00	1.10
Infiltration	2.00	0.65	0.80	0.50	1.10
Mass	1.00	0.70	0.75	0.50	1.10
Nodule	2.00	0.75	0.80	0.75	0.90

Appendix

Table 1: Calibrated OneBox parameters for the CLARiTy-S-16-224 and CLARiTy-S-16-512 variants.

\sigma

is given in pixels at the model’s input resolution; the underlying search grid is relative to the image side length. The minimum gate area is four pixels for both variants.

				Macro IoU Accuracy at T(IoU)
Model	Extractor	Boxes/inst	[email protected]	0.1	0.2	0.3	0.4	0.5
CLARiTy-S-16-224	Wang	3.89	3.55	0.830	0.730	0.590	0.424	0.302
	OneBox	1.60	1.29	0.780	0.667	0.529	0.401	0.303
	Wang, top-2	2.00	1.71	0.672	0.582	0.467	0.353	0.255
	OneBox, top-2	1.52	1.21	0.768	0.658	0.521	0.397	0.298
	Wang, top-1	1.00	0.84	0.507	0.375	0.252	0.200	0.144

Appendix

Table 1: Extractor comparison on the localization test split (438 images, 493 bounding-box instances): macro IoU accuracy over the eight bounding-box findings, mean predicted boxes per instance, and unmatched boxes per instance at

\text{T(IoU)}=0.5

(a false-positive rate). The localization baseline of Wang et al. (2017) (per-image thresholding at

60/255

and

180/255

with connected components) is evaluated on the same maps; top-

k

keeps the

k

highest-peak boxes per instance. The matched-budget comparison is between the OneBox and Wang top-

k

rows.

Metric	$\Delta$	95% CI	$p$
Macro AUC	+0.0169	[0.015, 0.019]	< 0.0005
IoU Acc. @0.1	+0.0422	[0.009, 0.077]	0.0130
IoU Acc. @0.2	+0.0294	[-0.007, 0.066]	0.1170
IoU Acc. @0.3	+0.0273	[-0.011, 0.067]	0.1720
IoU Acc. @0.4	+0.0219	[-0.018, 0.061]	0.2700
IoU Acc. @0.5	-0.0034	[-0.037, 0.029]	0.8430

Appendix

Table F.1: Paired (common-resample) bootstrap differences for CLARiTy-S-16-512 minus CLARiTy-S-16-224, for the Macro AUC and the Macro IoU Accuracy at each

\mathrm{T(IoU)}

\Delta

is the mean difference across 2,000 paired resamples; the 95% confidence interval is the percentile interval of the per-draw differences and

p

is the two-sided bootstrap sign test. Classification uses the official NIH test split; localization uses the localization test split (438 images, 493 instances).

$\mathrm{T(IoU)}$	CLARiTy-S-16-224	CLARiTy-S-16-512
0.1	0.780 [0.745, 0.814]	0.822 [0.789, 0.854]
0.2	0.667 [0.631, 0.705]	0.698 [0.662, 0.733]
0.3	0.529 [0.491, 0.568]	0.552 [0.516, 0.596]
0.4	0.401 [0.365, 0.436]	0.419 [0.384, 0.459]
0.5	0.303 [0.271, 0.333]	0.295 [0.263, 0.331]

Appendix

Table F.2: 95% stratified percentile bootstrap confidence intervals for the Macro IoU Accuracy over the 8 bounding-box pathologies at each

\mathrm{T(IoU)}

on the localization test split (438 images, 493 instances; 2,000 image-level resamples). Each cell gives the point estimate followed by the 95% interval.

Pathology	CLARiTy-S-16-224	CLARiTy-S-16-512	ConvNeXtV2-B-224	ConvNeXtV2-B-512
Atelectasis	0.765 [0.757, 0.773]	0.791 [0.783, 0.798]	0.772 [0.764, 0.780]	0.787 [0.779, 0.794]
Cardiomegaly	0.902 [0.894, 0.909]	0.902 [0.894, 0.909]	0.892 [0.884, 0.900]	0.898 [0.890, 0.905]
Consolidation	0.760 [0.751, 0.770]	0.764 [0.754, 0.773]	0.750 [0.740, 0.759]	0.758 [0.749, 0.767]
Edema	0.852 [0.843, 0.861]	0.856 [0.847, 0.864]	0.853 [0.844, 0.861]	0.855 [0.846, 0.863]
Effusion	0.833 [0.827, 0.839]	0.837 [0.831, 0.843]	0.828 [0.822, 0.834]	0.832 [0.826, 0.837]
Emphysema	0.895 [0.885, 0.904]	0.936 [0.929, 0.942]	0.900 [0.891, 0.909]	0.938 [0.931, 0.944]

Appendix

Table 3: 95% stratified percentile bootstrap confidence intervals for the per-pathology and macro AUC of CLARiTy-S-16-224, CLARiTy-S-16-512 and their ConvNeXtV2-B teachers on the official NIH test split (25,595 images; 2,000 image-level resamples). Each cell gives the point estimate followed by the 95% interval.

Chest X-Ray Classification

Chest X-ray classification uses machine learning to detect and categorize diseases and findings in radiographs for screening, triage, and clinical decision support.

See all 101 papers

Sep 27, 2026

Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays

Weakly-supervised thoracic disease detection in chest X-rays (CXR) is challenging due to subtle appearances and complex anatomical overlap, motivating anatomy-aware modeling for improved localization. However, prior anatomy-aware methods typically rely on coarse region proxies or static spatial priors, which may restrict dynamic instance discovery and limit precise localization of small abnormalities. We propose Anatomy-Structured Hierarchical Multiple Instance Learning (ASH-MIL), a framework that introduces parallel anatomy-structured observation branches (cardiac, pulmonary, and agnostic) combined with hierarchical MIL aggregation. Anatomical priors are injected as soft spatial biases into decoder cross-attention, enabling anatomically grounded evidence maps without disease bounding-box supervision. Instance localization is derived directly from MIL-weighted cross-attention maps without bounding box supervision. Experiments on CXR8 and cross-domain MIMIC-CXR held-out sets demonstrate consistent improvements over prior weakly-supervised and anatomy-aware approaches, particularly under stricter localization criteria. Our code is available at https://github.com/jn-kim/ash-mil.

Jeongin Kim, Sohyun Ahn, Seo Young Kang +6

Division of Artificial Intelligence & Software, Ewha Womans University · Ewha Medical Artificial Intelligence Research Institute, Ewha Womans University · Department of Nuclear Medicine, Ewha Womans University +2

Chest X-Ray Classification Medical Image Analysis Medical Image Localization Multiple Instance Learning

Ask PDF

Aug 10, 2026

C $^2$ A: Coupling Spatial Evidence with Clinical Priors via Co-occurrence Aware Class Attention for Multi-Label Chest X-Ray Classification

Thoracic pathologies rarely occur in isolation, yet standard multi-label classifiers rely on shared global descriptors, discarding where findings lie and how they co-occur. We propose \textbf{C

\mathbf{^2}

A} (Co-occurrence Aware Class Attention), a classification head that explicitly couples spatial evidence with clinical priors. First, C

^2

A casts pooling as an expectation over learned per-class spatial attention maps, yielding localized descriptors for each disease. Second, it couples these descriptors via a learnable graph warm-started from empirical label co-occurrence. A single residual message-passing step shares evidence among related findings, proving to be a bounded perturbation of the identity where co-occurrence enters each logit through an explicit bilinear interaction. On CheXpert, C

^2

A achieves a superior

0.895

macro-mean AUROC, outperforming advanced context-gating baselines. Crucially, gains concentrate on highly co-occurrent classes with ambiguous spatial evidence (rescuing Atelectasis by

+1.5

over GCG), demonstrating the prior's regularizing effect with a negligible overhead of one linear projection and a

C\!\times\!C

edge matrix.

Akash Gogineni, Nagur Shareef Shaik, Aasrith Mandava +2

Department of Computer Science, Georgia State University, Atlanta, GA 30324, USA · UST Global Inc., Aliso Viejo, CA 92656, USA

Chest X-Ray Classification Multi-Label Classification

Ask PDF

Jul 5, 2026

PulmoSight-XAI: An Explainable Multi-View Attention Ensemble with Gradient Boosting Meta-Learning for Multi-Label Chest X-Ray Classification

Automated chest X-ray classification remains challenging due to severe class imbalance, co-occurring pathologies, and the loss of localized features in conventional architectures. To address these, we propose an explainable hierarchical multi-view ensemble framework for the robust classification of 14 thoracic pathologies. The framework employs view-specific training by independently modeling frontal and lateral radiographs using an ensemble of five complementary convolutional neural networks. Replacing global average pooling, a multi-scale feature fusion strategy augmented with Convolutional Block Attention Modules (CBAM) preserves fine-grained intermediate representations while emphasizing high-level pathology-specific semantic features. To mitigate positive-negative imbalance and varying inter-class difficulty, models are optimized using a novel hybrid objective combining Asymmetric Loss with Adaptive Focal Loss. Beyond simple probability averaging, the framework incorporates a hierarchical meta-learning strategy where test-time augmentation (TTA) predictions and cross-model uncertainty measures are integrated into Level-1 gradient-boosting meta-learners (XGBoost, LightGBM, and CatBoost), followed by Level-2 stacking with optimized alpha blending. Evaluated on a large-scale CheXpert-style dataset, the framework achieves state-of-the-art macro-average AUROC scores of 0.9319 for frontal and 0.9154 for lateral radiographs. Furthermore, comprehensive explainability analysis using seven post-hoc attribution techniques demonstrates strong anatomical consistency and clinically meaningful decision localization. By integrating architectural diversity, multi-scale attention, hierarchical meta-learning, and rigorous explainability, the proposed framework provides a transparent, highly accurate, and clinically practical computer-aided diagnosis system for thoracic disease classification.

Moshiur Rahman, Shafqat Alam, Tasnia Binte Mamun

Department of Biomedical Engineering Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh

Class-Imbalanced Learning Chest X-Ray Classification Multi-Label Classification Multi-View Learning Deep Ensembles Explainable Medical Image Analysis

Ask PDF

Abstract

Chest X-Ray Classification

Explore similar work

Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays

C2^22A: Coupling Spatial Evidence with Clinical Priors via Co-occurrence Aware Class Attention for Multi-Label Chest X-Ray Classification

PulmoSight-XAI: An Explainable Multi-View Attention Ensemble with Gradient Boosting Meta-Learning for Multi-Label Chest X-Ray Classification

C $^2$ A: Coupling Spatial Evidence with Clinical Priors via Co-occurrence Aware Class Attention for Multi-Label Chest X-Ray Classification