cs.LGDec 18, 2025

CLARiTy: A Vision Transformer for Multi-Label Classification and Weakly-Supervised Localization of Chest X-ray Pathologies

Authors: John M. Statheros, Hairong Wang, Richard Klein

Organizations: School of Computer Science and Applied Mathematics, University of the Witwatersrand, Johannesburg, South Africa

Abstract

The interpretation of chest X-rays poses significant challenges in achieving accurate multi-label pathology classification and spatial localization. These tasks demand different annotation granularities but are constrained by the scarcity of region-level (dense) annotations. We introduce CLARiTy (Class Localizing and Attention Refining Image Transformer), a vision transformer-based model for joint multi-label classification and weakly-supervised localization of thoracic pathologies. CLARiTy employs multiple class-specific tokens to generate discriminative attention maps and a SegmentCAM module for foreground segmentation and background suppression using explicit anatomical priors. Trained on image-level labels from the NIH ChestXray14 dataset, it leverages distillation from a ConvNeXtV2 teacher for efficiency. Localization heatmaps are converted to bounding boxes with OneBox, a hysteresis-gated extractor whose parameters are calibrated per finding on held-out validation data. Evaluated on the official NIH split, CLARiTy achieves competitive classification across 14 pathologies and strong weakly-supervised localization on 8 pathologies, outperforming prior methods by up to 50.7% relative Macro Intersection over Union (IoU) Accuracy at stringent thresholds. Gains are most pronounced for small pathologies: at T(IoU) = 0.5, the 224 variant reaches an IoU Accuracy of 0.154 on nodules, where the compared methods remain at or below 0.013. The lower-resolution CLARiTy-S-16-224 offers high efficiency while surpassing baselines, and could potentially be applied in low-resource settings. An ablation study confirms the contributions of SegmentCAM, DINO pretraining, and attention pooling. CLARiTy advances beyond CNN-ViT hybrids by harnessing ViT self-attention for global context and class-specific localization, refined through convolutional background suppression for precise, noise-reduced heatmaps.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays

    Sep 27, 2026Jeongin Kim, Sohyun Ahn, Seo Young Kang +6Chest X-Ray ClassificationMedical Image Analysis

  2. C2^2A: Coupling Spatial Evidence with Clinical Priors via Co-occurrence Aware Class Attention for Multi-Label Chest X-Ray Classification

    Aug 10, 2026Akash Gogineni, Nagur Shareef Shaik, Aasrith Mandava +2Chest X-Ray ClassificationMulti-Label Classification

  3. PulmoSight-XAI: An Explainable Multi-View Attention Ensemble with Gradient Boosting Meta-Learning for Multi-Label Chest X-Ray Classification

    Jul 5, 2026Moshiur Rahman, Shafqat Alam, Tasnia Binte MamunClass-Imbalanced LearningChest X-Ray Classification