TRiST: Topology-Reinforced Swin Transformer for Medical Image Analysis
Authors: Pengfei Gu, Huimin Li, Guangyu Meng, Hao Zheng, Haoteng Tang, Bin Fu, Danny Z. Chen
Organizations: Department of Computer Science, The University of Texas Rio Grande Valley · School of Mathematical and Statistical Sciences, The University of Texas Rio Grande Valley · Department of Computer Science and Engineering, University of Notre Dame · School of Computing and Informatics, University of Louisiana at Lafayette
Medical images contain topological patterns, such as connected components and loops, at scales ranging from small lesions to organ boundaries. Although hierarchical Vision Transformers have advanced medical image analysis, they do not explicitly incorporate topology throughout their multiscale representations. We propose Topology-Reinforced Swin Transformer (TRiST), a framework that integrates topological information into feature construction, refinement, and attention. First, a Hierarchical Multi-Scale Topology Representation algorithm computes 2D persistent homology (H0 and H1) independently at four native Swin stages, producing descriptors aligned with each stage's token grid rather than downsampling fine-scale topology. Second, a Band-Aware H0/H1 Residual Refinement module uses gated residual networks to process short-, medium-, and long-persistence information separately. Third, a Persistence-Band Progressive Attention Bias converts stage-specific descriptors into learned key-saliency scores that directly modify shifted-window attention logits. We instantiate TRiST as Topo-SwinV2-B for classification and Topo-Swin-UNet for segmentation, preserving the underlying Swin-based backbones. Experiments on five medical image datasets show consistent improvements over Swin-based baselines and competitive performance relative to existing topology-augmented methods.
Figures & tables
Figure 1: (a) The overall architecture of our Topology-Reinforced Swin Transformer (TRiST). In parallel to the Swin backbone, we pre-compute hierarchically aligned multi-scale topological representations at four native scales. (b) The structure of a TRiST block. Each stage-specific topological representation is normalized and refined by the proposed Band-Aware H0/H1 Residual Refinement module, and then decomposed into persistence bands to construct an additive topology bias for window attention. The resulting topology bias is injected into the attention logits together with the relative position bias. Stages 1, 2, and 3–4 use short, medium, and long bands, respectively.
Table 5: Trainable parameter breakdown for the topology modules added to Topo-SwinV2-B. The Swin-V2-B backbone has 87,930,848 parameters. The total topology overhead is 0.095% of the backbone.
Figure 3: GradCAM visualizations comparing Topo-SwinV2-B w/ PS (top row) and SwinV2-B (bottom row) on a Severe DR (Grade 3) fundus image. Columns show the raw image followed by activation maps from Stage 1 to Stage 4. Warmer colors indicate stronger GradCAM attribution for the target prediction.
Sparse vision transformers have gained popularity as efficient encoders for medical volumetric segmentation, with Swin emerging as a prominent choice. Swin uses local attention to reduce complexity and yields excellent performance for many tasks but still tends to overfit on small datasets. To mitigate this weakness, we propose a novel architecture that further enhances Swin's inductive bias by introducing Inception blocks in the feed-forward layers. The introduction of these multi-branch convolutions enables more direct reasoning over local, multi-scale features within the transformer block. We have also modified the decoder layers in order to capture finer details using fewer parameters. We demonstrate a performance improvement on eleven different medical datasets through extensive experimentation. We specifically showcase advancements over the previous state-of-the-art backbones on benchmark challenges like the Medical Segmentation Decathlon and Beyond the Cranial Vault. By showing that the existing inductive bias in Swin can be further improved, our work presents a promising avenue for enhancing the capabilities of sparse vision transformers for both medical and natural image segmentation tasks. Code and pre-trained weights can be accessed at https://github.com/Eiphodos/SwInception.
David Hagerman, Roman Naeem, Jakob Lindqvist +3
Chalmers University of technology, 412 96 Gothenburg, Sweden · Zenseact, Lindholmspiren 2, 417 56 Gothenburg, Sweden
Deep clustering offers a promising approach for analyzing large unlabeled medical image datasets, yet prevailing methods still face two limitations: they may neglect class-agnostic structural priors that capture object boundaries and regions, and they do not explicitly encode topological patterns (e.g., connectivity and holes). These sources of information play distinct and complementary roles: visual-structural features provide the primary discriminative representation for clustering, while topological features provide additional global geometric cues for residual visual ambiguities. We introduce VTRL-Med, a progressive visual-structural learning framework with topology-guided regularization for medical image clustering. VTRL-Med first adapts the Segment Anything Model (SAM) to extract image features that retain structural cues inherited from mask-supervised pretraining. It then generates reliable pseudo-labels through a two-step filtering process that combines prototype-based selection with k-nearest neighbor consistency validation. Finally, it progressively refines the visual branch while an auxiliary topology branch, constructed from persistent homology (PH) descriptors, provides additional geometric regularization during training. Across 13 medical image benchmarks, VTRL-Med achieves state-of-the-art clustering performance, improving average accuracy by 4.48% over existing methods. On three representative datasets, topology-guided regularization further improves accuracy by 1.3--4.3% over visual refinement alone.
Guangyu Meng, Pengfei Gu, Peixian Liang +3
Dept. of Computer Science and Engineering, University of Notre Dame · Dept. of Computer Science, The University of Texas Rio Grande Valley · Dept. of IT, Analytics, and Operations, University of Notre Dame
Contrastive learning (CL) has become a powerful approach for learning representations from unlabeled images. However, existing CL methods focus predominantly on visual appearance features while neglecting topological characteristics (e.g., connectivity patterns, boundary configurations, cavity formations) that provide valuable cues for medical image analysis. To address this limitation, we propose a new topological CL framework (TopoCL) that explicitly exploits topological structures during contrastive learning for medical imaging. Specifically, we first introduce topology-aware augmentations that control topological perturbations using a relative bottleneck distance between persistence diagrams, preserving medically relevant topological properties while enabling controlled structural variations. We then design a Hierarchical Topology Encoder that captures topological features through self-attention and cross-attention mechanisms. Finally, we develop an adaptive mixture-of-experts (MoE) module to dynamically integrate visual and topological representations. TopoCL can be seamlessly integrated with existing CL methods. We evaluate TopoCL on five representative CL methods (SimCLR, MoCo-v3, BYOL, DINO, and Barlow Twins) and five diverse medical image classification datasets. The experimental results show that TopoCL achieves consistent improvements: an average gain of +3.26% in linear probe classification accuracy with strong statistical significance, verifying its effectiveness.
Guangyu Meng, Pengfei Gu, Peixian Liang +3
Dept. of Computer Science and Engineering, University of Notre Dame · The University of Texas Rio Grande Valley · Dept. of IT, Analytics, and Operations, University of Notre Dame