Context-aware Skin Cancer Epithelial Cell Classification with Scalable Graph Transformers
Authors: Lucas Sancéré, Noémie Moreau, Katarzyna Bozek
Organizations: Faculty of Mathematics and Natural Sciences, University of Cologne, Cologne, North Rhine-Westphalia, Germany · Institute for Biomedical Informatics, Faculty of Medicine and University Hospital Cologne, University of Cologne, North Rhine-Westphalia, Germany · Center for Molecular Medicine Cologne (CMMC), Faculty of Medicine and University Hospital Cologne, University of Cologne, North Rhine-Westphalia, Germany · Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Cologne, North Rhine-Westphalia, Germany
Whole-slide images (WSIs) from cancer patients contain rich information that can be used for medical diagnosis or to follow treatment progress. To automate their analysis, numerous deep learning methods based on convolutional neural networks and Vision Transformers have been developed and have achieved strong performance in segmentation and classification tasks. However, due to the large size and complex cellular organization of WSIs, these models rely on patch-based representations, losing vital tissue-level context. We propose using scalable Graph Transformers on a full-WSI cell graph for classification. We evaluate this methodology on a challenging task: the classification of healthy versus tumor epithelial cells in cutaneous squamous cell carcinoma (cSCC), where both cell types exhibit very similar morphologies and are therefore difficult to differentiate for image-based approaches. We first compared image-based and graph-based methods on a single WSI. Graph Transformer models SGFormer and DIFFormer achieved balanced accuracies of 85.2±1.5 (± standard error) and 85.1±2.5 in 3-fold cross-validation, respectively, whereas the best image-based method reached 81.2±3.0. By evaluating several node feature configurations, we found that the most informative representation combined morphological and texture features as well as the cell classes of non-epithelial cells, highlighting the importance of the surrounding cellular context. We then extended our work to train on several WSIs from several patients. To address the computational constraints of image-based models, we extracted four 2560×2560 pixel patches from each image and converted them into graphs. In this setting, DIFFormer achieved a balanced accuracy of 83.6±1.9 (3-fold cross-validation), while the state-of-the-art image-based model CellViT256 reached 78.1±0.5.
Figures & tables
Figure 1 : Description of WSI-Graph. (a) Steps to generate WSI-Graph. First a WSI from cSCC patient is segmented using SCC Hovernet and tumor regions are annotated by an expert to refine segmentation. From this segmentation map we build a graph, simplify it around anchor nodes and optionally split it with K-means on centroid coordinate features. (b) Zoom into the graph, edges generated with threshold distance r0=50pixels corresponding to r0≈11.5\text{,}\mathrm{\SIUnitSymbolMicro m}$$ are shown. (c) Number of edges, nodes, node features and instances of given cell classes before and after simplification (here k=3 max-hops simplification).
Figure 2 : Description of TILE-Graphs. (a) Steps to generate TILE-Graphs. First, patches from cSCC patient sample extracted from tumor epithelial and healthy epithelial regions are segmented using SCC Hovernet. Then from these segmentation maps 372 graphs are built. (b) TILE-Graphs dataset statistics. It includes 372 patches from 93 samples from 84 patients. The resulting 372 graphs are then split keeping graphs of the same patients in the same split during cross-validation.
Method
Subgraphs
Random Nodes
DIFFormer
85.2 ± 1.5
91.1 ± 0.1
SGFormer
85.1 ± 2.5
94.9 ± 0.2
SIGN
80.5 ± 2.8
84.8 ± 0.6
GAT
80.4 ± 1.0
73.3 ± 2.0
SGCMLP
80.4 ± 2.7
79.3 ± 0.4
NodeFormer
79.0 ± 0.5
85.0 ± 1.4
Table 1 : Comparison of GNN models on binary epithelial node classification performance on simplified WSI-Graph. Balanced accuracy (%) is reported as mean ± standard error over 3-fold cross-validation under subgraph-based and random node evaluation protocols and k=3 max-hops simplification for the graph ( WSI-Graph (3) ).
Method
Training set
3-fold crossval
Hovernet
all patches
73.7 ± 1.4
epithelial only
79.3 ± 2.3
CellViT256
all patches
76.5 ± 1.9
epithelial only
81.2 ± 3.0
CellViT-SAM-B
all patches
OOM
epithelial only
OOM
Table 2 : Comparison of image-based models on binary epithelial cell classification performance on WSI images. Balanced accuracy (%) is reported as mean ± standard error over 3-fold cross-validation for different methods and training set configurations, using image-based representations.
Node Features
z-score norm
Subgraphs
Random Nodes
morphology
no
67.8 ± 3.6
92.6 ± 0.3
morphology
yes
79.6 ± 2.4
94.0 ± 0.5
morphology & texture
no
70.6 ± 9.0
94.0 ± 0.9
morphology & texture
yes
84.2 ± 1.6
94.3 ± 0.6
morphology & cell class
no
73.6 ± 3.7
93.7 ± 0.5
morphology & cell class
yes
84.0 ± 2.8
94.5 ± 0.4
Table 3 : Impact of node features on binary node classification performance on simplified WSI-Graph. Balanced accuracy (%) is reported as mean ± standard error over 3-fold cross-validation for different node features under subgraph and random node evaluation protocols with SGFormer model and WSI-Graph (3) .
Graph Simplifications
Subgraphs
Random Nodes
No simplification
82.2 ± 2.9
94.6 ± 0.1
50 max-hops
83.3 ± 3.3
94.9 ± 0.4
10 max-hops
86.6 ± 2.2
95.0 ± 0.2
5 max-hops
82.5 ± 1.9
94.6 ± 0.5
4 max-hops
82.1 ± 4.3
95.0 ± 0.4
3 max-hops
85.1 ± 2.5
94.9 ± 0.2
Table 4 : Impact of graph simplification on binary node classification performance on simplified WSI-Graph. Balanced accuracy (%) is reported as mean ± standard error over 3-fold cross-validation for different maximum hop thresholds under subgraph and random node evaluation protocols with SGFormer model.
Method
3-fold crossval
CellViT256
78.1 ± 0.5
DIFFormer
83.6 ± 1.9
NodeFormer
78.7 ± 4.4
SGCMLP
66.4 ± 1.1
SIGN
66.3 ± 1.0
SGC
63.8 ± 1.2
Table 5 : Comparison of graph-based and image-based models on binary epithelial cell classification performance on TILE-Graphs and baseline dataset. Balanced accuracy (%) is reported as mean ± standard error over 3-fold cross-validation for different methods and training set configurations.
Automated skin cancer classification from dermoscopic images remains challenging due to heterogeneous lesion structure, strong intra-class variability, and subtle visual differences between benign and malignant cases. Existing CNN/ViT pipelines typically rely on global or patch-level features and often combine patient metadata via late fusion, which limits spatially grounded multimodal reasoning. We present a novel region-based graph learning framework that explicitly models lesions as graphs of spatially coherent superpixel regions represented as frozen CNN features. To capture fine-grained lesion arrangements, we encode inter-regional geometry as edge attributes and introduce a dedicated metadata context node connected to all regions, providing structured integration of demographic/clinical variables within the same relational space. Node representations are updated using our edge-aware graph transformer followed by attention-driven propagation, and a final graph-level embedding for benign-malignant classification. Experiments on four public benchmarks demonstrate that explicit region-level relational modeling and graph-native multimodal fusion yield consistent gains over the state-of-the-art. Consequently, we establish a new graph-centric perspective in which CNN features are modeled as relational nodes and improved through contextual integration, yielding more expressive and robust classifications.
Muhammad Azeem, Tanveer Hussain, Amr Ahmed +1
Edge Hill University, Ormskirk, Lancashire, L39 4QP, United Kingdom
Vision Transformers (ViTs) and their hierarchical variants have achieved strong performance in Computational Pathology (CPath). However, most are pre-trained on single-resolution Whole Slide Images (WSIs), limiting their generalization across arbitrary resolutions. Gigapixel WSIs inherently contain diagnostic patterns at multiple scales, including cellular morphologies, tissue architectures, and global context, mirroring how expert pathologists examine WSIs. We introduce Multi-Resolution Pyramid Transformer (MRPT), a model that hierarchically aggregates multi-resolution information from cellular to tissue and WSI levels. MRPT employs a biologically meaningful Consecutive Cross-Resolution Attention (CCRA) mechanism to capture scale-independent interactions and enforces multi-resolution semantic consistency by aligning embeddings across resolutions, yielding robust and generalizable WSI representations. Pre-trained in a multi-resolution self-supervised manner on 624M patches, 2.4M regions, and 36K WSIs, MRPT learns rich coarse-to-fine histopathology features. Extensive experiments on 34 diverse datasets show that MRPT surpasses recent foundation models and Multimodal Large Language Models (MLLMs) in cancer subtype classification, tissue phenotyping, and Visual Question Answering (VQA) for WSI understanding.
Basit Alawode, Moshira Ali Abdalla, Dwarikanath Mahapatra +2
Khalifa University of Science and Technology, UAE · University of Western Australia, Australia
Cancer survival prediction from whole slide images (WSIs) is a challenging task in computational pathology due to the large size, irregular shape, and high granularity of the WSIs. These characteristics make it difficult to capture the full spectrum of patterns, from subtle cellular abnormalities to complex tissue interactions, which are crucial for accurate prognosis. To address this, we propose CrossFusion, a novel multi-scale feature integration framework that extracts and fuses information from patches across different magnification levels. By effectively modeling both scale-specific patterns and their interactions, CrossFusion generates a rich feature set that enhances survival prediction accuracy. We validate our approach across six cancer types from public datasets, demonstrating significant improvements over existing state-of-the-art methods. Moreover, when coupled with domain-specific feature extraction backbones, our method shows further gains in prognostic performance compared to general-purpose backbones. The source code is available at: https://github.com/RustinS/CrossFusion
Rustin Soraki, Huayu Wang, Sitong Liu +2
University of Washington, Seattle, WA · University of California, Los Angeles, CA