TAN: A Size- and Form-Adaptive Masked-Convolution Encoder for Tissue Graphs

Jan Herschel; Anselm von Gladiss

cs.CVOct 16, 2026

TAN: A Size- and Form-Adaptive Masked-Convolution Encoder for Tissue Graphs

Authors: Jan Herschel, Anselm von Gladiss

Organizations: Institute for Computer Science, University of Koblenz, Germany · MTI Mittelrhein, University of Koblenz, Germany

Abstract

Tissue graphs represent histology images as graphs over tissue-conforming superpixels, which keeps homogeneous tissue regions intact instead of cutting them into rectangular patches. To embed superpixels, current encoders crop them to bounding boxes and then pad, resize, or mask them, so background or fill values can leak into the node features. This matters in practice, and especially in medical imaging, where a prediction should depend on the tissue and not on an arbitrary preprocessing choice: on BACH, changing only the fill of the masked region shifts the accuracy of frozen encoders by up to 6 percentage points, larger than the differences among the four best encoders under mean fill, and the preferred fill differs between datasets, so it cannot be fixed once and for all. We present TAN (Tissue-Aware Network), a compact convolutional encoder (0.41M parameters) that processes each superpixel at its native shape and extent and excludes background from every operation, which makes its embeddings exactly invariant to the fill by construction. Under identical masks and an identical graph classifier, TAN's accuracy on BACH is not significantly different from that of the 303M-parameter foundation model UNI (81.1% vs. 80.7-82.6%, depending on the fill), with ~740x fewer parameters, ~1,190x fewer FLOPs, and no pretraining. On BRACS, TAN is close to a ResNet18 trained under the same protocol and trails the histopathology foundation models. An ablation with TAN's architecture but standard processing of fill-padded crops shows that masking keeps accuracy similar (nominally higher on BACH, lower on BRACS, within noise) at less than half the FLOPs.

Figures & tables

Figure 1: TAN encoder: four stages of masked convolution (SolidConv), mask-aware GroupNorm, LeakyReLU, and masked

2{\times}2

max-pooling (32, 64, 128, 128 channels), then masked attention pooling to a 1024-d node embedding. The mask is downsampled alongside the features; invalid positions are ignored in every operation.

Figure 2: A superpixel from a BRACS test image passing through a trained TAN. Top row: input crop with the superpixel outline and, for stages 1–4, the channel mean of each stage’s output. Bottom row: the mask and the strongest channel of each stage, i.e., the channel with the largest spatial variance. Dark gray: outside the mask; light gray: inside the mask but without a fully valid $3{\times}3$ neighborhood (bias only). Panels are scaled independently.

Encoder	Params	GFLOPs	Mean fill	White fill
TAN	0.41M	0.10	$81.1\pm 4.5$
TAN-unmasked	0.41M	0.22	$80.9\pm 3.9$	$79.3\pm 4.2$
UNI	303.4M	119.3	$80.7\pm 3.7$	$82.6\pm 4.4$
ResNet18-scratch	11.2M	1.18	$76.8\pm 3.7$	$75.6\pm 6.2$
CTransPath	27.5M	8.99	$71.2\pm 6.2$	$75.6\pm 5.5$
ResNet18-ImageNet	11.2M	1.18	$65.2\pm 5.2$	$71.3\pm 4.8$

Table 1: BACH, 15-fold repeated CV, accuracy (mean

\pm

std, %). GFLOPs: forward pass per superpixel (1 multiply-accumulate, MAC, = 2 FLOPs; mean of 300 sampled superpixels; PyTorch FLOP counter; excludes training and the GAT). TAN’s parameters include its classifier head (0.28M without). RF baseline:

62.5\pm 5.2

	Accuracy		Macro-F1
Encoder	Mean	White	Mean	White
TAN	$45.8\pm 3.5$		$42.0\pm 3.0$
TAN-unmasked	$49.7\pm 4.1$	$46.4\pm 3.0$	$46.4\pm 3.5$	$43.8\pm 2.3$
UNI	$53.3\pm 2.1$	$49.4\pm 2.4$	$50.7\pm 2.3$	$46.4\pm 2.6$
ResNet18-scratch	$47.0\pm 2.1$	$45.2\pm 2.4$	$43.6\pm 2.3$	$42.1\pm 3.8$
CTransPath	$50.5\pm 3.7$	$47.5\pm 2.2$	$47.7\pm 3.0$	$45.3\pm 1.4$

Table 2: BRACS, official split, 5 seeds (mean

\pm

population std over seeds, %). RF baseline: 32.0 / 26.1.

Medical Image Representation Learning

Medical image representation learning builds reusable visual features for diagnosis, segmentation, retrieval, and other clinical imaging tasks.

See all 257 papers

Sep 3, 2026

TAP-Path: Task-Adaptive Structural and Token Pruning for Efficient and Trustworthy Pathology Foundation Models

Pathology foundation models improve transferable representation learning for histopathology, but recent gains often rely on encoders with hundreds of millions of parameters and high inference cost. We propose TAP-Path, a task-adaptive compression framework that directly restructures a pretrained Virchow2 encoder rather than distilling it into a separate student. TAP-Path combines validation-driven transformer-block selection, physical removal of redundant blocks, input-adaptive patch-token pruning, multi-depth feature recovery, and a lightweight gated task head. The final model retains 24 of 32 transformer blocks and 70% of patch tokens after pruning, reducing encoder parameters by 24.96% (631.24M to 473.70M) and analytical encoder compute by 35.20% (340.13G to 220.40G FLOPs). Across three task-head optimization seeds, TAP-Path achieved

87.98 \pm 0.067%

test accuracy,

81.26 \pm 0.49%

balanced accuracy, and

82.38 \pm 0.48%

macro-F1 on a 32-class histopathology benchmark, compared with 86.89% for full Virchow2 and 87.67% for UNI2-h. TAP-Path achieved a Brier score of

0.1800 \pm 0.0005

and failure-detection AUROC of

0.9047 \pm 0.0060

. A validation-only rare-aware objective improved rare-class balanced accuracy in a secondary operating analysis. Frozen external evaluation on 433 CPTAC samples yielded

91.22 \pm 0.83%

accuracy and

91.10 \pm 0.81%

balanced accuracy. These results show that task-adaptive structural and token sparsification can improve the accuracy-efficiency trade-off of large pathology foundation models while preserving reliability under internal and external evaluation.

Mehedi Hasan, Ashfak Yeafi, Md Khairul Islam

Department of Computer Science and Engineering, Brac University, Dhaka, Bangladesh · EEE, Khulna University of Engineering & Technology, Khulna, Bangladesh · Mathematics and Computer Science, Hobart and William Smith Colleges, Geneva, NY, USA

Computational Pathology Pathology Foundation Models Transformer Pruning Histopathology Image Classification Visual Token Pruning Neural Network Compression+1

Ask PDF

Feb 24, 2026

GrapHist: Graph Self-Supervised Learning for Histopathology

Self-supervised vision models have achieved notable success in digital pathology. However, their domain-agnostic transformer architectures are not originally designed to account for fundamental biological elements of histopathology images, namely cells and their complex interactions. In this work, we hypothesize that a biologically-informed modeling of tissues as cell graphs offers a more efficient representation learning. Thus, we introduce GrapHist, a novel graph-based self-supervised learning framework for histopathology, which learns generalizable and structurally-informed embeddings that enable diverse downstream tasks. GrapHist integrates masked autoencoders and heterophilic graph neural networks that are explicitly designed to capture the heterogeneity of tumor microenvironments. We pre-train GrapHist on a large collection of 11 million cell graphs derived from breast tissues and evaluate its transferability across in- and out-of-domain benchmarks. Our results show that GrapHist achieves competitive performance compared to its vision-based counterparts in slide-, region-, and cell-level tasks, while requiring four times fewer parameters. It also drastically outperforms fully-supervised graph models on cancer subtyping tasks. Finally, we also release five graph-based digital pathology datasets used in our study at https://huggingface.co/ogutsevda/datasets , establishing the first large-scale graph benchmark in this field. Our code is available at https://github.com/ogutsevda/graphist .

Sevda Öğüt, Cédric Vincent-Cuaz, Natalia Dubljevic +4

LTS4, EPFL, Lausanne, 1015, Vaud, Switzerland

Computational Pathology Graph SSL Medical Image Representation Learning Graph Representation Learning Graph Neural Networks

Ask PDF

May 31, 2026

AMN: An Adaptive Multi-Scale Fusion Network with Boundary and Uncertainty Modeling for Nuclei Segmentation

Accurate classification of nuclei subtypes in histopathology images is critical for downstream tasks including tumor grading, immune infiltrate quantification, and prognosis prediction. Existing approaches rely on either convolutional or transformer-based encoders in isolation, limiting their ability to simultaneously capture fine-grained local texture and long-range spatial context. We present AMN (Adaptive Multi-Scale Nuclei Network), a dual-encoder segmentation framework that jointly leverages a Swin Transformer and a ResNet-50 feature pyramid, fused via a learned per-channel gating mechanism that dynamically weighs each encoder's contribution at every scale. AMN is trained with a multi-objective loss combining class-weighted focal loss, boundary-aware loss with positive-pixel emphasis, and a novel uncertainty-modulated classification term that suppresses overconfident erroneous predictions. Evaluated on the CoNIC benchmark across seven nuclei classes, AMN achieves a mean Dice of 0.82 and mean F1 of 0.68, with an F1 of 0.67 on the diagnostically challenging lymphocyte class. AMN outperforms eight baseline models spanning pure-CNN, pure-transformer, and recent hybrid architectures: U-Net, ResU-Net, DeepLabV3+, SegNet, ViT-Small, HmsU-Net, ConvFormer-UNet, and BEFUnet. Cross-dataset evaluation on MoNuSeg demonstrates strong generalization without retraining and validating the domain robustness of the learned representations.

Spoorthi M, Suja Palaniswamy

Department of Computer Science & Engineering, Amrita School of Computing, Bengaluru, Amrita Vishwa Vidyapeetham, India

Histopathology Image Segmentation Hybrid CNN-Transformer Architectures Multi-Scale Feature Fusion Medical Image Segmentation

Ask PDF

Abstract

Medical Image Representation Learning

Explore similar work

TAP-Path: Task-Adaptive Structural and Token Pruning for Efficient and Trustworthy Pathology Foundation Models

GrapHist: Graph Self-Supervised Learning for Histopathology

AMN: An Adaptive Multi-Scale Fusion Network with Boundary and Uncertainty Modeling for Nuclei Segmentation