ADMIL: Attention-Distilled Multiple Instance Learning for Selective Foundation Model Inference in Pathology

Duncan Stothers; Ren-Chin Wu; William Lotter

cs.CVAug 22, 2026

ADMIL: Attention-Distilled Multiple Instance Learning for Selective Foundation Model Inference in Pathology

Authors: Duncan Stothers, Ren-Chin Wu, William Lotter

Organizations: Department of Data Science, Dana-Farber Cancer Institute, Boston, MA · Department of Pathology, Dana-Farber Cancer Institute, Boston, MA · Department of Pathology, Brigham and Women’s Hospital, Boston, MA · Harvard Medical School, Boston, MA

Abstract

Attention-based multiple instance learning (ABMIL) using pathology foundation model embeddings is effective for slide-level tasks, but exhaustive inference requires applying a large image encoder to every foreground tile despite the subsequent attention distribution often concentrating over a small subset of informative regions. We introduce ADMIL (Attention-Distilled Multiple Instance Learning), a selective-compute framework that distills an ABMIL teacher's attention into a lightweight tile-selection model, PriorNet. Using an EfficientNet architecture, PriorNet learns the teacher attention distribution from raw tile pixels with KL divergence; at inference, it scores the foreground pool, selects the top-K tiles, and invokes the expensive foundation model only on that subset before a selected-bag ABMIL student predicts the slide label. Across BRACS, PANDA, and CAMELYON16, ADMIL matches full-teacher headline performance at K=4, 8, and 128 tiles, respectively, avoiding >98% of foundation model (Virchow2) tile embeddings and model inference FLOPs. Random and teacher-attention oracle controls show that this result depends on task-relevant selection rather than tile-count reduction alone. Quantitative and qualitative analyses suggest that PriorNet recovers the teacher's tile ordering with high fidelity while focusing on task-relevant morphological regions. ADMIL shows that nearly all expensive tile encodings can be removed without sacrificing slide-level performance, providing a potential path for more efficient deployment in clinical settings where latency and compute costs are key considerations.

Figures & tables

Figure 1: ADMIL overview. Stage 1: A lightweight image encoder (PriorNet) learns to estimate the attention distribution produced by a frozen foundation model-based ABMIL teacher. Stage 2: PriorNet selects the top- $K$ tiles for generating foundation model embeddings, which are used to train a separate top- $K$ ABMIL predictor. At deployment, ADMIL reduces compute cost by decreasing the number of foundation model calls.

Figure 2: Slide-level performance versus selected tile budget. Columns correspond to CAMELYON16, PANDA, and BRACS. The top row reports each dataset’s primary metric and the bottom row its secondary metric. The black dotted line denotes full-teacher performance, which is compared to ADMIL, a random tile selector, and a top- $K$ teacher oracle. The red dashed vertical line indicates the $K$ value where ADMIL reaches teacher performance. The x-axis is logarithmic and reports both raw $K$ and the corresponding mean foreground fraction. Error bars correspond to 95% confidence intervals.

Figure 3: Teacher-support retrieval by PriorNet. Values correspond to the fraction of the teacher top- $K$ tiles found among the PriorNet top- $T$ tiles, averaged across test slides within each dataset. Cells with $T<K$ are masked. All panels use the same color scale.

Figure 4: Example attention maps. Each row contains a representative test slide from one dataset (left: CAMELYON16, right: PANDA). Panels show the WSI thumbnail, full-bag teacher attention, selected-bag student attention, and the top attended tiles.

Figure 5: Sensitivity to the PriorNet supervision tile budget. Each panel compares PriorNet models trained with 5%, 10%, or 20% of each slide’s foreground candidates. The black dotted line denotes the full-teacher reference. Error bars correspond to 95% confidence intervals. The 10% setting is the default used for main results; the other curves assess robustness to the supervision fraction.

Dataset	Metric	$K_{\mathrm{crit}}$	Full teacher	Teacher 95% CI	ADMIL at $K_{\mathrm{crit}}$	ADMIL 95% CI	FM saved (%)	FLOPs saved (%)
BRACS	Balanced Acc.	4	0.648	[0.594, 0.700]	0.670	[0.617, 0.720]	99.8	99.6
PANDA	QWK	8	0.854	[0.838, 0.870]	0.858	[0.842, 0.874]	98.5	98.3
CAMELYON16	AUROC	128	0.976	[0.954, 0.991]	0.978	[0.960, 0.992]	99.0	98.7

Table 1: Performance at the descriptive critical tile budget

K_{\mathrm{crit}}

. FM saved indicates the percentage of foundation model tile embeddings avoided. Model FLOPs saved accounts for PriorNet, foundation model, and ABMIL operations at inference.

Dataset	Pearson	Pearson 95% CI	Spearman	Spearman 95% CI
CAMELYON16	0.456	[0.425, 0.487]	0.799	[0.778, 0.821]
PANDA	0.622	[0.532, 0.713]	0.839	[0.816, 0.861]
BRACS	0.342	[0.246, 0.412]	0.579	[0.518, 0.628]

Table 2: Correlation between teacher attention and PriorNet scores. Values are computed per WSI and then averaged across WSIs.

Selective Inference

Selective inference develops valid statistical conclusions after data-dependent choices such as feature selection, clustering, or adaptive evaluation.

See all 23 papers

One Tile, Multiple Instances: Rethinking MIL for Sparse Diagnostic Evidence
Oct 4, 2026Runsheng Liu, Cheng Jin, Hao Jiang +1Multiple Instance Learning Whole-Slide Image Analysis
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China. · Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Hong Kong, China. · Division of Life Science, The Hong Kong University of Science and Technology, Hong Kong, China. +3
From Patches to Patients: A study of the tile-to-slide performance transferability in Digital Pathology
Jun 9, 2026Sofiène Boutaj, Leo Fillioux, Maria Vakalopoulou +2Pathology Foundation Models Medical Image Benchmarks
Université Paris-Saclay, CentraleSupélec, Gustave Roussy, INSERM, IHU PRISM, Cancer Data Science Unit, France · Université Paris-Saclay, CentraleSupélec, MICS Laboratory, France
Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models
Jul 16, 2026Mingxi Fu, Jiawen Li, Renao Yan +4Multi-Teacher Knowledge Distillation Multiple Instance Learning
Tsinghua Shenzhen International Graduate School, Tsinghua University · University of Washington · City University of Hong Kong (Dongguan) +2

Oct 4, 2026J/K move · Enter open · S save

One Tile, Multiple Instances: Rethinking MIL for Sparse Diagnostic Evidence

Runsheng Liu, Cheng Jin, Hao Jiang, Hao Chen

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China. · Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Hong Kong, China. · Division of Life Science, The Hong Kong University of Science and Technology, Hong Kong, China.+3

Multiple Instance Learning Whole-Slide Image Analysis Histopathology Image Classification

Open paper AskPDF arXiv

In weakly supervised Whole Slide Image (WSI) classification, feature extractors typically compress each image tile into a single global embedding. Consequently, slide-level aggregators are restricted to this coarse tile scale, concealing fine-grained sub-tile evidence from the attention mechanism. We introduce DI-MIL, a framework that decouples encoding context from instance granularity through decomposed instances. By clustering dense spatial tokens from a frozen foundation model within each tile, DI-MIL converts a single tile into multiple independently weighted instance embeddings. As a training-free post-encoding module, DI-MIL integrates seamlessly into existing pipelines without requiring re-encoding or downstream architectural modifications. We evaluate DI-MIL on cytopathology, a challenging testbed where sparse diagnostic signals are easily diluted within standard tiles. Across four datasets, three frozen foundation models, and two attention-based aggregators, DI-MIL demonstrates consistent efficacy, improving 67 of 72 metric-level comparisons, with the largest mean gains reaching 3.64 points under cytopathology-specific backbones. In a broader comparison against seven representative MIL baselines, DI-MIL paired with ACMIL achieves highest mean performance in 33 of 36 backbone-dataset-metric comparisons. Ablations show that direct smaller tiling inflates the extracted tile count by up to 43.3

\times

with non-monotonic performance, whereas DI-MIL incurs zero additional image-extraction overhead while achieving the strongest overall results. These results establish instance construction as an orthogonal design dimension in MIL, supporting DI-MIL as a cost-efficient solution under sparse diagnostic evidence.

Abstract

Selective Inference

Explore similar work

One Tile, Multiple Instances: Rethinking MIL for Sparse Diagnostic Evidence

From Patches to Patients: A study of the tile-to-slide performance transferability in Digital Pathology

Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models