Organizations: School of Mathematics and Statistics, Nanjing University of Science and Technology, Nanjing, 210094, China · School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, China · Department of Mathematics, Nanjing University, Nanjing, 210093, China · Department of Radiology, Affiliated Hospital of Nanjing University of Chinese Medicine, Nanjing, 210018, China · Department of Pancreatic and Bariatric Surgery, Nanjing Drum Tower Hospital, the Affiliated Hospital of Nanjing University Medical School, Nanjing, 210008, China · School of Artificial Intelligence and Information Technology, Nanjing University of Chinese Medicine, Nanjing, 210023, China · School of Computing and Mathematical Sciences, University of Leicester, Leicester, LE1 7RH, United Kingdom
Joint nuclei instance segmentation and classification requires precise boundaries and neighborhood information that helps distinguish morphologically similar cell types. We present TME-ContextNet, a multi-task framework that learns class-conditioned spatial context from existing nuclear annotations and organizes decoding into geometry-related and semantic task groups. Class-conditioned spatial supervision extends nuclear labels into a dense target using Euclidean distance transforms, requiring no additional tissue-region annotation. We derive its nearest-region interpretation and characterize its stability to small annotation perturbations. A local--global context module combines dilated convolution and self-attention, while a grouped task decoder reduces redundant computation. The spatial head supplies training supervision and is omitted during inference. Experiments on PanMix, CoNSeP, and PanNuke yield Dice scores of 80.23%, 65.32%, and 79.13%, and mean multi-class panoptic quality (mPQ+) scores of 58.67%, 35.50%, and 43.74%, respectively. PanNuke results are averaged across three test folds. Target and decoder controls support the combination of distance-based supervision and grouped decoding while identifying their dependence on task configuration. The framework uses 36.40 million parameters, compared with 54.74 million for HoVer-Net. These findings support a compact, context-aware formulation for joint nuclei analysis in computational pathology.
Figures & tables
Fig. 1: Overview of TME-ContextNet. (a) The image encoder and shared decoder feed geometry-related and semantic task groups. Nuclear foreground (NP), horizontal–vertical displacement (HV), and nuclear type (TP) predictions are used for typed instance assembly. Class-conditioned spatial supervision (CSS) provides an annotation-derived auxiliary training target, and its head is omitted at inference. Encoder dilation and decoder self-attention form the local–global context module (LGCM), while squeeze-and-excitation (SE) supplies separate channel recalibration. (b) Dilated residual block with a shortcut connection. (c) Spatial self-attention with a learnable residual gate. GTD denotes the grouped task decoder
Fig. 2: Construction of the EDT-based CSS target. The nuclear type map is separated into class-specific supports. Each support produces a Euclidean distance transform and an influence score with σ=5 pixels and ε=10−6 . Background pixels receive the class with the largest score, while annotated nuclei retain their original labels. Black outlines identify nuclear regions on the final target. Distance and score maps use continuous color scales, and categorical maps use the corresponding nuclear class colors
Fig. 3: Nuclear class composition from the PanMix and PanNuke dataset statistics and the CoNSeP training partition. Bars show annotated nucleus counts from the dataset statistics. Colors follow the class order used for each dataset in the qualitative figures. Axis scales differ among panels
PanMix
Method
Dice (%)
IoU (%)
mPQ+ (%)
PSPNet
75.62
73.04
—
KNet
79.17
76.01
—
Mask2Former
77.77
75.67
—
Segformer
76.18
73.49
—
HoVer-Net
69.80
67.22
47.85
Table 1: Comparison on PanMix, CoNSeP, and PanNuke. Values are percentages; bold indicates the highest value in each dataset and metric.
Fig. 4: Qualitative examples of joint nuclei instance segmentation and classification on PanMix, CoNSeP, and PanNuke. Columns show the H&E input, ground truth, HoVer-Net, Zhang et al. (2024), HoVer-NeXt, CellViT, and TME-ContextNet. Identical image regions and class colors are used across methods within each dataset. Colored contours indicate individual nuclei and their assigned types, with dataset-specific class legends below each row
Class
Method
Dice (%)
IoU (%)
Pixel P×R (%)
Neoplastic
HoVer-Net
67.67 ± 4.26
61.41 ± 4.33
60.30 ± 4.31
Zhang et al. (2024)
63.75 ± 5.42
57.39 ± 5.28
56.23 ± 5.31
HoVer-NeXt
55.04 ± 0.43
48.46 ± 0.84
47.12 ± 0.90
CellViT
75.49 ± 0.51
69.43 ± 0.12
68.33 ± 0.09
TME-ContextNet
78.18 ± 1.37
72.22 ± 1.06
71.18 ± 1.03
Inflammatory
HoVer-Net
58.31 ± 2.14
53.13 ± 2.29
52.02 ± 2.36
Table 2: Per-class comparison on PanNuke. Bold indicates the highest mean within each class and metric.
Configuration
GTD
LGCM
CSS
Dice (%)
IoU (%)
mPQ+ (%)
HoVer-Net
—
—
—
69.80
67.22
47.85
HoVer-Net + CSS
—
—
✓
73.83
71.36
52.48
HoVer-Net + CSS + attention
—
Partial
✓
71.89
69.34
50.70
w/o self-attention
✓
Partial
✓
77.45
75.10
56.19
w/o LGCM and SE
✓
—
✓
75.69
73.28
54.03
w/o SE
✓
✓
✓
70.50
68.18
32.54
Table 3: Component configurations on PanMix.
Target
Seeds
Dice (%)
IoU (%)
No CSS
3
51.91 ± 3.30
43.60 ± 3.15
EDT
3
53.60 ± 1.30
45.50 ± 0.99
Uniform Gaussian
1
50.45
42.49
Class-weighted Gaussian
1
48.86
40.81
Constant class weights
1
51.26
42.73
Permuted class weights
1
51.32
41.47
Table 4: CSS target constructions in a dedicated HoVer-Net-based auxiliary-supervision model on CoNSeP. EDT is the distance-transform construction adopted for CSS.
Fig. 5: Examples of spatial supervision targets and predictions. From left to right, each row shows the input image, enlarged region of interest, nuclear annotations, annotation-derived CSS target, and predicted CSS classes. The spatial target extends class assignments into non-nuclear regions while retaining the annotated nuclear labels
Configuration
Params (M)
Dice (%)
IoU (%)
PanMix: controlled sharing depth
HoVer-Net
54.74
69.80
67.22
Integration Stage I
40.60
74.33
71.86
Integration Stage II
36.80
76.00
73.61
Integration Stage III
35.31
66.28
64.03
CoNSeP: decoder / no CSS
Table 5: Controlled comparisons of decoder sharing depth, decoder organization, and LGCM.
Fig. 6: Effect of auxiliary-loss weight ω on PanMix with the EDT-based CSS target. Colors denote weights from 0.0 to 1.0 in increments of 0.1. Each weight uses one training run with the remaining settings held fixed. The boxplots summarize per-image test scores, rather than variation across random seeds. Scores are shown as fractions.
Method
Params
GFLOPs
Network
Image
P95
Memory
(M)
(ms)
(ms)
(ms)
(GiB)
RTX 4090: main-model session
HoVer-Net
54.74
398.32
24.78
444.17
463.02
0.50
TME-ContextNet
36.40
267.77
19.37
357.24
410.72
0.43
TME-ContextNet w/o CSS
36.40
267.77
19.21
351.31
381.46
0.43
RTX 4090: session 1
Table 6: Computational cost of TME-ContextNet and decoder variants on RTX 4090. Measurement sessions are presented separately.
Fig. 7: Examples of residual prediction errors on CoNSeP. Each row shows the input, ground truth, and TME-ContextNet output. Outlined regions highlight (a) incomplete separation or disagreement in nuclear extent, (b) additional predicted nuclear regions absent from the annotation, and (c) disagreement between predicted and annotated nuclear types. Nuclear contour colors follow the CoNSeP legend in Fig. 4
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 8: Decoder organization and shared decoding depth. (a) Independent decoders process NP, HV, TP, and CSS separately. (b) A plain shared decoder uses one representation through stages u3, u2, and u1 before separate task heads. (c) GTD shares u3 and u2, then separates u1 refinement into geometry-related (NP/HV) and semantic (TP/CSS) groups. (d) In the controlled PanMix sharing-depth comparison in Table 5 , Stage I shares u3, Stage II shares u3 and u2, and Stage III shares u3, u2, and u1. CSS, SE, and self-attention are disabled in all three stages, and all other settings are fixed. The NP, HV, and TP task heads remain separate
Fig. 9: PanNuke examples with enlarged details. Columns show the H&E input, ground truth, HoVer-Net, Zhang et al. (2024), HoVer-NeXt, and TME-ContextNet. Each overview is followed by an enlarged view of the outlined region. All methods use the same image coordinates and class colors. The examples illustrate nuclear delineation and type assignments for the five PanNuke categories
Fig. 10: Additional qualitative class-map comparisons on PanMix, PanNuke, and CoNSeP. Two examples per dataset are shown for the H&E input, ground truth, PSPNet, K-Net, Mask2Former, SegFormer, HoVer-Net, and TME-ContextNet. These examples compare local class assignments and region boundaries. All methods use the same image coordinates within each example
Fig. 11: PanNuke example of local nuclear type assignments. The outlined region identifies the same group of nuclei across the input, ground truth, and six method outputs. The example includes disagreement between neoplastic and dead-cell labels among methods. It illustrates local type ambiguity and complements the class-wise quantitative results
Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Macau University of Science and Technology, Macau, China · Chinese Medicine Guangdong Laboratory/Hengqin Laboratory, Hengqin, China