UniSemAlign: Text-Prototype Alignment with a Foundation Encoder for Semi-Supervised Histopathology Segmentation

Le-Van Thai; Tien Dat Nguyen; Hoai Nhan Pham; Lan Anh Dinh Thi; Duy-Dong Nguyen; Ngoc Lam Quang Bui

cs.CVApr 10, 2026

UniSemAlign: Text-Prototype Alignment with a Foundation Encoder for Semi-Supervised Histopathology Segmentation

Authors: Le-Van Thai, Tien Dat Nguyen, Hoai Nhan Pham, Lan Anh Dinh Thi, Duy-Dong Nguyen, Ngoc Lam Quang Bui

Organizations: AI VIETNAM Lab, Vietnam · Hanoi University of Science and Technology, Vietnam

Abstract

Semi-supervised semantic segmentation in computational pathology remains challenging due to scarce pixel-level annotations and unreliable pseudo-label supervision. We propose UniSemAlign, a dual-modal semantic alignment framework that enhances visual segmentation by injecting explicit class-level structure into pixel-wise learning. Built upon a pathology-pretrained Transformer encoder, UniSemAlign introduces complementary prototype-level and text-level alignment branches in a shared embedding space, providing structured guidance that reduces class ambiguity and stabilizes pseudo-label refinement. The aligned representations are fused with visual predictions to generate more reliable supervision for unlabeled histopathology images. The framework is trained end-to-end with supervised segmentation, cross-view consistency, and cross-modal alignment objectives. Extensive experiments on the GlaS and CRAG datasets demonstrate that UniSemAlign substantially outperforms recent semi-supervised baselines under limited supervision, achieving Dice improvements of up to 2.6% on GlaS and 8.6% on CRAG with only 10% labeled data, and strong improvements at 20% supervision. Code is available at: https://github.com/thailevann/UniSemAlign

Figures & tables

Figure 1: Overview of UniSemAlign. An input image is encoded by UNI ViT-B/16 and decoded by DeepLabV3+ to produce visual logits. In parallel, pixel embeddings are projected into a shared semantic space and aligned with learnable class prototypes and prompt-guided text embeddings generated by a frozen CONCH text encoder with learnable context tokens. Cosine similarity between pixel embeddings and these class representations produces alignment logits, which are fused with the decoder logits to generate pseudo-labels for semi-supervised training.

Labeled Ratio	Method	GlaS		CRAG
Labeled Ratio	Method	mDice (%) $\uparrow$	mJaccard (%) $\uparrow$	mDice (%) $\uparrow$	mJaccard (%) $\uparrow$
100%	Fully-Supervised	89.97	81.77	89.13	80.44
10%	UAMT (MICCAI 2019) [ 53 ]	78.57	64.70	77.85	63.74
	FixMatch (NeurIPS 2020) [ 38 ]	66.01	51.39	72.42	64.31
	CPS (CVPR 2021) [ 29 ]	61.70	49.34	47.10	40.77
	CT (MIDL 2022) [ 26 ]	81.75	70.44	77.33	65.56

Table 1: Quantitative results on GlaS-2017 and CRAG-2019 datasets under two labeled ratios. We highlight the best performance for each metric in bold , and the second-best in underlined .

Figure 2: Qualitative results for different semi-supervised methods under the 10% labeling setting on GlaS-2017 and CRAG-2019. Pink boxes highlight notable differences between competing methods and our approach.

Encoder	mDice (%)	mJaccard (%)
ResNet101	86.37	76.01
UNI	88.15	78.82

Table 2: Ablation study on encoder architectures of DeepLabV3+ on GlaS (10% labeled). The best results are shown in bold .

# Context Tokens	mDice (%)	mJaccard (%)
0	87.52	77.80
2	87.85	77.86
3	88.07	78.68
4	88.15	78.82
5	87.79	78.25

Table 3: Impact of the number of learnable context tokens on GlaS (10% labeled). The best results are shown in bold .

Figure 3: Qualitative comparison of the dual semantic alignment branches on GlaS (10% labeled). From left to right: (a) input image, (b) UniSemAlign (ours), (c) Ground truth, (d) without text branch, (e) without prototype branch. Pink boxes highlight notable differences. The prototype branch primarily improves structural completeness and boundary coherence, while the text branch suppresses ambiguous/background activations.

Text	Prototype	mDice (%)	mJaccard (%)
$\times$	$\times$	87.66	78.04
$\times$	$\checkmark$	88.00	78.57
$\checkmark$	$\times$	88.06	78.66
$\checkmark$	$\checkmark$	88.15	78.82

Table 4: Ablation study of semantic alignment branches on GlaS (10% labeled). The best results are shown in bold .

Alignment loss	mDice (%)	mJaccard (%)
No alignment	87.56	77.87
Cosine similarity	87.69	78.08
KL divergence	87.43	77.67
MSE	88.15	78.82

Table 5: Comparison of alignment loss formulations on GlaS (10% labeled). The best results are shown in bold .

Histopathology Image Segmentation

Histopathology image segmentation identifies tissue regions, cells, and their boundaries in microscopic pathology images.

See all 53 papers

CoDiR: Confidence-Guided Diffusion Refinement for Semi-Supervised Histopathology Segmentation
Aug 12, 2026Hoai Nhan Pham, Dang-Nguyen Bui, Le-Van Thai +7Histopathology Image Segmentation Semi-Supervised Medical Image Segmentation
AI VIETNAM Lab, Vietnam · Portland Community College, USA · University of Science, VNU-HCM, Ho Chi Minh City, Vietnam +4
ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation
Aug 12, 2026Duy-Dong Nguyen, Le-Van Thai, Hoai Nhan Pham +3Histopathology Image Segmentation Weakly Supervised Semantic Segmentation
AI VIETNAM Lab, Vietnam · Department of Mechanical System Engineering, Jeonbuk National University, Republic of Korea · Washington University School of Medicine, Saint Louis, MO, USA +1
Semantic Segmentation for Histopathology using Learned Regularization based on Global Proportions
Apr 27, 2026Yangping Li, Thomas Pinetz, Michael Hölzel +2Computational Pathology Image Segmentation
Institute of Pathology, University Hospital Bonn, Sigmund-Freud-Str. 25, 53127 Bonn, Germany · Institute for Applied Mathematics, University of Bonn, Endenicher Allee 60, 53115 Bonn, Germany · Institute of Experimental Oncology, University Hospital Bonn, Sigmund-Freud-Str. 25, 53127 Bonn, Germany

Aug 12, 2026J/K move · Enter open · S save

CoDiR: Confidence-Guided Diffusion Refinement for Semi-Supervised Histopathology Segmentation

Hoai Nhan Pham, Dang-Nguyen Bui, Le-Van Thai, Thanh-Hiep Vo+6

AI VIETNAM Lab, Vietnam · Portland Community College, USA · University of Science, VNU-HCM, Ho Chi Minh City, Vietnam+4

Histopathology Image Segmentation Semi-Supervised Medical Image Segmentation Pseudo-Labeling

Open paper AskPDF arXiv

Semi-supervised histopathology segmentation is challenging due to scarce annotations and unreliable pseudo-labels in ambiguous gland regions. To address this problem, we propose Confidence-Guided Diffusion Refinement (CoDiR), a semi-supervised framework that combines a Mean Teacher segmentation model with diffusion-based pseudo-label refinement. Given an unlabeled image, the teacher first produces a soft prediction, and only low-confidence regions are refined by a conditional diffusion model trained to capture plausible mask structures from labeled data. The refined mask is then fused with reliable teacher predictions and used to train the student with confidence weighting and consistency regularization. On the GlaS and CRAG datasets CoDiR reaches 88.09% and 89.83% mDice with 10% labeled data, and 89.19% and 90.29% mDice with 20%, matching or exceeding the strongest published method on seven of the eight benchmark metrics. Ablations attribute the largest single contribution to the refinement module, which adds +6.36% mDice over the Mean Teacher baseline. The implementation code is publicly available at: https://github.com/vongla345/codir

Abstract

Histopathology Image Segmentation

Explore similar work

CoDiR: Confidence-Guided Diffusion Refinement for Semi-Supervised Histopathology Segmentation

ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation

Semantic Segmentation for Histopathology using Learned Regularization based on Global Proportions