Segmentation-before-Staining Improves Structural Fidelity in Virtual IHC-to-Multiplex IF Translation

Junhyeok Lee; Han Jang; Heeseong Eum; Joon Jang; Kyu Sung Choi

cs.CVMar 17, 2026

Segmentation-before-Staining Improves Structural Fidelity in Virtual IHC-to-Multiplex IF Translation

Authors: Junhyeok Lee, Han Jang, Heeseong Eum, Joon Jang, Kyu Sung Choi

Organizations: Interdisciplinary Program in Cancer Biology, Seoul National University College of Medicine, Seoul, Republic of Korea · Interdisciplinary Program in Bioengineering, Seoul National University, Seoul, Republic of Korea · Department of Biomedical Sciences, Seoul National University, Seoul, Republic of Korea · Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea ·

Abstract

Multiplex immunofluorescence (mIF) enables simultaneous single-cell quantification of multiple biomarkers within intact tissue architecture, yet its high reagent cost, multi-round staining protocols, and need for specialized imaging platforms limit routine clinical adoption. Virtual staining can synthesize mIF channels from widely available brightfield immunohistochemistry (IHC), but current translators optimize pixel-level fidelity without explicitly constraining nuclear morphology. In pathology, this gap is clinically consequential: subtle distortions in nuclei count, shape, or spatial arrangement propagate directly to quantification endpoints such as the Ki67 proliferation index, where errors of a few percent can shift treatment-relevant risk categories. This work introduces a supervision-free, architecture-agnostic conditioning strategy that injects a continuous cell probability map from a pretrained nuclei segmentation foundation model as an explicit input prior, together with a variance-preserving regularization term that matches local intensity statistics to maintain cell-level heterogeneity in synthesized fluorescence channels. The soft prior retains gradient-level boundary information lost by binary thresholding, providing a richer conditioning signal without task-specific tuning. Controlled experiments across Pix2Pix with U-Net and ResNet generators, deterministic regression U-Net, and conditional diffusion on two independent datasets demonstrate consistent improvements in nuclei count fidelity and perceptual quality, as the sole modifications. Code will be made publicly available upon acceptance.

Figures & tables

Figure 1: Overview of the proposed soft prior conditioning framework. (a) Cellpose [ 22 ] extracts a continuous probability map $P_{\text{soft}}$ , concatenated as a fourth channel. (b) The conditioned input is passed to any translation architecture. (c) Multi-channel outputs; $\mathcal{L}_{\text{var}}$ is added to the base objective to constrain local intensity statistics.

		DeepLIIF [ 5 ] (3-ch mIF)			HNSCC [ 6 ] (DAPI 1-ch)
Model	Config	SSIM $\uparrow$	LPIPS $\downarrow$	Ki67 † $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$	pMAE ‡ $\downarrow$
Pix2Pix [ 11 ]	Baseline	.719	.318	.018	.834	.134	.019
(UNet)	+Prior	.732	.296	.018	.855	.113	.017
Pix2Pix [ 11 ]	Baseline	.759	.293	.019	.880	.121	.016
(ResNet)	+Prior	.763	.291	.021	.880	.131	.015
Regression [ 21 ]	Baseline	.724	.315	.018	.812	.197	.020

Table 1: Translation quality with off-the-shelf nuclei prior. +Prior adds the soft probability map and variance-preserving loss. DeepLIIF [ 5 ] translates IHC to 3-channel mIF. HNSCC [ 6 ] translates mIHC to DAPI. † Ki67 positivity error. ‡ Per-pixel MAE on DAPI channel. Best per architecture in bold .

Model	Prior	$M_{\text{cell}}$	$P_{\text{soft}}$	$\mathcal{L}_{\text{var}}$	SSIM $\uparrow$	LPIPS $\downarrow$	pMAE $\downarrow$
Pix2Pix [ 11 ]	None				.834	.134	.019
(UNet)	Binary	✓			.846	.175	.017
	Soft		✓		.855	.113	.017
	Soft+Var		✓	✓	.855	.113	.017
Pix2Pix [ 11 ]	None				.880	.121	.016
(ResNet)	Binary	✓			.879	.125	.015

Table 2: Ablation on prior representation and loss (HNSCC [ 6 ] , DAPI,

n_{\text{test}}{=}71

M_{\text{cell}}

: binary mask (1 ch).

P_{\text{soft}}

: soft probability map (1 ch).

\mathcal{L}_{\text{var}}

: variance-preserving loss (

\lambda{=}50

k{=}15

). Best per architecture in bold .

Figure 2: Qualitative comparison on DeepLIIF test cases across three target channels (DAPI, Lap2, Ki67) from bladder and lung tissue.

Virtual Staining

Virtual staining uses AI to predict histochemical or immunohistochemical appearances from unstained or differently stained tissue images.

See all 18 papers

Mar 13, 2026

UNIStainNet: Foundation-Model-Guided Virtual Staining of H&E to IHC

Virtual immunohistochemistry (IHC) staining from hematoxylin and eosin (H&E) images can accelerate diagnostics by providing preliminary molecular insight directly from routine sections, reducing the need for repeat sectioning when tissue is limited. Existing methods improve realism through contrastive objectives, prototype matching, or domain alignment, yet the generator itself receives no direct guidance from pathology foundation models. We present UNIStainNet, a SPADE-UNet conditioned on dense spatial tokens from a frozen pathology foundation model (UNI), providing tissue-level semantic guidance for stain translation. A misalignment-aware loss suite preserves stain quantification accuracy, and learned stain embeddings enable a single model to serve multiple IHC markers simultaneously. On MIST, UNIStainNet achieves state-of-the-art distributional metrics on all four stains (HER2, Ki67, ER, PR) from a single unified model, where prior methods typically train separate per-stain models. On BCI, it also achieves the best distributional metrics. A tissue-type stratified failure analysis reveals that remaining errors are systematic, concentrating in non-tumor tissue. Code is available at https://github.com/facevoid/UNIStainNet.

Jillur Rahman Saurav, Thuong Le Hoai Pham, Pritam Mukherjee +3

Department of Computer Science, University of Texas at Arlington, Arlington, Texas, USA · Department of Radiology, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA · Department of Pathology, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA

Virtual Staining Computational Pathology Image-to-Image Translation

Ask PDF

Sep 1, 2026

Semi-Supervised Virtual Staining via Morphology Preservation and Histopathological Realism Constraints

Virtual staining aims to computationally generate target-stained histopathological images while reducing the cost and time associated with conventional staining procedures. However, existing methods rely predominantly on strictly paired and accurately registered training data, which are difficult and expensive to obtain in routine practice. To reduce this dependence, we propose a stable semi-supervised virtual staining framework that jointly exploits both limited paired data and abundant unpaired source images. Directly incorporating unpaired images is challenging because their generated results lack corresponding targets for supervision, potentially leading to unrealistic staining, morphological degradation, or even training collapse. To obtain reliable supervision from these images, Hessian-derived morphology preservation extracts structural cues from each source image and constrains the generated output to retain tissue morphology. Histopathological realism constraints further guide the output toward plausible target-stain characteristics, preventing the source-derived structural supervision from degenerating into contour enhancement or simple color transformation. Together, the two components suppress structural and appearance drift, stabilize semi-supervised stain translation, and promote the preservation of diagnostically relevant information. Extensive experiments on H&E-to-IHC translation for Ki67 and HER2, as well as FFPE-to-H&E translation, demonstrate consistent improvements in image quality, morphology preservation, robustness, and downstream diagnostic performance. Code will be available.

Baoshun Wang, Weiping Lin, Linwu Wang +3

Department of Computer Science, School of Informatics, Xiamen University, Xiamen 361005, China · EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France · Service de M´edecine Nucl´eaire, Centre Hospitalier Universitaire de Nˆımes, Universit´e de Montpellier, Nˆımes, France

Computational Pathology Image-to-Image Translation Virtual Staining Unpaired Image-to-Image Translation Medical Image Synthesis

Ask PDF

Apr 26, 2026

VitaminP: cross-modal learning enables whole-cell segmentation from routine histology

Accurate whole-cell and nuclear segmentation is essential for precision pathology and spatial omics, yet routine hematoxylin and eosin (H&E) staining provides limited cytoplasmic contrast, restricting analyses to nuclei. Multiplex immunofluorescence (mIF) facilitates precise whole-cell delineation but remains constrained by cost and accessibility. We introduce VitaminP, a cross-modal learning framework enabling whole cell segmentation from H&E images. By learning from paired H&E-mIF data, VitaminP transfers molecular boundary information from mIF to overcome cytoplasmic contrast in H&E, establishing cross-modal supervision as a general strategy for recovering missing biological structure. We train VitaminP on 14 public datasets covering 34 cancer types and over 7 million instances, integrating publicly available labels with extensive annotations generated in this study, forming one of the largest resources for segmentation. VitaminP outperforms four state-of-the-art methods and generalizes to unseen datasets, including an in-house dataset spanning 24 rare cancer types. We further developed VitaminPScope, an open-source platform providing an interface for scalable inference and enabling broad adoption.

Yasin Shokrollahi, Karina B. Pinao Gonzales, Elizve N. Barrientos Toro +5

Department of Translational Molecular Pathology, Division of Pathology and Laboratory Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA. · Institute for Data Science in Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA. · The University of Texas MD Anderson Cancer Center, Houston, TX, USA.

Cross-Modal Learning Computational Pathology Cross-Modality Medical Image Segmentation Image Segmentation Histopathology Image Segmentation Multimodal Learning

Ask PDF

Abstract

Virtual Staining

Explore similar work

UNIStainNet: Foundation-Model-Guided Virtual Staining of H&E to IHC

Semi-Supervised Virtual Staining via Morphology Preservation and Histopathological Realism Constraints

VitaminP: cross-modal learning enables whole-cell segmentation from routine histology