PGVMS: A Prompt-Guided Unified Framework for Virtual Multiplex IHC Staining with Pathological Semantic Learning

Fuqiang Chen; Ranran Zhang; Wanming Hu; Deboch Eyob Abera; Yue Peng; Boyun Zheng; Yiwen Sun; Jing Cai

cs.CVFeb 26, 2026

PGVMS: A Prompt-Guided Unified Framework for Virtual Multiplex IHC Staining with Pathological Semantic Learning

Authors: Fuqiang Chen, Ranran Zhang, Wanming Hu, Deboch Eyob Abera, Yue Peng, Boyun Zheng, Yiwen Sun, Jing Cai,

Organizations: Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences and University of Chinese Academy of Sciences, Beijing, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · Department of Pathology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Guangzhou, China · Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR 999077, China ·

Abstract

Immunohistochemical (IHC) staining enables precise molecular profiling of protein expression, with over 200 clinically available antibody-based tests in modern pathology. However, comprehensive IHC analysis is frequently limited by insufficient tissue quantities in small biopsies. Therefore, virtual multiplex staining emerges as an innovative solution to digitally transform H&E images into multiple IHC representations, yet current methods still face three critical challenges: (1) inadequate semantic guidance for multi-staining, (2) inconsistent distribution of immunochemistry staining, and (3) spatial misalignment across different stain modalities. To overcome these limitations, we present a prompt-guided framework for virtual multiplex IHC staining using only uniplex training data (PGVMS). Our framework introduces three key innovations corresponding to each challenge: First, an adaptive prompt guidance mechanism employing a pathological visual language model dynamically adjusts staining prompts to resolve semantic guidance limitations (Challenge 1). Second, our protein-aware learning strategy (PALS) maintains precise protein expression patterns by direct quantification and constraint of protein distributions (Challenge 2). Third, the prototype-consistent learning strategy (PCLS) establishes cross-image semantic interaction to correct spatial misalignments (Challenge 3).

Figures & tables

Fig. 1: The comparison of traditional IHC staining and virtual staining workflows. From (a) to (c), they are conventional IHC staining, biomarker-specific virtual staining and our PGVMS.

Fig. 2: Three challenges of virtual multi-staining. (a) Inadequate semantic guidance for multi-staining. (b) Inconsistent distribution of immunochemistry staining. (c) Spatial misalignment of pathological semantics.

Fig. 3: The framework of PGVMS. (a) Overall architecture of PGVMS, including the pathological semantics–style guided (PSSG) generator and schematic overviews of the protein-aware learning strategy (PALS) and the prototype-consistent learning strategy (PCLS). (b) and (c) show detailed illustrations of PALS and PCLS in the bottom row, highlighting their key components and operations.

Type	Method	HER2						ER
		Quality (reference)		Pathology		Perception		Quality (reference)		Pathology		Perception
		PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
One-to-one	CycleGAN [ 6 ]	12.2042	0.1588	+4.5848	0.1995	63.8952	0.2864	13.2516	0.2187	-1.6250	0.2916	54.1309	0.2730
	Pix2Pix [ 4 ]	12.9840	0.1676	-2.5782	0.2986	84.3667	0.2699	14.7293	0.2200	-5.8452	0.6307	90.6261	0.3050
	CUT [ 5 ]	13.8998	0.1680	-2.9478	0.7164	53.2911	0.2641	14.2172	0.2201	-4.5073	0.5841	45.8074	0.2630
	ASP [ 10 ]	14.1841	0.2004	-5.7422	0.0659	50.8364	0.2517	14.2088	0.2178	-4.5875	0.4802	54.7745	0.2705

TABLE I: COMPARISON OF VIRTUAL STAINING PERFORMANCE ON MIST DATASET. THE BEST SCORES ARE IN BOLD. MODELS ARE GROUPED INTO ONE-TO-ONE AND ONE-TO-MANY CATEGORIES.

Type	Method	HER2						ER
		Quality (reference)		Pathology		Perception		Quality (reference)		Pathology		Perception
		PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{8}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
One-to-one	CycleGAN [ 6 ]	10.1229	0.0930	+1.9836	0.3071	143.8269	0.3385	17.3148	0.4406	+0.8744	0.3878	50.9366	0.2545
	Pix2Pix [ 4 ]	11.3910	0.1228	-0.9739	0.6214	117.3533	0.3027	20.7537	0.4931	-1.0956	0.2171	129.9073	0.3050
	CUT [ 5 ]	10.9131	0.1227	-1.2030	-0.0613	63.3108	0.2657	19.4466	0.4745	-1.0011	0.0705	41.8358	0.2618
	ASP [ 10 ]	11.2034	0.1245	-1.1742	-0.0988	63.9972	0.2658	19.5277	0.4834	-0.6886	0.1810	37.9753	0.2644

TABLE II: COMPARISON OF VIRTUAL STAINING PERFORMANCE ON IHC4BC DATASET. THE BEST SCORES ARE IN BOLD. MODELS ARE GROUPED INTO ONE-TO-ONE AND ONE-TO-MANY CATEGORIES.

Fig. 4: Virtual IHC-stained image results of different methods on the MIST dataset. The first column is the H&E-stained images. Columns 2 to 14 are the images virtually stained by different methods. The last column is the ground truth. The figure also demonstrates virtual IHC results alongside positive cell visualization from DeepLIIF.

Fig. 5: Consistent with the protein progression tendency. The horizontal axis (abscissa) represents the sample index, while the vertical axis (ordinate) denotes the accumulated optical density (OD) value. The figure illustrates the accumulated OD curves of various virtual staining methods for IHC positive regions, comparing the virtual results with the reference IHC images on the MIST and IHC4BC datasets. The results of the ControlNet model are excluded because they deviate significantly from the ground truth.

Fig. 6: Virtual IHC-stained image results of different methods on the IHC4BC dataset. The first column is the H&E-stained images. Columns 2 to 14 are the images virtually stained by different methods. The last column is the ground truth. The figure also demonstrates virtual IHC results alongside positive cell visualization from DeepLIIF.

Method	MIST				IHC4BC
Method	HER2	ER	PR	Ki67	HER2	ER	PR	Ki67
CycleGAN	0.453	0.570	0.534	0.767	0.495	0.600	0.479	0.771
Pix2Pix	0.490	0.671	0.671	0.847	0.422	0.595	0.666	0.613
CUT	0.457	0.610	0.664	0.807	0.258	0.573	0.492	0.565
ASP	0.378	0.623	0.637	0.759	0.256	0.547	0.559	0.799
PyramidP2P	0.587	0.695	0.705	0.842	0.505	0.497	0.640	0.686

TABLE III: Classification accuracy (%) of virtual staining results on MIST and IHC4BC datasets. HER2: 4-class (0, 1+, 2+, 3+); ER, PR, Ki67: binary (negative/positive). Models are grouped into one-to-one and one-to-many categories.

Fig. 7: Visual comparison illustrating the severe cell spatial misalignment issue in TDKStain results, where virtual staining causes substantial cell displacement relative to the input H&E image. In contrast, our PGVMS method preserves accurate cell spatial localization.

Type	Method	MIST					IHC4BC
		HER2	ER	PR	Ki67	Mean	HER2	ER	PR	Ki67	Mean
One-to-one	CycleGAN	102.1581	52.9350	125.7399	48.0381	82.2178	218.4938	44.1311	81.4923	49.0361	98.2884
	Pix2Pix	95.9983	189.5522	182.9658	216.5276	171.2610	153.2364	105.1925	95.7887	128.3961	120.6534
	CUT	62.6960	39.4071	32.4882	35.4223	42.5034	80.8654	32.3410	39.4285	247.6296	100.0661
	ASP	39.9833	52.4421	38.3425	43.7905	43.6396	76.8791	25.4769	31.9873	20.9709	38.8285
	Pyramid	151.2503	153.7980	145.9719	172.3845	155.8512	237.9637	172.1971	162.0819	106.7118	169.7386

TABLE IV: Feature-level FID between virtual IHC and ground-truth IHC computed using the pathology foundation model CONCH. Lower values indicate better alignment. The best and second-best results are highlighted in bold and underlined , respectively.

Fig. 8: Pathologist evaluation of virtual IHC-stained images on the MIST datasets across three criteria: cellular localization, staining heterogeneity, and staining intensity.

Fig. 9: Pathologist evaluation of virtual IHC-stained images on the IHC4BC datasets across three criteria: cellular localization, staining heterogeneity, and staining intensity.

Fig. 10: Virtual multiplex IHC generation from H&E-stained input images. For each H&E input, the first row (gray background) displays PGVMS-generated staining for HER2, ER, PR, and Ki-67, and the second row (yellow background) shows the corresponding ground-truth IHC images.

Fig. 11: Visualization of the effects of PALS, PCLS, and PSSG on our PGVMS. The color-coded rectangles indicate regions corresponding to different experimental configurations: black for the input H&E image, yellow for the baseline output, green for baseline + PALS, blue for baseline + PALS + PCLS, orange for baseline + PALS + PCLS + PSSG, and red for the ground truth.

Baseline	PALS	PCLS	PSSG	Quality		Pathology		Perception
Baseline	PALS	PCLS	PSSG	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
$\checkmark$				14.2183	0.2046	-3.1237	0.6229	45.6044	0.2598
$\checkmark$	$\checkmark$			14.6252	0.2205	-2.3128	0.8436	49.7711	0.2577
$\checkmark$	$\checkmark$	$\checkmark$		14.4773	0.2195	-2.0626	0.8361	39.2908	0.2508
$\checkmark$			$\checkmark$	13.7738	0.1905	-5.0098	0.3756	62.5494	0.2709
$\checkmark$	$\checkmark$		$\checkmark$	13.0808	0.1887	-2.9507	0.3998	67.4985	0.2813

TABLE V: ABLATION STUDY OF STAIN TRANSFER PERFORMANCE ON MIST DATASET. THE BEST SCORES ARE HIGHLIGHTED IN BOLD.

Guidance	Quality		Pathology		Perception
Guidance	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
One Hot	14.9029	0.2310	-2.1429	0.8729	38.5514	0.2379
CLIP	14.2745	0.1895	-1.4198	0.8678	39.2263	0.2454
CONCH	14.3421	0.2032	-0.3223	0.8649	38.0654	0.2344

TABLE VI: ABLATION STUDY OF GUIDANCE MECHANISM ON MIST DATASET. THE BEST SCORES ARE HIGHLIGHTED IN BOLD.

Method	Quality		Pathology		Perception
Method	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
w/o bias	14.0289	0.2099	-2.2577	0.7400	58.2488	0.2482
w/ bias	14.3421	0.2062	-0.3223	0.8648	38.0654	0.2344

TABLE VII: ABLATION STUDY OF IMAGE-GUIDED BIAS MECHANISM ON MIST DATASET. THE BEST SCORES ARE HIGHLIGHTED IN BOLD.

Pooling Strategy	Quality		Pathology		Perception
Pooling Strategy	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
Only Maxpooling	14.5682	0.2184	-1.7611	0.7890	53.5903	0.2522
Only Avgpooling	14.6032	0.2086	-2.4378	0.7607	83.3022	0.2646
Max & Avg Pooling	14.3421	0.2032	-0.3223	0.8648	38.0654	0.2344

TABLE VIII: ABLATION STUDY OF POOLING STRATEGY ON MIST DATASET. THE BEST SCORES ARE HIGHLIGHTED IN BOLD.

Guidance	Quality		Pathology		Perception
Guidance	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
Addition	13.8117	0.1904	-1.5385	0.7942	53.1891	0.2451
Cross Attention	13.8461	0.2028	-2.4021	0.8041	55.5236	0.2438
PGSN	14.3421	0.2032	-0.3223	0.8649	38.0654	0.2344

TABLE IX: ABLATION STUDY OF FUSION MECHANISM ON MIST DATASET. THE BEST SCORES ARE HIGHLIGHTED IN BOLD.

Normalization Strategy	Quality		Pathology		Perception
Normalization Strategy	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
Only IN	14.4614	0.2184	-3.2292	0.7939	69.9662	0.2600
Only LN	15.6425	0.2887	-3.9779	0.7672	153.7774	0.3414
Both IN&LN	14.3421	0.2032	-0.3223	0.8649	38.0654	0.2344

TABLE X: ABLATION STUDY OF NORMALIZATION STRATEGY ON MIST DATASET. THE BEST SCORES ARE HIGHLIGHTED IN BOLD.

FOD $\alpha$	Quality		Pathology		Perception
FOD $\alpha$	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
1.0	14.6189	0.2115	-2.2031	0.8331	43.8208	0.2395
1.4	14.6242	0.2078	-1.7750	0.8598	38.3692	0.2372
1.8	14.3421	0.2032	-0.3223	0.8649	38.0654	0.2344
2.2	14.2708	0.2197	-1.8733	0.8507	43.6559	0.2449

TABLE XI: ABLATION STUDY OF FOD

\alpha

ON MIST DATASET. THE BEST SCORES ARE HIGHLIGHTED IN BOLD.

Guidance	Quality		Pathology		Perception
Guidance	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
PLIP	14.2897	0.2111	+0.6578	0.8476	41.4264	0.2511
MUSK	14.7346	0.2089	-2.5387	0.8408	40.6992	0.2499
CONCH	14.3421	0.2032	-0.3223	0.8649	38.0654	0.2344

TABLE XII: ABLATION STUDY OF PATHOLOGICAL GUIDANCE MECHANISM ON MIST DATASET. THE BEST SCORES ARE HIGHLIGHTED IN BOLD.

Method	PSNR				SSIM
Method	No Blur	3×3	5×5	7×7	No Blur	3×3	5×5	7×7
CycleGAN	13.3060	13.4611	13.5518	13.6656	0.2000	0.2264	0.2401	0.2561
Pix2Pix	14.3445	14.4810	14.5597	14.6546	0.2143	0.2407	0.2558	0.2726
CUT	14.2183	14.3632	14.4553	14.5692	0.2047	0.2318	0.2466	0.2633
ASP	14.2746	14.4244	14.5189	14.6330	0.2144	0.2406	0.2553	0.2719
Pyramid	14.3320	14.4816	14.5731	14.6877	0.2137	0.2402	0.2554	0.2715

TABLE XIII: COMPARISON OF PSNR AND SSIM VALUES WITH DIFFERENT GAUSSIAN BLUR KERNELS

Stage	Input Size	GPU (MB)	Time
Training	$512\times 512$	16,540	125 h / 80 epochs
Inference	$1024\times 1024$	10,284	0.0208 s / tile

TABLE XIV: Computational efficiency of PGVMS during training and inference.

Length	Prompt	Quality		Pathology		Perception
Length	Prompt	PSNR $\uparrow$	SSIM $\uparrow$	IOD ${}_{\times 10^{7}}$	Pearson-R $\uparrow$	FID $\downarrow$	DISTS $\downarrow$
Short	1	14.4628	0.2211	-0.9492	0.8284	44.6465	0.2420
Short	2	14.2855	0.2092	-0.5466	0.8433	38.4802	0.2385
[0.8pt/2pt] Medium	3	14.2589	0.2333	-1.7759	0.7235	53.0694	0.2541
Medium	4	14.4086	0.2160	-1.4156	0.8326	40.1634	0.2410
[0.8pt/2pt] Long	5	14.4193	0.2245	-1.2309	0.7994	46.1909	0.2441

TABLE XV: ROBUSTNESS EVALUATION OF PGVMS UNDER DIFFERENT PROMPT EXPRESSIONS. THE BEST SCORES ARE HIGHLIGHTED IN BOLD.

Fig. 12: Virtual IHC-stained image results on the clinical dataset, showing four stains (HER2, ER, PR, Ki-67) and positive cell visualization obtained using DeepLIIF. The first column shows the input H&E images. Images with a gray background represent PGVMS-generated multiplex stains along with their corresponding positive cell visualization, while images with an yellow background show the ground truth multiplex stains and the associated positive cell visualization.

Fig. 13: Virtual IHC-stained image results on the clinical dataset. PGVMS preserves accurate cellular localization and realistic staining intensity across different biomarker.

Fig. 14: Virtual Ki-67-stained image results on the clinical colon cancer dataset.

Fig. 15: Virtual HepPar1 IHC-stained image results on the clinical HCC dataset.

Fig. 16: Virtual CK-stained image results on the clinical NPC dataset.

Fig. 17: Virtual EBER-stained image results on the clinical NPC dataset.

Fig. 18: Virtual PD-L1 IHC-stained image results on the clinical HNSCC dataset.

Fig. 19: Representative cross-organ virtual IHC staining results on liver, head-and-neck, colorectal, and nasopharyngeal samples. PGVMS preserves accurate cellular localization and realistic staining intensity across different biomarkers.

Virtual Staining

Virtual staining uses AI to predict histochemical or immunohistochemical appearances from unstained or differently stained tissue images.

See all 18 papers

UNIStainNet: Foundation-Model-Guided Virtual Staining of H&E to IHC
Mar 13, 2026Jillur Rahman Saurav, Thuong Le Hoai Pham, Pritam Mukherjee +3Virtual Staining Computational Pathology
Department of Computer Science, University of Texas at Arlington, Arlington, Texas, USA · Department of Radiology, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA · Department of Pathology, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA
Bridging the Modality Bottleneck in Pathology MIL through Virtual Molecular Staining
May 12, 2026Yucheng Xing, Pei Liu, Jingying Ma +6Computational Pathology Cross-Modal Representation Learning
National University of Singapore, Singapore · Hunan University, China · Peking Union Medical College Hospital (PUMCH), China +1
Segmentation-before-Staining Improves Structural Fidelity in Virtual IHC-to-Multiplex IF Translation
Mar 17, 2026Junhyeok Lee, Han Jang, Heeseong Eum +2Virtual Staining Medical Image Synthesis
Interdisciplinary Program in Cancer Biology, Seoul National University College of Medicine, Seoul, Republic of Korea · Interdisciplinary Program in Bioengineering, Seoul National University, Seoul, Republic of Korea · Department of Biomedical Sciences, Seoul National University, Seoul, Republic of Korea +3

Mar 13, 2026J/K move · Enter open · S save

UNIStainNet: Foundation-Model-Guided Virtual Staining of H&E to IHC

Jillur Rahman Saurav, Thuong Le Hoai Pham, Pritam Mukherjee, Paul Yi+2

Department of Computer Science, University of Texas at Arlington, Arlington, Texas, USA · Department of Radiology, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA · Department of Pathology, St. Jude Children’s Research Hospital, Memphis, Tennessee, USA

Virtual Staining Computational Pathology Image-to-Image Translation

Open paper AskPDF arXiv

Virtual immunohistochemistry (IHC) staining from hematoxylin and eosin (H&E) images can accelerate diagnostics by providing preliminary molecular insight directly from routine sections, reducing the need for repeat sectioning when tissue is limited. Existing methods improve realism through contrastive objectives, prototype matching, or domain alignment, yet the generator itself receives no direct guidance from pathology foundation models. We present UNIStainNet, a SPADE-UNet conditioned on dense spatial tokens from a frozen pathology foundation model (UNI), providing tissue-level semantic guidance for stain translation. A misalignment-aware loss suite preserves stain quantification accuracy, and learned stain embeddings enable a single model to serve multiple IHC markers simultaneously. On MIST, UNIStainNet achieves state-of-the-art distributional metrics on all four stains (HER2, Ki67, ER, PR) from a single unified model, where prior methods typically train separate per-stain models. On BCI, it also achieves the best distributional metrics. A tissue-type stratified failure analysis reveals that remaining errors are systematic, concentrating in non-tumor tissue. Code is available at https://github.com/facevoid/UNIStainNet.

Abstract

Virtual Staining

Explore similar work

UNIStainNet: Foundation-Model-Guided Virtual Staining of H&E to IHC

Bridging the Modality Bottleneck in Pathology MIL through Virtual Molecular Staining

Segmentation-before-Staining Improves Structural Fidelity in Virtual IHC-to-Multiplex IF Translation