Segmentation-before-Staining Improves Structural Fidelity in Virtual IHC-to-Multiplex IF Translation
Organizations: Interdisciplinary Program in Cancer Biology, Seoul National University College of Medicine, Seoul, Republic of Korea · Interdisciplinary Program in Bioengineering, Seoul National University, Seoul, Republic of Korea · Department of Biomedical Sciences, Seoul National University, Seoul, Republic of Korea · Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea ·
Abstract
Multiplex immunofluorescence (mIF) enables simultaneous single-cell quantification of multiple biomarkers within intact tissue architecture, yet its high reagent cost, multi-round staining protocols, and need for specialized imaging platforms limit routine clinical adoption. Virtual staining can synthesize mIF channels from widely available brightfield immunohistochemistry (IHC), but current translators optimize pixel-level fidelity without explicitly constraining nuclear morphology. In pathology, this gap is clinically consequential: subtle distortions in nuclei count, shape, or spatial arrangement propagate directly to quantification endpoints such as the Ki67 proliferation index, where errors of a few percent can shift treatment-relevant risk categories. This work introduces a supervision-free, architecture-agnostic conditioning strategy that injects a continuous cell probability map from a pretrained nuclei segmentation foundation model as an explicit input prior, together with a variance-preserving regularization term that matches local intensity statistics to maintain cell-level heterogeneity in synthesized fluorescence channels. The soft prior retains gradient-level boundary information lost by binary thresholding, providing a richer conditioning signal without task-specific tuning. Controlled experiments across Pix2Pix with U-Net and ResNet generators, deterministic regression U-Net, and conditional diffusion on two independent datasets demonstrate consistent improvements in nuclei count fidelity and perceptual quality, as the sole modifications. Code will be made publicly available upon acceptance.
Figures & tables
| DeepLIIF [ 5 ] (3-ch mIF) | HNSCC [ 6 ] (DAPI 1-ch) | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | Config | SSIM | LPIPS | Ki67 † | SSIM | LPIPS | pMAE ‡ | |
| Pix2Pix [ 11 ] | Baseline | .719 | .318 | .018 | .834 | .134 | .019 | |
| (UNet) | +Prior | .732 | .296 | .018 | .855 | .113 | .017 | |
| Pix2Pix [ 11 ] | Baseline | .759 | .293 | .019 | .880 | .121 | .016 | |
| (ResNet) | +Prior | .763 | .291 | .021 | .880 | .131 | .015 | |
| Regression [ 21 ] | Baseline | .724 | .315 | .018 | .812 | .197 | .020 | |
| Model | Prior | SSIM | LPIPS | pMAE | |||
|---|---|---|---|---|---|---|---|
| Pix2Pix [ 11 ] | None | .834 | .134 | .019 | |||
| (UNet) | Binary | ✓ | .846 | .175 | .017 | ||
| Soft | ✓ | .855 | .113 | .017 | |||
| Soft+Var | ✓ | ✓ | .855 | .113 | .017 | ||
| Pix2Pix [ 11 ] | None | .880 | .121 | .016 | |||
| (ResNet) | Binary | ✓ | .879 | .125 | .015 |
Virtual Staining
Virtual staining uses AI to predict histochemical or immunohistochemical appearances from unstained or differently stained tissue images.
See all 18 papers