The DynaCell Evaluation Suite for Dynamic 3D Virtual Staining of Live Cells
Organizations: Biohub, San Francisco, 94158 · UCSF, San Francisco, 94158
Abstract
Understanding how cells respond to perturbations over time is central to cell biology and drug discovery. Live-cell imaging captures the dynamics of perturbation-induced phenotypes in individual cells. While fluorescence imaging can visualize specific molecular phenotypes, labeling and repeated illumination often affect cell health. Virtual staining offers an alternative by predicting these fluorescence images from images acquired without fluorescent labels. Although many virtual staining models have been developed, their performance on volumetric time-lapse data remains poorly characterized. We address this gap with an evaluation suite for 3D time-lapse virtual staining of live cells. The suite combines paired 3D time-lapse images acquired with and without fluorescent labels, a set of baseline models, and three metric tiers measuring pixel-level fidelity, organelle segmentation performance, and single-cell phenotype similarity. Using this evaluation suite, we study metric and model behavior across 3 cell types, 2 microscopes, 4 organelles, and 3 perturbation states. We find that regression baselines better predict the spatial localization of target organelles, whereas the generative baseline better captures the distribution of phenotypes across cells. These differences expose trade-offs between structural and phenotypic fidelity that are obscured by single-metric evaluation and suggest how virtual staining models can be developed for diverse downstream tasks such as localization, segmentation, tracking, and phenotypic profiling. We release the curated datasets, baseline checkpoints, and evaluation code as part of DynaCell, a broader framework for image-based profiling of cell-state dynamics.
Figures & tables
| iPSC test | A549 test | |||||||||||||||||
| VSCyto3D | Pix2Pix3D | CELL-Diff3D | VSCyto3D | Pix2Pix3D | CELL-Diff3D | |||||||||||||
| Metric | iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | A549 | iPSC | Joint | A549 | iPSC | Joint | A549 | iPSC | Joint |
| Nucleus | ||||||||||||||||||
| si-SSIM ( ) | ||||||||||||||||||
| SPCC ( ) | ||||||||||||||||||
| Dice ( ) | ||||||||||||||||||
Appendix figures & tables43 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | 3D | Live | Perturbation | Organelles | …as time-lapse | …under perturbation | Cell types | Protocol | Tiers | Paired examples |
|---|---|---|---|---|---|---|---|---|---|---|
| Light My Cells (2026) | no | yes | no | 4 | 0 | 0 | n/r | yes | px, phe § | 4,602 (2D) |
| Trizna et al. (2023) | no | yes | drug | 0 | 0 | 0 | 1 | yes | seg ¶ | 504 (2D) |
| VIRVS (2025) | no | mixed | 5 viruses | 0 | 0 | 0 | 3 | yes | px, seg | 3,820 (2D) |
| JUMP Cell Painting (2023) | no | no | chem + gene | 8 | 0 | 0 | 1–2 | no | — | ~1.9 M (2D) |
| Ounkomol et al. (2018) | yes | yes | no | 12 | 0 | 0 | 4 | no | px | ~420 (3D) |
| Guo et al. QLIPP (2020) | yes | no | no | 3 | 0 | 0 | tissue | no | px | 320 (3D) |
| Feature extractor | |||||
|---|---|---|---|---|---|
| Statistic | GLCM+ | DINOv3 | DynaCLR | CELL-DINO | MorphEm |
| FNet3D mean rank | |||||
| VSCyto3D mean rank | |||||
| Pix2Pix3D mean rank | |||||
| CELL-Diff3D mean rank | |||||
| CELL-Diff3D wins | |||||
| Rand-init | FNet3D | |||||
| Metric | iPSC | A549 | iPSC iPSC | A549 iPSC | A549 A549 | iPSC A549 |
| Nucleus | ||||||
| PSNR ( ) | {\color[rgb]{0.5469,0.2891,0.6133}21.1\pm 1.3} | |||||
| si-PSNR ( ) | {\color[rgb]{0.5469,0.2891,0.6133}28.9\pm 4.2} | |||||
| SSIM ( ) | {\color[rgb]{0.75,0.3008,0.2617}0.70\pm 0.07} | |||||
| si-SSIM ( ) | ||||||
| Rand-init | FNet3D | |||||
| Metric | iPSC | A549 | iPSC iPSC | A549 iPSC | A549 A549 | iPSC A549 |
| Nucleus | ||||||
| Dice ( ) | ||||||
| IoU ( ) | ||||||
| Precision (px) ( ) | ||||||
| Recall (px) ( ) | ||||||
| Rand-init | FNet3D | |||||
| Metric | iPSC | A549 | iPSC iPSC | A549 iPSC | A549 A549 | iPSC A549 |
| Nucleus | ||||||
| GLCM+ cos ( ) | {\color[rgb]{0.5469,0.2891,0.6133}0.14} | |||||
| GLCM+ FID ( ) | {\color[rgb]{0.5469,0.2891,0.6133}8.29} | |||||
| GLCM+ KID ( ) | {\color[rgb]{0.75,0.3008,0.2617}2.60\pm 0.03} | |||||
| DINOv3 Median cos ( ) | ||||||
| Model | Training | Data | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Params | Pretrained | Duration | Batch | LR | LR schedule | Loss | Tgt. norm a | |||
| CELL-Diff3D | 93.4M | No | 20 ep | 4 | Cosine | 8 | Flow matching | min-max c | ||
| Pix2Pix3D | 77.8M+disc. | No | 40 ep | 4 | (G,D) | Warmup cosine | 8 | Non-sat. GAN+ d | min-max c | |
| UNetViT3D | 77.8M | No | 20 ep | 4 | Cosine | 8 | MSE | med/IQR | ||
| VSCyto3D | 32.1M | Yes b | 200 ep | 32 | Cosine | 15 | MAE+SSIM | med/IQR | ||
| UNeXt2 | 32.1M | No | 200 ep | 32 | Cosine | 15 | MAE+SSIM | med/IQR | ||
| Augmentation | FNet3D | UNeXt2 | VSCyto3D | UNetViT3D | Pix2Pix3D | CELL-Diff3D |
|---|---|---|---|---|---|---|
| Weighted random crop (CPU) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Random flip, Y/X axes ( ) | ✓ | |||||
| Random 3D affine ( ) | ✓ | ✓ | ✓ | ✓ | ✓ | |
| with safe-crop constraint | ✓ | ✓ | ✓ | |||
| Gamma contrast adjustment ( ) | ✓ | ✓ | ✓ | ✓ | ✓ | |
| Intensity scaling ( ) | ✓ | ✓ | ✓ | ✓ | ✓ |
| Organelle | Domain | Spectral PCC | Dice | CELL-DINO KID |
|---|---|---|---|---|
| Membrane | iPSC | |||
| A549 | ||||
| Nucleus | iPSC | |||
| A549 | ||||
| ER | iPSC | |||
| A549 |
| Organelle (train test) | Model | si-SSIM | Spectral PCC | Dice | DINOv3 KID | CELL-DINO KID | GLCM+ cos |
|---|---|---|---|---|---|---|---|
| Nucleus (iPSC iPSC) | UNetViT3D | 0.54 0.16 | 0.89 0.07 | 0.86 0.08 | 2.01 0.04 | 23.34 0.17 | -0.00 0.43 |
| Pix2Pix3D | 0.58 0.17 | 0.90 0.06 | 0.89 0.08 | 1.82 0.06 | 23.99 0.24 | -0.01 0.45 | |
| CELL-Diff3D (mean) | 0.54 0.16 | 0.87 0.07 | 0.85 0.09 | 1.77 0.03 | 27.71 0.21 | 0.00 0.43 | |
| CELL-Diff3D (sampled) | 0.47 0.17 | 0.81 0.09 | 0.84 0.08 | 0.08 0.02 | 1.72 0.05 | 0.26 0.41 | |
| Membrane (iPSC iPSC) | UNetViT3D | 0.57 0.12 | 0.88 0.14 | 0.66 0.10 | 2.57 0.05 | 34.52 0.24 | 0.22 0.46 |
| Pix2Pix3D | 0.60 0.12 | 0.88 0.15 | 0.69 0.11 | 2.62 0.05 | 31.30 0.24 | 0.26 0.45 |
| Organelle | Inference | Spectral PCC | Dice | CELL-DINO KID |
|---|---|---|---|---|
| Cell Membrane | Iterative | |||
| Sliding-Window | ||||
| Nucleus | Iterative | |||
| Sliding-Window | ||||
| ER (SEC61B) | Iterative | |||
| Sliding-Window |
| (A) iPSC test set | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| VSCyto3D | Pix2Pix3D | CELL-Diff3D | |||||||
| Metric | iPSC iPSC | A549 iPSC | Joint iPSC | iPSC iPSC | A549 iPSC | Joint iPSC | iPSC iPSC | A549 iPSC | Joint iPSC |
| Nucleus | |||||||||
| Spectral PCC ( ) | 0.899 0.06 | 0.557 0.15 | 0.904 0.06 | 0.904 0.06 | 0.592 0.14 | 0.892 0.06 | 0.812 0.09 | 0.377 0.15 | 0.805 0.09 |
| Dice ( ) | 0.895 0.07 | 0.631 0.12 | 0.891 0.07 | 0.888 0.08 | 0.635 0.11 | 0.888 0.07 | 0.841 0.08 | 0.394 0.09 | 0.834 0.08 |
| CELL-DINO KID ( ) | 24.93 0.22 | 30.62 0.19 | 24.91 0.24 | 23.99 0.24 | 29.14 0.23 | 16.40 0.14 | 1.720 0.05 | 12.28 0.11 | 1.652 0.05 |
| Organelle | Model | Drop to HEK (cell type) | Drop to iPSC (cell type + scope) |
|---|---|---|---|
| Cell membrane | FNet3D | +0.302 | +0.479 |
| Cell membrane | VSCyto3D | +0.175 | +0.627 |
| Mitochondria | FNet3D | +0.143 | +0.589 |
| Mitochondria | VSCyto3D | +0.099 | +0.360 |
| Organelle | Test set | FNet3D iPSC | FNet3D A549 | FNet3D Joint | VSCyto3D iPSC | VSCyto3D A549 | VSCyto3D Joint | Raw phase no model |
|---|---|---|---|---|---|---|---|---|
| Cell membrane KRAS on HEK, CAAX elsewhere | iPSC (confocal) | 0.894 | 0.399 | 0.876 | 0.889 | 0.256 | 0.895 | -0.001 |
| A549 (Mantis) | 0.594 | 0.878 | 0.882 | 0.567 | 0.883 | 0.889 | 0.015 | |
| HEK (Mantis) | 0.642 | 0.576 | 0.692 | 0.709 | 0.708 | 0.765 | 0.044 | |
| Mitochondria TOMM70A on HEK, TOMM20 elsewhere | iPSC (confocal) | 0.910 | 0.331 | 0.903 | 0.893 | 0.526 | 0.895 | 0.139 |
| A549 (Mantis) | 0.223 | 0.921 | 0.922 | 0.793 | 0.887 | 0.898 | 0.176 | |
| HEK (Mantis) | 0.090 | 0.777 | 0.782 | 0.807 | 0.787 | 0.818 | 0.110 |
| iPSC test | A549 test | |||||||||
| Metric | FNet2D | UNeXt2-2D | VSCyto2D | Pix2Pix2D | CELL-Diff2D | FNet2D | UNeXt2-2D | VSCyto2D | Pix2Pix2D | CELL-Diff2D |
| Nucleus | ||||||||||
| si-SSIM ( ) | ||||||||||
| SPCC ( ) | ||||||||||
| Dice ( ) | ||||||||||
| CD KID ( ) | ||||||||||
| 2D vs 3D, median | A549 conditions | Seed contexts, median | ||||||
|---|---|---|---|---|---|---|---|---|
| Metric | All | In-distribution | OOD | Joint | Same best | 2D vs 2D | 2D seed vs seed | 2D vs 3D |
| si-SSIM | ||||||||
| SPCC | ||||||||
| Dice | ||||||||
| CD KID | ||||||||
| GLCM+ cos | ||||||||
| Cell type | Metric | FNet3D | UNeXt2 | VSCyto3D | UNetViT3D | Pix2Pix3D | CELL-Diff3D | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | iPSC | A549 | Joint | iPSC | A549 | Joint | ||
| iPSC | |||||||||||||||||
| A549 Mock | |||||||||||||||||
| Cell type | Metric | FNet3D | UNeXt2 | VSCyto3D | UNetViT3D | Pix2Pix3D | CELL-Diff3D | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | iPSC | A549 | Joint | iPSC | A549 | Joint | ||
| iPSC | |||||||||||||||||
| A549 Mock | |||||||||||||||||
| Cell type | Metric | FNet3D | UNeXt2 | VSCyto3D | UNetViT3D | Pix2Pix3D | CELL-Diff3D | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | ||
| iPSC | |||||||||||||||||||
| — | — | ||||||||||||||||||
| A549 Mock | |||||||||||||||||||
| Cell type | Metric | FNet3D | UNeXt2 | VSCyto3D | UNetViT3D | Pix2Pix3D | CELL-Diff3D | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | iPSC | A549 | Joint | ||
| iPSC | |||||||||||||||||||
| A549 Mock | |||||||||||||||||||
| Organelle | CV range (%) | T10 - T1 range | Leader T1 T10 |
|---|---|---|---|
| Nucleus | 1.0–1.9 | -0.039 to -0.019 | Pix2Pix3D Pix2Pix3D |
| Membrane | 1.0–2.5 | -0.004 to +0.006 | Pix2Pix3D Pix2Pix3D |
| ER | 0.9–2.7 | -0.029 to +0.017 | VSCyto3D Pix2Pix3D |
| Mitochondria | 1.2–1.9 | -0.044 to -0.033 | FNet3D FNet3D |
| train/test frame gap | elapsed | AUROC, DENV | AUROC, ZIKV | pooled | probe fits averaged |
|---|---|---|---|---|---|
| same frame | 0 h | 0.639 0.009 | 0.649 0.009 | 0.644 0.007 | 240 |
| 1 | 2 h | 0.628 0.006 | 0.638 0.006 | 0.633 0.004 | 432 |
| 2 | 4 h | 0.609 0.006 | 0.629 0.006 | 0.619 0.004 | 384 |
| 3 | 6 h | 0.596 0.006 | 0.607 0.007 | 0.601 0.005 | 336 |
| 4 | 8 h | 0.579 0.006 | 0.598 0.007 | 0.588 0.005 | 288 |
| 5 | 10 h | 0.560 0.006 | 0.592 0.007 | 0.576 0.004 | 240 |
| Organelle | Virus | DINOv3 | CELL-DINO | MorphEm |
|---|---|---|---|---|
| Membrane | DENV | 0.49 0.54 | 0.55 0.76 | 0.53 0.72 |
| Membrane | ZIKV | 0.47 0.77 | 0.55 0.92 | 0.47 0.87 |
| Nucleus | DENV | 0.53 0.71 | 0.54 0.66 | 0.45 0.58 |
| Nucleus | ZIKV | 0.51 0.67 | 0.64 0.79 | 0.56 0.67 |
| ER | DENV | 0.56 0.70 | 0.66 0.83 | 0.64 0.79 |
| ER | ZIKV | 0.51 0.72 | 0.64 0.82 | 0.51 0.76 |
| Pixel | Seg. | Phenotype (FID) | |||||
| Statistic | Spectral PCC | PCC | SSIM | si-SSIM | Dice | CELL- DINO | DINOv3 |
| Frame effect SD | 0.012 | 0.020 | 0.025 | 0.019 | 0.018 | 84.0 | 36.8 |
| Model frame SD | 0.005 | 0.007 | 0.017 | 0.005 | 0.006 | 41.2 | 17.1 |
| Split-half r | +0.27 | +0.44 | +0.42 | +0.49 | +0.33 | +0.45 | +0.19 |
| Vs. top-two gap | 0.80 | 0.57 | 0.31 | 0.72 | 0.70 | 0.03 | 0.08 |
| 1-frame window winner | 113/360 (31%) | 85/360 (24%) | 62/360 (17%) | 75/360 (21%) | 108/360 (30%) | 16/360 (4%) | 47/360 (13%) |
| Model | cos, DENV | cos, ZIKV | amp, DENV 1 | amp, ZIKV 1 | CELL-Diff3D cos win, pooled |
|---|---|---|---|---|---|
| FNet3D | 0.270 | 0.298 | 0.876 | 0.845 | 74% (4e-29) |
| UNeXt2 | 0.314 | 0.329 | 0.857 | 0.889 | 67% (5e-14) |
| VSCyto3D | 0.295 | 0.324 | 0.858 | 0.878 | 69% (4e-17) |
| UNetViT3D | 0.274 | 0.245 | 0.890 | 0.939 | 66% (9e-13) |
| Pix2Pix3D | 0.333 | 0.373 | 0.901 | 0.923 | 61% (7e-06) |
| CELL-Diff3D | 0.401 | 0.420 | 0.996 | 1.040 | – |
| retention | FNet3D | VSCyto3D | CELL-Diff3D |
|---|---|---|---|
| mock | 0.868 | 0.760 | 0.849 |
| infected | 0.739 | 0.592 | 0.616 |
| drop (mock infected) | +0.129 | +0.167 | +0.232 |
Virtual Staining
Virtual staining uses AI to predict histochemical or immunohistochemical appearances from unstained or differently stained tissue images.
See all 22 papers