Distribution-Conditioned Transport
Organizations: Harvard University · MIT · Beth Israel Deaconess Medical Center · Harvard Medical School ·
Abstract
Learning a transport model that maps a source distribution to a target distribution is a canonical problem in machine learning, but scientific applications increasingly require models that can generalize to source and target distributions unseen during training. We introduce distribution-conditioned transport (DCT), a framework that conditions transport maps on learned embeddings of source and target distributions, enabling generalization to unseen distribution pairs. DCT also allows semi-supervised learning for distributional forecasting problems: because it learns from arbitrary distribution pairs, it can leverage distributions observed at only one condition to improve transport prediction. DCT is agnostic to the underlying transport mechanism, supporting models ranging from flow matching to distributional divergence-based models (e.g. Wasserstein, MMD). We demonstrate the practical performance benefits of DCT on synthetic benchmarks and four applications in biology: batch effect transfer in single-cell genomics, perturbation prediction from mass cytometry data, learning clonal transcriptional dynamics in hematopoiesis, and modeling T-cell receptor sequence evolution.
Figures & tables
| Generator | -to- | Any-to-any |
|---|---|---|
| scVI | 0.6178 0.0210 | |
| Harmony | 0.2057 0.0335 | |
| FM | 0.2260 0.0292 | 0.0839 0.0089 |
| SWD | 0.2835 0.0217 | 0.0928 0.0061 |
| Energy | 0.1826 0.0180 | 0.0428 0.0036 |
| IID | OOD | |||||
| Sup. | Semi | Oracle | Sup. | Semi | Oracle | |
| scGen | .0219 .0057 | – | – | .0265 .0022 | – | – |
| CellOT | .0198 .0046 | – | – | .0304 .0033 | – | – |
| CellFlow | .0212 .0033 | – | – | .0338 .0018 | – | – |
| FM | .0101 .0023 | .0106 .0025 | .0030 .0003 | .0227 .0006 | .0208 .0018 | .0053 .0013 |
| SWD | .0064 .0022 | .0094 .0018 | .0010 .0000 | .0259 .0008 | .0197 .0022 | .0026 .0009 |
| Supervised | Semi-sup. | Oracle | CT MSE | |
|---|---|---|---|---|
| LineageOT [ 13 ] | 0.2865 0.0067 | – | – | – |
| FM | 0.3085 0.0056 | 0.2369 0.0052 | 0.1362 0.0029 | 0.0213 |
| SWD | 0.2536 0.0056 | 0.2249 0.0053 | 0.1158 0.0029 | 0.0193 |
| Energy (MSE) | 0.2524 0.0055 | 0.2289 0.0055 | 0.1180 0.0031 | 0.0187 |
| Energy (mix.) | – | 0.1944 0.0047 | – | – |
| Supervised | Semi-sup. | Oracle | |
|---|---|---|---|
| ProGen2 | 0.0599 .0211 | 0.0520 .0372 | 0.0103 .0041 |
| DFM | 0.0587 .0216 | 0.0499 .0306 | 0.0110 .0035 |
Appendix figures & tables32 assets
Supplementary material from the paper’s appendix.
Appendix
| Target-latent evaluation | IID | OOD |
|---|---|---|
| Expectation predictor (MSE) | ||
| Target-aided best-of- | ||
| Oracle STC |
| Target-latent evaluation | Energy |
|---|---|
| Expectation predictor (MSE) | |
| Mixture predictor | |
| Target-aided best-of- | |
| Oracle STC |
| Generator | Dominant fitted matrix structure | Interpretation |
|---|---|---|
| FM | on diagonal ( ), | approximately translational |
| Energy | off diagonal ( ), | coherent off-diagonal coupling |
| SWD | off diagonal ( ), | coherent off-diagonal coupling |
| Generator | Regularization | Selected matrix structure | Energy |
|---|---|---|---|
| FM | None | on diagonal | 0.019 |
| SWD | None | off diagonal | 0.022 |
| SWD | on diagonal | 0.028 | |
| SWD | on diagonal | 0.020 | |
| Energy | None | off diagonal | 0.021 |
| Energy | on diagonal | 0.042 |
| Hyperparameter | MVN | GMM |
|---|---|---|
| Dataset | ||
| Number of sample sets | 50,000 | 50,000 |
| Samples per set | 100 | 1,000 |
| Data dimension | 2 | 2 |
| Mean prior range | ||
| Covariance prior (inv-Wishart ) | 10 | 10 |
| Model | MMD | SWD | Energy | MMD | SWD | Energy | MMD | SWD | Energy | MMD | SWD | Energy |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SWD ( ) | 0.001 | 0.036 | 0.001 | 0.003 | 0.041 | 0.002 | 0.006 | 0.083 | 0.010 | 0.016 | 0.104 | 0.016 |
| SWD (Any) | 0.000 | 0.026 | 0.000 | 0.001 | 0.029 | 0.000 | 0.001 | 0.031 | 0.000 | 0.001 | 0.031 | 0.000 |
| Energy ( ) | 0.001 | 0.033 | 0.000 | 0.003 | 0.056 | 0.002 | 0.005 | 0.093 | 0.008 | 0.016 | 0.104 | 0.015 |
| Energy (Any) | 0.000 | 0.026 | -0.001 | 0.001 | 0.030 | 0.000 | 0.001 | 0.031 | 0.000 | 0.001 | 0.030 | 0.000 |
| FM ( ) | 0.009 | 0.074 | 0.006 | 0.010 | 0.079 | 0.008 | 0.013 | 0.095 | 0.011 | 0.026 | 0.133 | 0.021 |
| Model | MMD | SWD | Energy | MMD | SWD | Energy | MMD | SWD | Energy | MMD | SWD | Energy |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SWD ( ) | 0.001 | 0.108 | 0.000 | 0.002 | 0.125 | 0.002 | 0.003 | 0.149 | 0.008 | 0.006 | 0.189 | 0.018 |
| SWD (Any) | 0.001 | 0.066 | -0.001 | 0.001 | 0.093 | 0.001 | 0.002 | 0.111 | 0.003 | 0.002 | 0.121 | 0.005 |
| Energy ( ) | 0.001 | 0.121 | 0.001 | 0.001 | 0.129 | 0.001 | 0.002 | 0.151 | 0.005 | 0.005 | 0.201 | 0.016 |
| Energy (Any) | 0.000 | 0.082 | -0.002 | 0.001 | 0.098 | 0.000 | 0.002 | 0.127 | 0.002 | 0.003 | 0.145 | 0.005 |
| FM ( ) | 0.015 | 0.295 | 0.042 | 0.020 | 0.362 | 0.055 | 0.021 | 0.357 | 0.058 | 0.019 | 0.346 | 0.054 |
| Encoder | Generator | ||||
|---|---|---|---|---|---|
| -to- | FM | 1.0926 | 0.1250 | 0.0506 | 0.0560 |
| Energy | 1.1110 | 0.1276 | 0.0409 | 0.0343 | |
| SWD | 1.0310 | 0.1449 | 0.0405 | 0.0325 | |
| KME | FM | 9.6780 | 9.2970 | 6.4750 | 6.9160 |
| Energy | 7.8860 | 6.1360 | 6.1110 | 6.2600 | |
| SWD | 7.2240 | 5.6940 | 4.3590 | 3.7510 |
| Encoder | Generator | Sup. | Semi. | Ora. |
|---|---|---|---|---|
| KME | FM | 0.0074 | 1.5982 | 1.5982 |
| Energy | 0.0427 | 3.1122 | 3.1122 | |
| SWD | 0.0308 | 3.7641 | 3.7641 | |
| DeepSets | FM | 0.0010 | 0.0111 | 0.0054 |
| Energy | 0.0866 | 0.0389 | 0.0135 | |
| SWD | 0.0481 | 0.0304 | 0.0133 |
| IID | OOD | ||||||
|---|---|---|---|---|---|---|---|
| Generator | Method | MMD | SWD | Energy | MMD | SWD | Energy |
| Flow Matching | Supervised | 0.000 | 0.003 | -0.000 | 0.000 | 0.006 | -0.000 |
| Semi-sup. | 0.001 | 0.019 | 0.001 | 0.001 | 0.021 | 0.001 | |
| Oracle | 0.001 | 0.020 | 0.001 | 0.002 | 0.028 | 0.003 | |
| Energy | Supervised | 0.000 | 0.003 | -0.000 | 0.018 | 0.062 | 0.020 |
| Semi-sup. | 0.000 | 0.021 | 0.000 | 0.002 | 0.033 | 0.002 | |
| IID | OOD | ||||||
|---|---|---|---|---|---|---|---|
| Generator | Method | MMD | SWD | Energy | MMD | SWD | Energy |
| Flow Matching | Supervised | 0.000 | 0.015 | 0.000 | 0.001 | 0.022 | 0.001 |
| Semi-sup. | 0.002 | 0.032 | 0.002 | 0.010 | 0.052 | 0.011 | |
| Oracle | 0.002 | 0.032 | 0.002 | 0.004 | 0.041 | 0.005 | |
| Energy | Supervised | 0.000 | 0.008 | 0.000 | 0.031 | 0.094 | 0.037 |
| Semi-sup. | 0.001 | 0.028 | 0.001 | 0.007 | 0.061 | 0.008 | |
| Generator | Encoder | Energy | SWD | MMD-RBF |
|---|---|---|---|---|
| scVI | 0.9074 0.0698 | 0.6643 0.0566 | 0.6178 0.0210 | |
| Harmony | 0.2584 0.0402 | 0.4999 0.0355 | 0.2057 0.0335 | |
| SWD | Any-to-any | 0.1164 0.0118 | 0.3051 0.0298 | 0.0928 0.0061 |
| -to- | 0.3160 0.0286 | 0.5164 0.0226 | 0.2835 0.0217 | |
| Energy | Any-to-any | 0.0639 0.0046 | 0.4248 0.0257 | 0.0428 0.0036 |
| -to- | 0.4000 0.0453 | 0.6222 0.0333 | 0.1826 0.0180 | |
| Model | Energy |
|---|---|
| DeepSets | |
| Set Transformer | |
| KME |
| Generator | Regime | Energy | SWD | MMD-RBF |
|---|---|---|---|---|
| IID (Replicate Holdout) | ||||
| SWD | Supervised | |||
| Semi-sup. | ||||
| Oracle | ||||
| Energy | Supervised | |||
| Semi-sup. | ||||
| Generator | Energy | SWD | MMD-RBF |
|---|---|---|---|
| IID (Replicate Holdout) | |||
| SWD | |||
| Energy | |||
| FM | |||
| OOD (Patient Holdout) | |||
| SWD | |||
| Generator | Predictor | Energy | SWD | MMD-RBF |
|---|---|---|---|---|
| IID (Replicate Holdout) | ||||
| SWD | Ridge | |||
| Random Forest | ||||
| MLP | ||||
| Energy | Ridge | |||
| Random Forest | ||||
| Metric | Regime | Oracle | MSE | ||||||
| IID (Replicate Holdout) | |||||||||
| Energy | Best-of-k | ||||||||
| Mixture-of-k | |||||||||
| SWD | Best-of-k | ||||||||
| Mixture-of-k | |||||||||
| MMD-RBF | Best-of-k | ||||||||
| Model | Split | SC / baseline | Semi-sup. | Oracle |
|---|---|---|---|---|
| scGen | IID | – | – | |
| OOD | – | – | ||
| CellOT | IID | – | – | |
| OOD | – | – | ||
| CellFlow | IID | – | – | |
| OOD | – | – |
| Generator | Energy | SWD | MMD-RBF |
|---|---|---|---|
| IID (Replicate Holdout) | |||
| SWD | |||
| Energy | |||
| FM | |||
| OOD (Patient Holdout) | |||
| SWD | |||
| Generator | Energy | SWD | MMD-RBF |
|---|---|---|---|
| IID (Replicate Holdout) | |||
| SWD | |||
| Energy | |||
| FM | |||
| OOD (Patient Holdout) | |||
| SWD | |||
| Generator | Energy | SWD | MMD-RBF |
|---|---|---|---|
| IID (Replicate Holdout) | |||
| SWD | |||
| Energy | |||
| FM | |||
| OOD (Patient Holdout) | |||
| SWD | |||
| Baseline | Split | Energy | SWD | MMD-RBF |
|---|---|---|---|---|
| scGen | IID | 0.2924 0.0881 | 0.2189 0.0383 | 0.0219 0.0057 |
| OOD | 0.3278 0.0345 | 0.2272 0.0162 | 0.0265 0.0022 | |
| CellOT | IID | 0.2795 0.0760 | 0.2155 0.0366 | 0.0198 0.0046 |
| OOD | 0.3990 0.0181 | 0.2499 0.0047 | 0.0304 0.0033 | |
| CellFlow | IID | 0.2968 0.0574 | 0.2305 0.0292 | 0.0212 0.0033 |
| OOD | 0.4600 0.0141 | 0.2828 0.0102 | 0.0338 0.0018 |
| Generator | Energy | SWD | MMD-RBF |
|---|---|---|---|
| IID (Replicate Holdout) | |||
| Energy | |||
| OOD (Patient Holdout) | |||
| Energy | |||
| Generator | Energy | SWD | MMD-RBF |
|---|---|---|---|
| IID (Replicate Holdout) | |||
| Energy | |||
| OOD (Patient Holdout) | |||
| Energy | |||
| Generator | Regime | MMD-RBF | Energy | SWD |
|---|---|---|---|---|
| LineageOT | External baseline | |||
| FM | Supervised (SC) | |||
| Semi-supervised (STC, MSE) | ||||
| Semi-supervised (STC, mix.) | ||||
| Oracle STC | ||||
| SWD | Supervised (SC) |
| Method | MSE | TV |
|---|---|---|
| DCT (FM) | ||
| DCT (SWD) | ||
| DCT (Energy) |
| Target embedding | Access | Energy |
|---|---|---|
| Expectation (MSE) | Source only | |
| Mixture of 32 samples | Source only | |
| Best of 32 samples | Held-out target | |
| True target embedding | Held-out target |
| Model | Energy |
|---|---|
| SC, paired only | |
| STC, paired only | |
| STC, with orphan marginals |
| Hyperparameter | DFM | ProGen2 |
|---|---|---|
| Dataset | ||
| Sequences per set (training) | 512 | 256 |
| Maximum sequence length | 30 | 30 |
| Test fraction (multi-TP patients) | 0.3 | 0.3 |
| Encoder | ||
| ESM2 backbone | esm2_t6_8M | esm2_t6_8M |
| Generator | Regime | Energy | SWD | Sinkhorn | MMD-RBF |
|---|---|---|---|---|---|
| DFM | Source-conditioned | ||||
| Predicted-target STC | |||||
| Oracle STC | |||||
| ProGen2 | Source-conditioned | ||||
| Predicted-target STC | |||||
| Oracle STC |
OOD Generalization
Out-of-distribution generalization studies how machine-learning models perform when test data, domains, tasks, or environments differ from training.
See all 392 papers