ESS-Flow: Training-free guidance as Bayesian inference in source space
Organizations: Department of Computer Science Linköping University, Sweden · Department of Information Technology Uppsala University, Sweden
Abstract
Guiding pretrained flow-based generative models for conditional generation or to produce samples with desired target properties enables solving diverse tasks without retraining on paired data. We present ESS-Flow, a gradient-free method that leverages the Gaussian prior of the source distribution in flow-based models to perform Bayesian inference directly in the source space using elliptical slice sampling. ESS-Flow only requires forward passes through the generative model and observation process, making it applicable even when gradients are unavailable, such as with non-differentiable simulation-based observations or quantization in the generation process. We demonstrate its effectiveness on scientific applications including generating materials with target properties, protein backbone structures with improved designability, and synthesizable, drug-like molecules.
Figures & tables
| Property | Target | Atoms | Potential function | Scale |
|---|---|---|---|---|
| Space group | P6 3 /mmc | 8 | – | |
| Bulk modulus | 300 GPa | 4 | 10 GPa | |
| Shear modulus | 200 GPa | 4 | 10 GPa | |
| Band gap | 10 eV | 12 | 0.1 eV | |
| Energy above hull | – | 8 | 0.01 eV |
| Method | Bulk modulus | Shear modulus | Band gap | Energy above hull |
|---|---|---|---|---|
| D-Flow | 205.88 60.79 | 165.93 27.72 | 9.24 1.18 | 1.92 1.38 |
| PnP-Flow | 49.93 32.33 | 75.48 33.45 | 5.63 1.03 | 0.02 0.06 |
| DAPS | 39.14 26.47 | 84.33 37.10 | 3.90 1.67 | 0.06 0.05 |
| ESS-Flow | 8.99 6.69 | 10.53 9.21 | 1.85 1.66 | 0.19 0.07 |
| Method | Bulk modulus | Shear modulus | Band gap | Energy above hull | Space group |
|---|---|---|---|---|---|
| Unconditional | 0.0 | 0.0 | 0.0 | 1.4 | 1.3 |
| D-Flow | 0.0 | 0.0 | 0.0 | 1.8 | – |
| PnP-Flow | 3.8 | 0.9 | 0.1 | 34.5 | – |
| DAPS | 9.4 | 0.5 | 0.0 | 17.6 | – |
| ESS-Flow | 13.7 | 4.7 | 16.0 | 37.6 | 25.5 |
| Method | scRMSD (Å) | Designability (%) | TM-score |
|---|---|---|---|
| Unconditional | 6.74 | 9.6 | 0.361 |
| ESS-Flow ( ) | 2.49 | 53.1 | 0.368 |
| ESS-Flow ( ) | 1.92 | 74.0 | 0.367 |
| ESS-Flow ( ) | 1.53 | 86.4 | 0.380 |
| Method | Conn. | Div. | Uniq. | QED | SA | Qual. |
|---|---|---|---|---|---|---|
| Unconditional (DDPM) | 70.0 | 0.843 | 100.0 | 0.413 0.183 | 6.954 0.934 | 0.0 |
| Unconditional (DDIM) | 44.3 | 0.848 | 100.0 | 0.351 0.177 | 6.922 0.945 | 0.1 |
| ESS-Flow ( ) | 99.8 | 0.849 | 91.6 | 0.435 0.189 | 6.689 1.116 | 0.3 |
| ESS-Flow ( ) | 99.1 | 0.848 | 80.4 | 0.520 0.190 | 6.377 1.336 | 0.4 |
| ESS-Flow ( ) | 97.9 | 0.826 | 39.7 | 0.635 0.159 | 4.760 2.252 | 0.9 |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| SWD | ||||||
|---|---|---|---|---|---|---|
| 1 | 0.037 0.010 | 0.044 0.019 | 0.064 0.025 | 2.8 1.3 | 2.8 1.1 | 2.7 0.5 |
| 2 | 0.034 0.014 | 0.034 0.009 | 0.037 0.016 | 3.2 0.7 | 2.8 1.2 | 3.2 1.6 |
| 4 | 0.028 0.008 | 0.031 0.014 | 0.038 0.024 | 3.5 1.2 | 3.0 1.4 | 4.1 2.4 |
| Method / Parameter | Bulk modulus | Shear modulus | Band gap | Energy above hull |
| D-Flow | ||||
| ODE integration steps | 50 | 50 | 50 | 50 |
| Optimizer | SGD | SGD | SGD | SGD |
| Optimization steps | 100 | 100 | 100 | 100 |
| Learning rate | ||||
| Learning rate |
| Task | Method | Valid | S. | U.N. | T. | S.U.N. | S.U.N.T. |
|---|---|---|---|---|---|---|---|
| Space group | Unconditional | 73.0 | 26.5 | 72.4 | 2.3 | 26.0 | 1.3 |
| ESS-Flow | 87.1 | 50.0 | 54.6 | 81.9 | 26.7 | 25.5 | |
| Bulk modulus | Unconditional | 77.9 | 32.4 | 73.2 | 0.4 | 28.3 | 0.0 |
| D-Flow | 75.2 | 31.0 | 70.2 | 0.4 | 26.3 | 0.0 | |
| PnP-Flow | 94.8 | 46.2 | 71.4 | 17.8 | 29.0 | 3.8 | |
| DAPS | 95.4 | 57.6 | 80.8 | 19.8 | 47.0 | 9.4 |
| Method | Gradients | ||
|---|---|---|---|
| D-Flow | and | ||
| PnP-Flow | only | ||
| DAPS | only |
| Task | Gradients | |||
|---|---|---|---|---|
| Bulk modulus | 6.7 | None | ||
| Shear modulus | 7.1 | None | ||
| Band gap | 17.4 | None | ||
| Energy above hull | 12.0 | None | ||
| Space group | 4.8 | None |
| Method | Bulk modulus | Shear modulus | Band gap | Energy above hull | Space group |
|---|---|---|---|---|---|
| D-Flow | 6.91 | 6.91 | 62.22 | 27.90 | – |
| PnP-Flow | 0.86 | 0.86 | 2.28 | 1.57 | – |
| DAPS | 0.86 | 0.86 | 2.28 | 1.57 | – |
| ESS-Flow | 0.29 | 0.29 | 0.60 | 0.45 | 0.18 |
| Method | Bulk modulus | Shear modulus | Band gap | Energy above hull |
|---|---|---|---|---|
| ESS-Flow | 8.99 6.69 | 10.53 9.21 | 1.85 1.66 | 0.19 0.07 |
| Reweighted | 9.36 6.70 | 10.00 9.35 | 0.85 0.47 | 0.22 0.08 |
| Method | Bulk modulus | Shear modulus | Band gap | Energy above hull |
|---|---|---|---|---|
| ESS-Flow | 13.7 | 4.7 | 16.0 | 37.6 |
| Reweighted | 9.5 | 3.3 | 0.9 | 12.3 |
Inference-Time Guidance
Inference-time guidance steers pretrained generative models, language models, and policies toward desired outputs without requiring full retraining.
See all 110 papers