Flow Perturbation++: Multi-Step Unbiased Jacobian Estimation for High-Dimensional Boltzmann Sampling
Organizations: State Key Laboratory of Information Photonics and Optical Communications, Beijing University of Posts and Telecommunications, Beijing 100876, China · School of Physical Science and Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
Abstract
The scalability of continuous normalizing flows (CNFs) for unbiased Boltzmann sampling remains limited in high-dimensional systems due to the cost of Jacobian-determinant evaluation, which requires backpropagation passes through the flow layers. Existing stochastic Jacobian estimators such as the Hutchinson trace estimator reduce computation but introduce bias, while the recently proposed Flow Perturbation method is unbiased yet suffers from high variance. We present Flow Perturbation++, a variance-reduced extension of Flow Perturbation that discretizes the probability-flow ODE and performs unbiased stepwise Jacobian estimation at each integration step. This multi-step construction retains the unbiasedness of Flow Perturbation while achieves substantially lower estimator variance. Integrated into a Sequential Monte Carlo framework, Flow Perturbation++ achieves significantly improved equilibrium sampling on a 1000D Gaussian Mixture Model and the all-atom Chignolin protein compared with Hutchinson-based and single-step Flow Perturbation baselines.
Figures & tables
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| System | Resampling | schedule | schedule | ||||
|---|---|---|---|---|---|---|---|
| 1000D GMM | 1,000 | 2,000 | 10 | .95 | systematic | ||
| Chignolin | 2,000 | 600 | 2 | .95 | KL reshuffling |
| Method | Cost (# ODE passes) | VJP Time | JVP Time |
|---|---|---|---|
| Hutchinson G2 / R2 | 3 | 30.4 h | 48.9 h |
| Hutchinson G10 / R10 | 11 | 100 h | 176 h |
| FP (single-step) | 2 | 23.8 h | 30.8 h |
| FP++ (multi-step) | 2 | 23.8 h | 30.8 h |
| Method | GMM 10D | GMM 100D | GMM 1000D |
|---|---|---|---|
| BF Jacobian | 0.9265 | 1.5931 | 11.0544 |
| FP1 | 1.6886 | 10.2365 | 37.6025 |
| FP2 | 1.4847 | 8.0238 | 35.3451 |
| FP5 | 1.1487 | 6.5414 | 31.2574 |
| FP10 | 0.9597 | 5.3718 | 29.4457 |
| FP1++ | 0.9456 | 1.8040 | 11.4718 |
| Method | Metric | 1 / 6 | 2 / 7 | 3 / 8 | 4 / 9 | 5 / 10 | Mean Std |
| FP++ | Proportion | 0.3640 | 0.3745 | 0.3600 | 0.3455 | 0.3540 | 0.3658 0.0146 |
| 0.3855 | 0.3775 | 0.3580 | 0.3515 | 0.3875 | |||
| FP++ | # Preserved | 1463 | 1495 | 1461 | 1451 | 1498 | 1473.8 18.0 |
| 1477 | 1491 | 1447 | 1480 | 1475 |
| Method | Basin fraction | Range | # Preserved |
|---|---|---|---|
| FP++ | – | ||
| FP | – | ||
| R2 | – | ||
| G2 | – | ||
| R10 | – | ||
| G10 | – |
| Method | Basin | Energy-W2 | Torsion-W2 | Lin. ESS/N | Imp. ESS/N | Unique/N |
|---|---|---|---|---|---|---|
| FP | 0.4750 | 1.6512 | 3.0853 | 0.036647 | 0.001190 | 0.7140 |
| FP++ | 0.3640 | 2.7330 | 2.7349 | 0.043632 | 0.003499 | 0.7315 |
| R2 | 0.2255 | 30.1826 | 2.7397 | 0.084090 | 0.002502 | 0.7685 |
| R10 | 0.2240 | 6.7210 | 2.6891 | 0.062073 | 0.001761 | 0.7675 |
| G2 | 0.1945 | 31.0080 | 2.8770 | 0.014943 | 0.000534 | 0.6650 |
| G10 | 0.2220 | 11.1386 | 2.7375 | 0.036490 | 0.002464 | 0.7225 |
| Method | Mean | Var. | CV 2 | ESS/ | |
|---|---|---|---|---|---|
| 10 | FP1 | 0.98836 | 15.20617 | 13.72306 | 0.44239 |
| 10 | FP2 | 0.98621 | 7.39192 | 7.32949 | 0.53670 |
| 10 | FP10 | 1.01540 | 5.91418 | 5.11221 | 0.67136 |
| 10 | FP1++ | 0.99862 | 0.15049 | 0.15189 | 0.88678 |
| 10 | FP2++ | 0.99951 | 0.07103 | 0.07135 | 0.93870 |
| 10 | FP10++ | 1.00008 | 0.01364 | 0.01364 | 0.98678 |
| Metric | FP++ direct | TBG official |
|---|---|---|
| Valid aligned samples | 199885 | 199871 |
| ESS | 3309.496 | 12854.656 |
| Normalized ESS | 0.016557 | 0.064315 |
| Energy-W2 (unw. w.) | ||
| torsion-W2 (unw. w.) | ||
| torsion-W2 (unw. w.) |
| Method | Time (s) | s/sample | Alloc. (MB) | Reserved (MB) |
|---|---|---|---|---|
| FP3++ | 80.511 | 0.805 | 608.22 | 676.00 |
| Exact Jacobian | 569.240 | 5.692 | 4276.49 | 6614.00 |
| Reweighting | ESS/N | Energy-W2 | Torsion-joint W2 |
|---|---|---|---|
| Exact Jacobian | 0.006896 | 6.0864 | 3.1380 |
| FP5++ | 0.000620 | 9.4396 | 2.7036 |