Manifold Attracting Flow Matching: Physics-Constrained Sampling via Continuous Geometric Dynamics
Organizations: Zhejiang University · University of California, Los Angeles · The Chinese University of Hong Kong · Pennsylvania State University
Abstract
Generating scientific data under previously unseen equality constraints requires satisfying the prescribed constraints without destroying the physical structure learned from data. Existing inference-time approaches often enforce physical constraints by repeatedly correcting predicted endpoints, but such discrete corrections can distort learned generative structure and introduce unstable feedback. In this work, we propose Manifold Attracting Flow Matching (MAFM), which continuously evolves a persistent anchor through an ODE whose dynamics drive it analytically toward the constraint manifold. Specifically, its geometric correction preserves model-guided tangent motion while prescribing the normal motion to contract constraint violations toward the manifold. This construction separates model tracking from constraint attraction and provides theoretical guarantees for residual contraction and local feedback stabilization. Experiments across geometric and partial differential equation benchmarks demonstrate improved generation quality over variant constraint-guided baselines while maintaining low constraint residuals. MAFM provides a flexible framework for coordinating learned physical structure and explicit constraints through controllable sampling dynamics, without retraining the underlying generative model.
Figures & tables
| (a) Partial observation | (b) Conservation | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Poisson, 64 sensors ( ) | Burgers ( ) | Reaction–diffusion ( ) | |||||||
| Method | Feas. | CE | PDE RelL2 | Feas. | CE | IC RelL2 | Feas. | CE | IC RelL2 |
| Vanilla FM | No | 1.40 | 1.13% | No | 1.13% | No | 1.31% | ||
| FM + projection | Yes | 0 | 22.7% | Yes | 27.0% | Yes | 10.4% | ||
| PCFM-Base | Yes | 0 | 11.5% | Yes | 7.61% | Yes | 3.03% | ||
| PCFM-Relaxed | Yes | 0 | 10.3% | No | 7.61% | No | 2.46% | ||
| (a) Poisson ( ) | (b) Conservation | ||||||
|---|---|---|---|---|---|---|---|
| Burgers ( ) | Reaction–diffusion ( ) | ||||||
| Variant | PDE RelL2 | Feas. | CE | IC RelL2 | Feas. | CE | IC RelL2 |
| Full MAFM | 1.70% | Yes | 0.965% | Yes | 2.74% | ||
| w/o adjoint | 38.0% | — | — | ||||
| Reset anchor | — | No | 9.44% | No | 6.14% | ||
| w/o support | 29.0% | Yes | 2.70% | Yes | 10.6% | ||
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
| Benchmark | Equation or data distribution | Structure learned from data | Explicit inference-time condition | Constraint type |
|---|---|---|---|---|
| Sphere–plane | Uniform unit-sphere distribution, . | Plane ; the desired support is the intersection circle. | Affine, single-coordinate algebraic | |
| Poisson | , | Joint samples satisfy the discrete Poisson equation and homogeneous boundary conditions. | ; only is observed, with no direct observations of . | Sparse, local linear observations |
| Burgers | Periodic viscous dynamics and their trajectory statistics. | Prescribed spatial mean at every frame. | Framewise global linear integral | |
| Reaction–diffusion | , | No-flux nonlinear dynamics, positive concentration support, and trajectory statistics. | Prescribed total mass at every frame. | Framewise global linear integral |
| Euler | Incompressible, zero-mean velocity trajectories generated by Euler dynamics. | Prescribed kinetic energy at every frame; incompressibility and zero mean are left to the learned prior. | Global nonlinear quadratic | |
| Controlled NS | Incompressible, zero-mean velocity trajectories generated by controlled NS dynamics. | Prescribed kinetic energy at every frame; incompressibility and zero mean are left to the learned prior. | Global nonlinear quadratic |
| Method | mean | q99 | Radial | SWD | MMD 2 | Joint |
|---|---|---|---|---|---|---|
| Vanilla FM | 0.5009 | 0.2877 | 0.2158 | 0.1281 | 0.05516 | 0.06% |
| Terminal projection | 0 | 0 | 0.2158 | 0.1078 | 0.03237 | 6.47% |
| PCFM-Base | 0 | 0 | 0.006273 | 0.009357 | 0.000559 | 56.57% |
| PCFM-Relaxed | 0.006328 | 0.009551 | 0.000562 | 56.08% | ||
| ECI (operator-equivalent) | 0 | 0 | 0.006273 | 0.009357 | 0.000559 | 56.57% |
| Stop-grad. guidance | 0 | 0 | 0.006310 | 0.01885 | 0.000768 | 57.35% |
| 64 sensors | 256 sensors | |||
| Method | CE | PDE RelL2 | CE | PDE RelL2 |
| Vanilla FM | 1.395 | 1.128% | 1.425 | 1.128% |
| Terminal projection | 0 | 22.73% | 0 | 46.31% |
| PCFM-Base | 0 | 11.51% | 0 | 18.05% |
| PCFM-Relaxed | 0 | 10.33% | 0 | 16.07% |
| ECI | 0 | 11.51% | 0 | 18.05% |
| Method | RelL2 | CE | Fluct. RelL2 | Field SWD | Relative std. error |
|---|---|---|---|---|---|
| Vanilla FM | 1.134% | 3.163% | 0.2695 | 0.2323 | |
| Terminal projection | 26.97% | 74.48% | 0.07782 | 0.0883 | |
| PCFM-Base | 7.610% | 13.48% | 0.1776 | 0.5160 | |
| PCFM-Relaxed | 7.610% | 13.47% | 0.1777 | 0.5163 | |
| ECI (operator-equivalent) | 7.610% | 13.48% | 0.1776 | 0.5160 | |
| ECI-2 | 7.537% | 13.38% | 0.1766 | 0.5125 |
| Method | Field SWD | RelL2 | CE | q95 RelL2 | Negative fraction |
|---|---|---|---|---|---|
| Vanilla FM | 0.09650 | 1.312% | 5.201% | ||
| Terminal projection | 0.08529 | 10.39% | 41.51% | ||
| PCFM-Base | 0.1303 | 3.030% | 10.26% | ||
| PCFM-Relaxed | 0.1157 | 2.458% | 8.482% | ||
| ECI | 0.1246 | 2.962% | 10.18% | ||
| Stop-grad. guidance | 0.1220 | 6.040% | 23.45% |
| Task | Variant | CE | Physical RelL2 | Field SWD | RelL2 | Paired 95% CI |
| Poisson | Full MAFM | 0.01705 | – | – | – | |
| w/o adjoint | 0.3803 | – | 0.3632 | [0.3521, 0.3751] | ||
| w/o support | 0.2899 | – | 0.2728 | [0.2534, 0.2937] | ||
| Burgers | Full MAFM | 0.009648 | 0.1574 | – | – | |
| Reset anchor | 0.008997 | 0.09437 | 0.1767 | 0.08473 | [0.07948, 0.09028] | |
| w/o support | 0.02703 | 0.3135 | 0.01738 | [0.01652, 0.01828] |
| Case | Method | Feature SWD | Field SWD | RelL2 | Div. | Mean | Energy |
|---|---|---|---|---|---|---|---|
| Euler | Vanilla FM | 0.152 | 0.0662 | 0.0368 | 0.0635 | 0.0131 | 0.0186 |
| Terminal projection | 0.152 | 0.0663 | 0.0332 | ||||
| PCFM-Base | 0.424 | 0.115 | 0.125 | ||||
| PCFM-Relaxed | 0.427 | 0.116 | 0.124 | 0.00766 | 0.00182 | ||
| ECI (operator-equivalent) | 0.424 | 0.115 | 0.125 | ||||
| Stop-grad. guidance | 0.427 | 0.0985 | 0.0429 | 0.0598 | 0.0161 | 0.0262 |
| Case | Sampler | NFE | Input VJPs | Sampling time | CE | Physical RelL2 |
| Poisson | Vanilla FM | 100 | 0 | 6.615 s/512 | 1.366 | 1.128% |
| (64 sensors) | PCFM-Relaxed | 100 | 0 | 15.77 s/512 | 0 | 10.26% |
| MAFM | 100 | 100 | 17.88 s/512 | 1.710% | ||
| Reaction–diffusion | PCFM-Base | 100 | 0 | 0.04249 s/sample | 3.030% | |
| ECI | 100 | 0 | 0.04361 s/sample | 2.962% | ||
| MAFM | 100 | 0 | 0.04375 s/sample | 2.740% |
| Mass | Sampler | CE | Own-IC (%) | Field SWD |
| 0.55 | CCFM ( ) | 5.394 | 0.06016 | |
| CCFM ( ) | 7.563 | 0.06381 | ||
| MAFM | 1.092 | 0.16484 | ||
| 0.90 | CCFM ( ) | 3.365 | 0.05831 | |
| CCFM ( ) | 4.848 | 0.05910 | ||
| MAFM | 0.829 | 0.14856 |
| Benchmark | CE | Relative | CE | Relative |
|---|---|---|---|---|
| Controlled NS | 4.05% | 3.48% | ||
| Euler | 4.20% | 3.68% | ||
| Burgers | 4.08% | 5.73% | ||
| Reaction–diffusion | 6.49% | 6.83% | ||
| Poisson (64 sensors) | 0 | 22.74% | 0 | 22.78% |
| Method | Burgers | Reaction–diffusion |
|---|---|---|
| Vanilla FM | 0.270 | 0.0965 |
| FM + projection | 0.0778 | 0.0853 |
| PCFM-Base | 0.178 | 0.130 |
| PCFM-Relaxed | 0.178 | 0.116 |
| ECI ∗ | 0.177 | 0.125 |
| Stop-grad. | 0.288 | 0.122 |
| Method | Euler | Controlled NS |
|---|---|---|
| Vanilla FM | 0.0662 | 0.0495 |
| Terminal projection | 0.0662 | 0.0495 |
| PCFM-Base | 0.114 | 0.0945 |
| PCFM-Relaxed | 0.115 | 0.0948 |
| ECI † | 0.114 | 0.0945 |
| Stop-grad. guidance | 0.118 | 0.0792 |
| Case | Sampler | Energy CE | Own-IC (%) | Divergence | Field SWD |
|---|---|---|---|---|---|
| Euler | PCFM-Relaxed replay | 13.56 | 0.2285 | 0.11503 | |
| + terminal projection | 13.56 | 0.2287 | 0.11506 | ||
| Two mix, fixed source | 13.41 | 0.2266 | 0.11407 | ||
| Two mix, refreshed source | 13.51 | 0.2303 | 0.09906 | ||
| Controlled NS | PCFM-Relaxed replay | 12.29 | 0.2123 | 0.09479 | |
| + terminal projection | 12.29 | 0.2124 | 0.09485 |
| Euler | ||||||
|---|---|---|---|---|---|---|
| training energy | training energy | |||||
| Method | Energy CE | IC RelL2 | Div. RMS | Energy CE | IC RelL2 | Div. RMS |
| Vanilla FM | 12.49% | 0.0621 | 9.93% | 0.0621 | ||
| Vanilla + projection | 4.01% | 0.0556 | 4.60% | 0.0682 | ||
| PCFM-Base / ECI-1 | 11.44% | 0.1810 | 19.79% | 0.3544 | ||
| PCFM-Relaxed (pre-proj.) | 11.75% | 0.1884 | 20.44% | 0.3642 | ||
| Euler | Controlled NS | |||||
|---|---|---|---|---|---|---|
| Method | Energy CE | Rel. | Div. RMSE | Energy CE | Rel. | Div. RMSE |
| PCFM-Relaxed + terminal projection | 37.92% | 0.805 | 25.20% | 0.745 | ||
| ECI-2 (refreshed source) | 35.03% | 0.785 | 25.56% | 0.731 | ||
| Full-gradient guidance | 30.72% | 0.061 | 30.53% | 0.066 | ||
| CCFM ( ) | 22.96% | 0.655 | 19.25% | 0.646 | ||
| MAFM + terminal projection | 9.85% | 0.210 | 10.01% | 0.222 | ||
| Euler | Controlled NS | |||||
|---|---|---|---|---|---|---|
| Method | Energy CE | Rel. | Div. RMSE | Energy CE | Rel. | Div. RMSE |
| CCFM ( ) | 3.68% | 0.06499 | 3.48% | 0.06402 | ||
| D-Flow | 0.00751 | 3.54% | 0.06256 | 0.00748 | 3.27% | 0.05994 |
| PnP-Flow | 0.0581 | 11.33% | 0.20279 | 0.0554 | 10.25% | 0.18868 |
| LDF (198 NFE) | 7.13% | 0.16506 | 5.86% | 0.15182 | ||
| MAFM | 2.94% | 0.04829 | 2.76% | 0.04736 | ||