We introduce Equilibrium Matching (EqM), a generative modeling framework built from an equilibrium dynamics perspective. EqM discards the non-equilibrium, time-conditional dynamics in traditional diffusion and flow-based generative models and instead learns the equilibrium gradient of an implicit energy landscape. Through this approach, we can adopt an optimization-based sampling process at inference time, where samples are obtained by gradient descent on the learned landscape with adjustable step sizes, adaptive optimizers, and adaptive compute. Empirically, EqM achieves the best FID among the tested methods, reaching 1.90 on ImageNet 256×256, while remaining competitive on other metrics. Theoretically, we show under standard assumptions that real images are attracting critical points of the learned field and the energy interpretation applies when the field is conservative. Beyond generation, EqM is a flexible framework that naturally handles tasks including partially noised image denoising, OOD detection, and image composition. By replacing time-conditional velocities with a unified equilibrium landscape, EqM offers a tighter bridge between flow and energy-based models and a simple route to optimization-driven inference.
Figures & tables
Figure 1: Conceptual 2D Visualization. We compare the conceptual 2D dynamics of Equilibrium Matching and Flow Matching under 2 ground truths (marked by stars). Left : Flow Matching learns non-equilibrium velocity that varies over time. Right : Equilibrium Matching learns an equilibrium gradient that is time-invariant.
Figure 11: Partially Noised Image Denoising. EqM’s generation quality improves with less noise while an FM baseline without the noise level drops.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
metric
EqM
SiT
Precision ( k=3 )
0.806
0.802
Recall ( k=3 )
0.609
0.597
Density ( k=5 )
1.130
1.143
Coverage ( k=5 )
0.9997
0.9997
Mean per-class coverage
0.943 (0.068)
0.943 (0.070)
DINOv2 Fréchet distance
101.3
117.3
Appendix
Table 9: EqM achieves competitive additional metrics. Distributional metrics for EqM and SiT under the identical XL/2 protocol. Parentheses denote the standard deviation of per-class coverage.
mean NFE
FID-10k
657
36.0
241
36.5
168
38.1
141
39.6
250 (fixed)
36.4
Appendix
Table 10: Quality–compute trade-off for adaptive stopping on EqM-B/2.
starting noise level
method
0.8
0.4
0.0
EqM
59.2
20.2
6.2
SiT w/o noise-level input
125.0
164.2
138.1
Oracle SiT
58.2
20.2
5.4
Appendix
Table 11: EqM matches oracle FM without time input. FID-10k for partially noised inputs. Oracle SiT is provided the true starting noise level, whereas EqM and standard SiT are not.
Figure 12: Energy Evolution During Sampling. We report the energy curve during sampling on EqM-E. The energy value decreases smoothly and consistently, matching our expectation.
setting
top-5 accuracy
single-class control
0.2%
EqM composed samples
1.8%
Appendix
Table 12: EqM composes classes with higher accuracy. To measure composition, we report ResNet-50 top-5 classification on both labels of EqM composed image and compare against a single-class control paired with a random second class. EqM’s composed samples provide much stronger signal to classification on both classes.
sampler
FID-10k
Recall
GD
20.42
0.627
NAG-GD
20.34
0.634
Annealed Langevin
21.11
0.634
Appendix
Table 13: EqM supports stochastic sampling. Stochastic sampling results for EqM-B/2. Annealed Langevin sampling uses a cosine schedule with T0=0.01 .
model
S/2
B/2
L/2
XL/2
model configurations
params (M)
33
130
458
675
depth
12
12
24
28
hidden dim
384
768
1024
1152
patch size
2
2
2
2
heads
6
12
16
16
Appendix
Table 14: Equilibrium Matching Configurations.
Table 19
sampler
GD
NAG-GD
Adam
FID
32.85
32.97
36.35
Appendix
Table 17: EqM Sampling with Adam. We find that Adam without momentum achieves decent generation quality on EqM-B/2.
Pixel-space generative models bypass lossy latent compression, yet necessitate joint learning of global structure and fine-grained details in a high-dimensional space. Standard flow matching interpolates noise toward a fixed clean-image endpoint, leaving the spectral evolution to be learned implicitly. In this paper, we introduce Energy-Guided Flow Matching(EG-FM) that explicitly models a coarse-to-fine generative trajectory by moving endpoint. Specifically, EG-FM replaces the fixed endpoint with a heat-kernel-filtered endpoint that evolves smoothly from low-frequency image to clean image. The fraction of high-frequency signal in moving endpoint is released by an image-specific energy-guided scheduling, leading to the re-targeting of velocity in flow matching. Our framework requires no adaptation of the backbone and training data, bringing negligible cost on the training and inference stages. In our experiment, EG-FM consistently achieves lower FID on the ImageNet class-conditional image generation task at 256×256 with fewer epochs, reaching an FID of 1.55 at 200 epochs and 1.45 at 600 epochs. We continue training the generation task on the setting of 512×512 resolution, yielding a FID of 1.58 after only 40 high-resolution adaptation epochs. Furthermore, we transfer EG-FM on text-to-image generation and achieve 0.85 on GenEval score and 83.9 on DPG-Bench. Code is available at https://github.com/ysng123/EG-FM.
Haoyang Tong, Yu He, Fang Li +6
MAIS & NLPR, CASIA · JD.com · Xi'an Jiaotong University +1
Generating an image with a diffusion or flow-matching model typically involves integrating a velocity field for hundreds of sequential steps. That computation suits a GPU poorly and an analog neuromorphic substrate well, where one fast, low-power physical relaxation could replace the whole sequence. Two things are needed: a generative algorithm whose computation is a relaxation, and a way to train it without backpropagation, whose global gradient transport is hard to realise in analog. Energy Matching supplies the first, sampling by gradient-descent relaxation on a time-independent potential - but obtaining the gradient that descent follows is itself a backward pass, once per integration step. We prove that a predictive coding network with a single linear injection term added to its energy relaxes to a state whose visible layer carries that gradient inherently, with an error that vanishes for small injection strengths. An Energy Matching model trained by backpropagation can therefore be loaded into this network unchanged and sampled by relaxation alone, matching the original model's FID to within 0.04 at CIFAR-10 and ImageNet-32 scale. The network can also be trained without backpropagation: Equilibrium Propagation reproduces backpropagation's parameter updates to a cosine similarity of 0.9998, and training end to end with those updates reaches an FID of 3.64 on unconditional CIFAR-10 against 3.54 for backpropagation. To our knowledge this is the first CIFAR-10 generative model trained with Equilibrium Propagation, and the first at competitive quality by any backpropagation-free rule. This quality does not require group normalisation or attention: an architecture of only convolutions and pointwise nonlinearities, both native to an analog crossbar, reaches an FID of 3.51 when trained by backpropagation.
Flow Matching (FM) is a simulation-free method for learning a continuous, invertible flow that interpolates between two distributions, and in particular generates data from noise. Inspired by the variational nature of the diffusion process as a gradient flow, we introduce a stepwise FM model, Local Flow Matching (LFM), which sequentially learns a sequence of FM submodels, each matching a diffusion process up to the time-step size in the data-to-noise direction. In each step, the two distributions to be interpolated by the sub-flow model are closer than those in the full-flow matching model, which interpolates data to noise distributions, enabling smaller models with more efficient training. This variational perspective also allows us to prove a theoretical generation guarantee for the proposed flow model in terms of the χ2-divergence between the generated and true data distributions, leveraging the contraction property of the diffusion process. In practice, the stepwise structure of LFM is naturally amenable to model distillation, and various distillation techniques can be applied to accelerate generation. We empirically demonstrate that LFM achieves competitive generative performance compared to FM on unconditional generation of tabular and image datasets, and on conditional generation of robotic manipulation policies.
Chen Xu, Xiuyuan Cheng, Yao Xie
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology. · Department of Mathematics, Duke University