Fokker–Planck Equation

Follow this topic Ask about this topic

Latest papers

16 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Oct 2, 2026

Probability flow ODEs in score-based and reflected diffusion models

Probability-flow ordinary differential equations (PF-ODEs) are widely used as deterministic samplers for score-based diffusion models. Their usual justification is that the Fokker--Planck equation of a diffusion can be rewritten as a continuity equation driven by the score function of the forward diffusion. This identity does not, however, guarantee that the resulting velocity field generates a well-posed flow. We provide theoretical insights into the design of such deterministic samplers for generative models based on diffusions and reflected diffusions. We identify sufficient conditions for a regular Lagrangian PF-ODE flow to exist; reverse sampling and invertibility require two-sided divergence control. For learned scores, sampler stability is controlled by an unweighted velocity error, exposing a mismatch with density-weighted score matching and motivating architectural control of Jacobians, divergence, growth, and compression. Under the manifold hypothesis, positive-time regularization justifies an early-stopped PF-ODE while constants deteriorate near the data endpoint; an explicit sphere example shows that the exact deterministic flow becomes singular as the noise level vanishes even though the diffusion marginals remain well defined. These theoretical insights translate into concrete design principles for stable, invertible, and constraint-preserving diffusion samplers. We illustrate the practical relevance of these design principles using controlled numerical experiments.

Rama CONT

Mathematical Institute, University of Oxford

Diffusion Sampling Diffusion Models Score-Based Generative Modeling Stochastic Differential Equations Fokker–Planck Equation

Sep 28, 2026

Beyond Gradient Flow: Identifiability and Recovery from Distribution Snapshots

Inferring dynamics from snapshots of evolving distributions is fundamentally underdetermined: the Fokker-Planck equation constrains the drift

F

only through its score-weighted divergence

\nabla\cdot F+F\cdot\nabla\logρ

, leaving a

ρ

-solenoidal gauge invisible to any single-time constraint. Time-indexed transport formulations cannot resolve this ambiguity: every admissible marginal path admits a curl-free explanation, minimum-action reconstruction selects it, and marginal fit alone cannot distinguish dynamically inequivalent explanations. Requiring one autonomous field to explain several marginals instead makes part of the hidden circulation visible as

\nabla\logρ

changes across marginals. Separating instantaneous Fokker-Planck source constraints from the snapshot experiment, we show that the source constraints identify the field modulo the kernel of a stacked score-weighted divergence operator. For generic Gaussian shape variation, source constraints at

K\ge m

time points in intrinsic dimension

m

eliminate every polynomial gauge direction, whereas finitely many density snapshots alone admit aliasing; we give the obstruction explicitly. At a Gaussian anchor, for Sobolev smoothness

s

and

n

samples per time point, we derive a conditional lower rate

(nK)^{-2s/(2s+m+1)}

for the tangent snapshot experiment, with a matching upper rate in a degreewise benchmark. Strong-form fitting is non-orthogonal to score error and cannot be repaired by spectral filtering. Instead, we estimate using smooth test functions while retaining the known diffusion term, and derive a finite-sample bound that separates sampling error from fixed-grid quadrature bias. Planted-circulation experiments confirm the predicted gauge contraction and expose a design tension between cross-slice information and covariance-aware whitening.

Nam D. Nguyen, Valeriya Malysheva

VIB, Center for Molecular Neurology, Antwerp, Belgium · VIB, Center for AI and Computational Biology, Leuven, Belgium · Faculty of Pharmaceutical, Biomedical and Veterinary Sciences, University of Antwerp, Antwerp, Belgium +2

Fokker–Planck Equation System Identification PDE Inverse Problems

Aug 3, 2026

A Hyperfinite Framework for Score-Based Generative Modeling

Score-based diffusion models are typically formulated using continuous-time stochastic differential equations and measure-theoretic stochastic calculus. In this paper, we develop a hyperfinite formulation of score-based generative modeling within the framework of Nonstandard Analysis. Starting from an internal diffusion process on a hyperfinite grid, we derive the associated infinitesimal generator and establish its correspondence with the classical Fokker--Planck equation. We then obtain a hyperfinite backward-mean identity that yields the reverse-time drift and provides a constructive derivation of the reverse-time SDE. Building on these results, we show that minimization of an internal score-matching objective recovers the score function required by the reverse-time dynamics, thereby connecting score estimation with generative sampling directly at the hyperfinite level. Under suitable assumptions, we further derive a hyperfinite Girsanov formula and establish a relationship between likelihood optimization and Fisher-divergence objectives. Finally, we analyze the second-order consistency of the hyperfinite dynamics and show that the leading correction term depends explicitly on the fourth moment of the increment distribution, with the Gaussian value

κ=3

eliminating the leading dispersion contribution. Taken together, these results provide a unified hyperfinite framework for diffusion-based generative modeling--while laying foundations for further extensions--that links discrete grid dynamics, reverse-time diffusion, score matching, and likelihood-based formulations within a common nonstandard setting.

Sunder Ram Krishnan

Kerala School of Mathematics, Kozhikode 673571, Kerala, India.

Score Matching Stochastic Differential Equations Fokker–Planck Equation Score-Based Generative Modeling

Jul 30, 2026

Reflected diffusion, no-flux continuity equations and confined Lagrangian flows in bounded domains

Motivated by marginal distribution flows of reflected diffusions in bounded domains, we investigate when a density/flux pair solving a no-flux continuity equation admits a regular Lagrangian flow that remains in the closed domain and generates the prescribed density flow. We give sufficient conditions in terms of interior bounded-variation regularity, bounded-variation control on a boundary collar, a one-sided bound on an absolutely continuous divergence, and vanishing normal trace of the velocity. The proof uses the fact that tangency removes the singular boundary contribution to the divergence of the zero extension, thereby making the extended velocity admissible for the Ambrosio-DiPerna-Lions theory. We show that these boundary assumptions cannot be jointly relaxed so as to admit a boundary current mechanism. We construct an explicit smooth density/flux pair carrying a boundary current. Its density evolution is unique in a weighted class and its characteristics are unique, confined and transport the marginals, yet it admits no regular Lagrangian flow because the compressibility bound fails arbitrarily close to the initial time. We also establish two uniqueness results for no-flux Fokker-Planck equations: a duality result for bounded measurable drifts and a weighted energy result for entrance-type drifts singular at the boundary. Our results provide a rigorous mathematical justification for using the ODE-based sampling of reflected diffusion models under minimal regularity assumptions on the coefficients, and also indicate when such ODE-based samplers may fail.

Rama Cont

Dynamical Systems Stochastic Differential Equations Fokker–Planck Equation PDE Solving

Jul 27, 2026

Stochastic Counterdiabatic Driving via Biorthogonal Liouvillian Eigenmodes

Finite-time driving of stochastic systems generates excess dissipation, causing the evolving probability distribution to lag behind the instantaneous equilibrium, and consequently degrading the convergence of nonequilibrium free energy estimators based on the Jarzynski equality. Escorted free energy simulations address the non-adiabatic lag by engineering control fields

\mathbf{u}

that eliminate the lag, enforcing the trajectory-wise equality

\mathcal{W}_\mathbf{u} = Δ\mathcal{F}

, and yielding zero-variance estimators. However, constructing the escorting field in closed form remains a challenge, approached variously through flow-field methods, targeted free energy perturbation, or learned diffeomorphisms. In this work, we construct a complementary numerical framework based on gauge-type transforms instead of generalized coordinate transforms for perfect escorting based on the exact spectral decomposition of the time-dependent Fokker-Planck generator. The biorthogonal decomposition of the Liouville operator directly yields a counterdiabatic correction whose action on the instantaneous equilibrium distribution exactly cancels the non-adiabatic lag at arbitrary driving speed in formal analogy with shortcuts-to-adiabaticity techniques such as Berry's transitionless driving for quantum systems. Numerical verification for simulations of an overdamped particle in a time-varying double-well potential and harmonic traps confirms that the counterdiabatic condition is satisfied to machine precision, with the non-adiabatic lag suppressed by roughly twelve orders of magnitude in total variation distance and sixteen orders in KL divergence relative to the unescorted dynamics. As a diagnostic, we demonstrate vanishing dissipated work

\mathcal{W}_{\text{diss}}(t) \approx 0

for the deterministically propagated Fokker-Planck density across all protocol speeds.

Sandeep Suresh Cranganore, Sebastian Lehner, Johannes Brandstetter +1

Institute for Machine Learning (Ellis Unit), Johannes Kepler University, Linz · Mistral AI, Paris, France · AMLab, Informatics Institute, University of Amsterdam, The Netherlands +1

Fokker–Planck Equation

Jun 8, 2026

Operator learning for solving Fokker-Planck equations with various initial conditions

The Fokker-Planck equation (FPE) plays a pivotal role in describing the time evolution of probability density functions (PDFs) for systems governed by stochastic dynamics. In this work, we propose a conditional normalizing flow-based physics-informed neural network (PINN) framework for efficiently approximating the solution operator of the FPE for a whole range of initial conditions. Leveraging the Chapman-Kolmogorov equation for Markovian stochastic processes, the problem is reformulated into approximating a transition PDF starting at initial time from a Dirac mass centered at an arbitrary point. The PDF of an associated linearized stochastic differential equation (SDE) is employed as the base distribution for the normalizing flow, providing a good approximation of the target PDF, especially for small times, and thereby avoiding the singularity of the map associated with the Dirac delta initial distribution. Furthermore, a time-weighted loss function is introduced to mitigate numerical instabilities arising at small times, achieving a balance between causality and training difficulty as time progresses. A variety of numerical experiments are presented to illustrate the effectiveness and robustness of the proposed method.

Li Zeng, Xiaoliang Wan, Yaobin Wang +2

School of Mathematics and Statistics, Fuzhou University, Fuzhou, China · Department of Mathematics and Center for Computation and Technology, Louisiana State University, Baton Rouge 70803, USA · Faculty of Science and Technology, Beijing Normal-Hong Kong Baptist University, Zhuhai 519087, China. +2

PDE Solving PDE Operator Learning Physics-Informed ML Fokker–Planck Equation

Jun 1, 2026

The Crowded Embedding Space: A Mean-Field Mechanism for Emergent Marginalization in Retrieval-Augmented Agents

Retrieval-augmented generative agents rely on retrieval for grounding, yet are typically evaluated on a query-by-query basis. This isolates interactions that are geometrically coupled in a shared embedding space. For example, we show that the high document density required to serve majority interests (e.g., generic "Crime" movies) can geometrically overcrowd the retrieval neighborhood of a semantically similar minority (e.g., "Film Noir"), effectively expelling minority content from top-

k

results. We introduce a formal framework to analyze how such goal collisions in dense retrieval induce fundamental performance limits and emergent fairness issues inherent to spatial crowding. In our static analysis, we demonstrate that for a fixed embedding space, a phase transition occurs where minority user goals suffer a catastrophic collapse in performance as the density of majority goals increases. We then extend this to a dynamic model and derive a non-linear Fokker-Planck equation that governs the evolution of document embeddings as the agent updates them to maximize retrieval accuracy. Our analysis reveals that this local relevance objective triggers an emergent global mechanism that systematically marginalizes minority interests. We prove that such objectives drive the system to self-organize into a state that exclusively serves majority interests. These results provide a theoretical foundation for understanding a critical grounding failure mode in retrieval-augmented agents.

Shwan Ashrafi, Dan Roth

Oracle AI

Fokker–Planck Equation Algorithmic Fairness Mean-Field Theory Agentic RAG Dense Retrieval

May 21, 2026

Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics

Stochastic Gradient Descent (SGD) is commonly modeled as a Langevin process, assuming that minibatch noise acts as Brownian motion. However, this approximation relies on a continuous-time limit and a sqrt(eta) noise scaling that does not match the discrete SGD update at finite learning rate. In this work, we propose an alternative formulation of SGD as deterministic dynamics in a fluctuating loss landscape induced by minibatch sampling. Starting directly from the discrete update, we derive a master equation for the parameter distribution and obtain a discrete Fokker--Planck equation that differs from the standard Langevin form at order eta^2. Using this framework, we analyze SGD dynamics near critical points of the loss. We show that the behavior decomposes along the eigenbasis of the mean Hessian into qualitatively distinct regimes. In particular, nearly-flat directions do not admit a stationary distribution: the variance grows over time, corresponding to effective diffusion along valleys with a coefficient proportional to the learning rate. We provide empirical evidence supporting these predictions on neural network models in computer vision and natural language processing, observing a clear qualitative separation between confined and diffusive modes.

Igor Ignashin, Anna Radovskaya, Andrew Semenov +7

Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · P.N. Lebedev Physical Institute of the Russian Academy of Sciences · Innopolis University

Fokker–Planck Equation Neural Network Training Dynamics Gradient Descent Dynamics Stochastic Gradient Descent

Apr 16, 2026

An Analysis of Regularization and Fokker-Planck Residuals in Diffusion Models for Image Generation

Recent work has shown that diffusion models trained with the denoising score matching (DSM) objective often violate the Fokker--Planck (FP) equation that governs the evolution of the true data density. Directly penalizing these deviations in the objective function reduces their magnitude but introduces a significant computational overhead. It is also observed that enforcing strict adherence to the FP equation does not necessarily lead to improvements in the quality of the generated samples, as often the best results are obtained with weaker FP regularization. In this paper, we investigate whether simpler penalty terms can provide similar benefits. We empirically analyze several lightweight regularizers, study their effect on FP residuals and generation quality, and show that the benefits of FP regularization are available at substantially lower computational cost. Our code is available at https://github.com/OnnoNiemann/fp_diffusion_analysis.

Onno Niemann, Gonzalo Martínez Muñoz, Alberto Suárez Gonzalez

Universidad Autónoma de Madrid

Diffusion-Based Image Generation Score-Based Generative Modeling Diffusion Models Fokker–Planck Equation

Apr 13, 2026

Generative Path-Finding Method for Wasserstein Gradient Flow

Wasserstein gradient flows (WGFs) describe the evolution of probability distributions in Wasserstein space as steepest descent dynamics for a free energy functional. Computing the full path from an arbitrary initial distribution to equilibrium is challenging, especially in high dimensions. Eulerian methods suffer from the curse of dimensionality, while existing Lagrangian approaches based on particles or generative maps do not naturally improve efficiency through time step tuning. We propose GenWGP, a generative path finding framework for Wasserstein gradient paths. GenWGP learns a generative flow that transports mass from an initial density to an unknown equilibrium distribution by minimizing a path loss that encodes the full trajectory and its terminal equilibrium condition. The loss is derived from a geometric action functional motivated by Dawson Gartner large deviation theory for empirical distributions of interacting diffusion systems. We formulate both a finite horizon action under physical time parametrization and a reparameterization invariant geometric action based on Wasserstein arclength. Using normalizing flows, GenWGP computes a geometric curve toward equilibrium while enforcing approximately constant intrinsic speed between adjacent network layers, so that discretized distributions remain nearly equidistant in the Wasserstein metric along the path. This avoids delicate time stepping constraints and enables stable training that is largely independent of temporal or geometric discretization. Experiments on Fokker Planck and aggregation type problems show that GenWGP matches or exceeds high fidelity reference solutions with only about a dozen discretization points while capturing complex dynamics.

Chengyu Liu, Xiang Zhou

Department of Data Science, City University of Hong Kong, Kowloon, Hong Kong SAR · Department of Mathematics, City University of Hong Kong, Kowloon, Hong Kong SAR

Fokker–Planck Equation Wasserstein Gradient Flows PDE Solving

Mar 19, 2026

Neural Galerkin Normalizing Flow for Transition Probability Density Functions of Diffusion Models

We propose a new Neural Galerkin Normalizing Flow framework to approximate the transition probability density function of a diffusion process by solving the corresponding Fokker-Planck equation with an atomic initial distribution, parametrically with respect to the location of the initial mass. By using Normalizing Flows, we look for the solution as a transformation of the transition probability density function of a reference stochastic process, ensuring that our approximation is structure-preserving and automatically satisfies positivity and mass conservation constraints. By extending Neural Galerkin schemes to the context of Normalizing Flows, we derive a system of ODEs for the time evolution of the Normalizing Flow's parameters. Adaptive sampling routines are used to evaluate the Fokker-Planck residual in meaningful locations, which is of vital importance to address high-dimensional PDEs. Numerical results show that this strategy captures key features of the true solution and enforces the causal relationship between the initial datum and the density function at subsequent times. After completing an offline training phase, online evaluation becomes significantly more cost-effective than solving the PDE from scratch. The proposed method serves as a promising surrogate model, which could be deployed in many-query problems associated with stochastic differential equations, like Bayesian inference, simulation, and diffusion bridge generation.

Riccardo Saporiti, Fabio Nobile

CSQI, École Polytechnique Féderale de Lausanne, Switzerland

Normalizing Flows Neural PDE Solvers PDE Solving PDE Surrogate Modeling Fokker–Planck Equation

Mar 17, 2026

Neural Pushforward Samplers for the Fokker-Planck Equation on Embedded Riemannian Manifolds

In this paper, we extend the Weak Adversarial Neural Pushforward Method to the Fokker--Planck equation on compact embedded Riemannian manifolds. The method represents the solution as a probability distribution via a neural pushforward map that is constrained to the manifold by a retraction layer, enforcing manifold membership and probability conservation by construction. Training is guided by a weak adversarial objective using ambient plane-wave test functions, whose intrinsic differential operators are derived in closed form from the geometry of the embedding, yielding a fully mesh-free and chart-free algorithm. Both steady-state and time-dependent formulations are developed, and numerical results on a double-well problem on the two-sphere demonstrate the capability of the method in capturing multimodal invariant distributions on curved spaces.

Andrew Qing He, Wei Cai

Department of Mathematics, Southern Methodist University, Dallas, TX 75275.

Fokker–Planck Equation PDE Solving Neural PDE Solvers

Mar 10, 2026

Differentiable Stochastic Traffic Dynamics: Physics-Informed Generative Modelling in Transportation

Macroscopic traffic flow is stochastic, but the physics-informed deep learning methods currently used in transportation literature embed deterministic PDEs and produce point-valued outputs; the stochasticity of the governing dynamics plays no role in the learned representation. This work develops a framework in which the physics constraint itself is distributional and directly derived from stochastic traffic-flow dynamics. Starting from an Ito-type Lighthill-Whitham-Richards model with Brownian forcing, we derive a one-point forward equation for the marginal traffic density at each spatial location. The spatial coupling induced by the conservation law appears as an explicit conditional drift term, which makes the closure requirement transparent. Based on this formulation, we derive an equivalent deterministic Probability Flow ODE that is pointwise evaluable and differentiable once a closure is specified. Incorporating this as a physics constraint, we then propose a score network with an advection-closure module, trainable by denoising score matching together with a Fokker-Planck residual loss. The resulting model targets a data-conditioned density distribution, from which point estimates, credible intervals, and congestion-risk measures can be computed. The framework provides a basis for distributional traffic-state estimation and for stochastic fundamental-diagram analysis in a physics-informed generative setting.

Wuping Xin

Caliper Corporation

Physics-Informed Generative Modeling Traffic Flow Estimation Physics-Informed Diffusion Models Stochastic Differential Equations Fokker–Planck Equation

Dec 22, 2025

Adaptive Probability Flow Residual Minimization for High-Dimensional Fokker-Planck Equations

Solving high-dimensional Fokker-Planck (FP) equations remains a challenging problem in computational physics and stochastic dynamics, due to the curse of dimensionality, unbounded domains, and complex probability landscapes. In this work, we propose an adaptive probability flow residual minimization (A-PFRM) method for this problem. The second-order FP equation is reformulated as an equivalent first-order continuity equation associated with the probability flow ordinary differential equation, based on which a loss function is constructed to train neural network approximations without Hessian computation. To further improve the computational efficiency, the Hutchinson trace estimator is applied to compute the divergence in the corresponding score function, such that the training time can be dimension-independent on GPUs. Adaptive sampling strategies are employed to generate the collocation points, and our analysis shows that the Kullback-Leibler divergence between our A-PFRM approximation and the exact solution is bounded by the residual loss weighted by the estimated density function. Numerical experiments are presented to demonstrate the performance of A-PFRM, which include Ornstein-Uhlenbeck (OU) processes problems, Brownian motions with time-varying diffusion, and Geometric OU processes featuring non-Gaussian solutions up to one hundred dimensions.

Xiaolong Wu, Qifeng Liao

School of Information Science and Technology, ShanghaiTech University, Shanghai, 201210, China

Neural PDE Solvers PDE Solving Adaptive Sampling Fokker–Planck Equation

Nov 8, 2025

A PDE Perspective on Generative Diffusion Models

Score-based diffusion models have emerged as a powerful class of generative methods, with successful applications across diverse domains. Despite their empirical success, their mathematical foundations remain only partially understood, particularly regarding the stability and consistency of the stochastic and partial differential equations underlying their dynamics. In this work, we develop a partial differential equation (PDE) framework for score-based diffusion processes associated with the heat flow of a compactly supported data measure. After establishing weak well-posedness on positive-time intervals, we derive a sharp one-sided divergence bound for the score and use it to obtain sharp uniform

L^p

-stability estimates for the generative Fokker--Planck equation. This bound is the Euclidean form of the Li--Yau differential inequality. Through entropy stability methods, we prove that the generated laws converge in the

2

-Wasserstein distance at rate

\sqrt t

as

t\to0^+

to a probability measure supported on the data support. This holds for both deterministic and stochastic generation, for positive terminal densities with finite absolute entropy and second moment. In the deterministic case, almost every trajectory converges to a point of the data support; for empirical data, we further obtain a trajectory-wise square-root rate. These results clarify how exact-score dynamics concentrate on the data support without necessarily recovering the data distribution. An explicitly solvable case identifies the limiting weights and their dependence on Gaussian initialization, while a relative-entropy estimate quantifies the effect of score approximation. Together, these findings connect the stability and concentration mechanisms with score learning and early stopping, providing a mathematical perspective on the balance between imitation fidelity and generative diversity.

Kang Liu, Enrique Zuazua

Universit´e Bourgogne Europe, CNRS, Institut de Mathematiques de Bourgogne, 21000 Dijon, France. · Friedrich - Alexander - Universität Erlangen - Nürnberg, Department of Mathematics, Chair for Dynamics, Control, Machine Learning, and Numerics (Alexander von Humboldt Professorship), 91058 Erlangen, Germany. · Universidad Aut´onoma de Madrid, Departamento de Matem´aticas, 28049 Madrid, Spain. +1

Fokker–Planck Equation Diffusion Models Score-Based Generative Modeling

Aug 18, 2025

Optimal Condition for Initialization Variance in Deep Neural Networks: An SGD Dynamics Perspective

Stochastic gradient descent (SGD), one of the most fundamental optimization algorithms in machine learning (ML), can be recast through a continuous-time approximation as a Fokker-Planck equation for Langevin dynamics, a viewpoint that has motivated many theoretical studies. Within this framework, we study the relationship between the quasi-stationary distribution derived from this equation and the initial distribution through the Kullback-Leibler (KL) divergence. As the quasi-steady-state distribution depends on the expected cost function, the KL divergence eventually reveals the connection between the expected cost function and the initialization distribution. By applying this to deep neural network models (DNNs), we can express the bounds of the expected loss function explicitly in terms of the initialization parameters. Then, by minimizing this bound, we obtain an optimal condition of the initialization variance in the Gaussian case. This result provides a concrete mathematical criterion, rather than a heuristic approach, to select the scale of weight initialization in DNNs. In addition, we experimentally confirm our theoretical results by using the classical SGD to train fully connected neural networks on the MNIST and Fashion-MNIST datasets. The result shows that if the variance of the initialization distribution satisfies our theoretical optimal condition, then the corresponding DNN model always achieves lower final training loss and higher test accuracy than the conventional He-normal initialization. Our work thus supplies a mathematically grounded indicator that guides the choice of initialization variance and clarifies its physical meaning of the dynamics of parameters in DNNs.

Hiroshi Horii, Sothea Has

SU · KHM

Fokker–Planck Equation Neural Network Initialization Neural Network Training Dynamics Stochastic Gradient Descent