math.NANov 17, 2026

Optimal Control of RKHS Diffeomorphic Flows for Classification and Generative Modeling

Authors: Christopher Chukwuemeka, Andreas Mang

Organizations: Department of Mathematics, Tufts University, Medford, MA, USA

Abstract

We develop a unified optimal-control framework that solves supervised classification and likelihood-based density estimation with a single diffeomorphic flow whose velocity lives in a reproducing kernel Hilbert space (RKHS); the two tasks share all structure except the objective. Variable projection eliminates the terminal model (a classifier, or a Gaussian-mixture (GMM) reference), leaving a reduced control problem whose gradient comes from an analytic discrete adjoint. The RKHS parameterization makes the flow's divergence exact and deterministic at the cost of one kernel matrix-vector product, avoiding the stochastic trace estimators and potential-flow restrictions of existing continuous normalizing flows. Because the objective and gradient are deterministic, matrix-free reduced Gauss-Newton (GN) and related second-order solvers are practical for both tasks, which we benchmark against a range of first- and second-order optimizers; the learned map is a diffeomorphism with explicitly controlled regularity. In experiments from dimension 2 to 256, the reduced GN solvers use the fewest iterations of any optimizer tested and, together with joint L-BFGS, the least work; a multiscale kernel improves the density fit, and a trainable GMM reference reduces the deformation needed for multimodal transport at comparable likelihood while lowering the likelihood on real data. The full-model solve is quadratic in the particle count and sub-linear in dimension; a control-point reduction makes the particle-count scaling linear at near-lossless accuracy.

Figures & tables

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Apr 17, 2025

On the minimax optimality of Flow Matching through the connection to kernel density estimation

Flow Matching has recently gained attention in generative modeling as a simple and flexible alternative to diffusion models. While existing statistical guarantees adapt tools from the analysis of diffusion models, we take a different perspective by connecting Flow Matching to kernel density estimation. We first verify that the kernel density estimator matches the optimal rate of convergence in Wasserstein distance up to logarithmic factors, improving existing bounds for the Gaussian kernel. Based on this result, we prove that for sufficiently large networks, Flow Matching achieves the optimal rate up to logarithmic factors. If the target distribution lies on a lower-dimensional manifold, we show that the kernel density estimator profits from the smaller intrinsic dimension on a small tube around the manifold. The faster rate also applies to Flow Matching, providing a theoretical foundation for its empirical success in high-dimensional settings.
Nov 14, 2026

Explicit Regularization for Flow Matching

Learning the underlying probability distribution from samples is a core problem in statistics and applied mathematics. Generative models based on diffusion and flow matching learn a velocity field whose flow transports a known distribution to the data distribution. They produce novel, realistic samples, although the empirical learning problem is ill posed near the data distribution. Its minimizer transports the samples onto the training data, and small perturbations of the data change the velocity field strongly near the end of the flow. The observed generalization is often attributed to implicit regularization by the network parameterization, early stopping of the training, and the time integration during sampling. Explicit regularization has advanced the theory and computation of many inverse problems and has recently been proposed to prevent memorization in generative models. With a regularizer on the magnitude of the velocity, however, the field approaches the memorizing velocity unless the regularization parameter grows to infinity fast enough, and it then stays bounded by a constant that is small only for large parameters. We show that penalizing the derivatives of order kk of the velocity renders the DD-dimensional empirical problem well posed uniformly in time if 2k>D2k > D and the training points are in general position. Our experiments in one and two dimensions discretize the velocity with splines and use direct solvers for the regularized least-squares problems to exclude optimization error. Refining the discretization in space and time changes the error of the Hessian-regularized flows only marginally, whereas without the regularizer the error tends to that of memorization. These flows generalize, with errors comparable to or lower than those of well-tuned kernel density estimates.
Oct 15, 2026

Normalizing Flow Maps

Scalable approaches to learning neural dynamical systems have placed diffusion, flow matching, and few-step generators in flow maps at the forefront of generative modeling. Such state-of-the-art models have limited access to explicit model densities despite evolving from classical generative modeling approaches that leveraged tractable likelihoods, e.g., Normalizing Flows and Autoregressive models. In this paper, we combine these threads of few-step generation and tractable sample likelihoods and introduce Normalizing Flow Maps (NFM), a class of exactly invertible pushforward maps that generalizes normalizing flows and flow maps. More precisely, NFMs learn the integrator of an underlying dynamical system, but crucially, they provide efficient exact likelihood evaluation by construction. To enable scalable training of NFMs, we derive two complementary axes of consistency inherent to NFMs that yield (self) distillation objectives: (1) pointwise matching, which unlocks ℓ2\ell_2-based flow-map style regression losses, and (2) distributional matching, which matches pushforward distributions to the marginals of a prescribed forward process. On conditional image generation on CIFAR-10 and ImageNet-64, NFMs achieve state-of-the-art sample quality among parameter-matched normalizing flows, while supporting likelihood-based RLHF. For equilibrium sampling in Boltzmann Generators, NFMs outperform prior state-of-the-art normalizing flows on both single-system and transferable peptide benchmarks.