Stochastic Interpolants

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 25

Sep 29, 2026cs.LG

Grokking through the Lens of Minimum-Norm Interpolation

Grokking shows that fitting the training data and learning the underlying signal can occur at very different stages. However, existing theories offer limited quantitative insight into how this delayed generalization depends on inductive bias and signal structure. Our work addresses the gap by developing a statistical theory that characterizes how regularization geometry and signal sparsity govern generalization near interpolation. In particular, we focus on the prototypical setting of high-dimensional regression and identify regimes in which sparsity-promoting regularization makes exact interpolation much more accurate than approximate fitting. In strongly overparameterized noiseless problems, we prove a zero--one generalization law and construct a family of convex norms whose interpolators transition from the trivial risk of the all-zero predictor to exact recovery, while keeping the training error equal to

0

. Furthermore, when feature dimension and sample size are proportional, we provide a precise characterization of training and generalization errors along

\ell_r

-regularization paths. This in turn allows us to quantify the generalization gain that remains near interpolation: we show that this gain increases as the norm becomes more sparsity-promoting and as the target becomes sparser, with a sharp drop in generalization reached for noiseless data and

\ell_1

regularization. Experiments on diagonal linear networks and transformers trained on modular arithmetic demonstrate the generality of our theoretical predictions. Finally, beyond grokking, our work reveals a statistical instability in minimum-norm interpolation: small perturbations in the regularization strength can lead to drastically different generalization, while preserving small training error.

Gil Kur, Ileana Rugina, Clémentine Carla Juliette Dominé +1

ETH Zürich · Institute of Science and Technology Austria · Harvard University

Stochastic Interpolants Interpolation Grokking Exact Lens

Sep 23, 2026cs.LG

Minimal-Norm Univariate Two-Layer ReLU Classification: Exact Solutions and Global Optimality with Skip Connections

We study minimal-norm interpolation and

\ell_2

-regularized logistic-loss minimization for binary classification by univariate two-layer ReLU networks. We give complete geometric characterizations of the optimal classifiers in function space, resolving how the solutions depend on whether hidden-layer biases are included in the parameter norm. When biases are unpenalized, the minimal-norm interpolators are exactly the continuous piecewise-affine functions that hug every label switch and have kinks of the appropriate convexity. When biases are penalized, the minimizer is unique in function space, has exactly one kink in each intermediate same-label segment, and is therefore a sparsest positive-margin classifier. We further show that adding a free affine skip connection leaves these function-space solutions unchanged but fundamentally improves the parameter-space landscape: every KKT point of the constrained problem becomes globally optimal, whereas suboptimal KKT points can occur without the skip connection. We establish analogous global-optimality and geometric results for sufficiently weak

\ell_2

-regularization of the logistic loss. In the unpenalized-bias case, we identify an additional sparsity-like restriction, implying that most minimal-norm interpolators cannot arise as small-regularization limits of margin-normalized logistic-loss minimizers. Numerical experiments across varying dataset complexity and network width support the predicted landscape and sparsity phenomena.

Karolina Drabik, Ben Lewis, Antoni Puch +4

Department of Mathematics, Informatics and Mechanics, University of Warsaw, Poland · Department of Computer Science, University of Warwick, UK · INRIA & LMO, Université Paris-Saclay, France

Rectified Linear Unit Networks Regularization Stochastic Interpolants Skip Optimality Exact

Sep 17, 2026cs.CV

FlowSGS: Improving Flow Matching Priors for Inverse Imaging with Stochastic Interpolants

Flow matching has emerged as the state-of-the-art generative model and has been used for plug-and-play (PnP) priors to solve inverse problems in computational imaging. However, existing flow-based inverse solvers assume linear forward models and/or make simplifying approximations in posterior sampling. To circumvent these problems, we introduce FlowSGS, a flow-based posterior sampling method using Split Gibbs Sampling (SGS) to decompose the posterior into a likelihood step and a prior step. Specifically, we sample from the likelihood step using Langevin dynamics and leverage the Stochastic Interpolants (SI) framework to integrate a pretrained flow model into the prior step. We provide a form for the prior step that uses SI's reverse-time SDE, and show connections to previous PnP methods. Moreover, with the aid of the flow prior's straight probability paths and a novel timestep correction technique for the reverse-time SDE, FlowSGS requires fewer network evaluations in its prior step than plug-and-play diffusion samplers. Our experiments show state-of-the-art performance on a range of inverse problems. For the first time, we provide an experiment on a nonlinear inverse problem (Fourier phase retrieval) for flow-based inverse solvers.

Tianao Li, Xinhui Qian, Emma Alexander

Department of Computer Science, Northwestern University · NSF-Simons AI Institute for the Sky (SkAI) · Department of Statistics and Data Science, Northwestern University

Bayesian Inverse Problems Posterior Sampling Stochastic Interpolants

Sep 15, 2026cs.LG

A Weighted Kernel Method for Approximation that Adapts to Learned Multivariable Structure

Approximating the input-output behavior of a multivariable black-box function from limited data is challenging when blind to the importance of its inputs and their interactions. We introduce total sensitivity kernels (TSKs), a method based on families of weighted ANOVA kernels that learn and adapt to this multivariable structure. TSKs parameterize the weights on each multivariable component of the target function by factors for each input. We propose learning these factors directly from function evaluations by selecting the reproducing kernel Hilbert space (RKHS) in which the target function has minimum norm. Under suitable conditions, we show that this norm-minimization problem admits a unique solution, and we establish consistency of a finite-data formulation based on minimum-norm interpolation. The learned TSK factors characterize the participation of individual inputs across interactions and main effects, providing a kernel-dependent notion of input sensitivity related to total Sobol indices. Numerical experiments demonstrate that adapting the kernel to learned multivariable structure can substantially improve approximation accuracy over a standard product kernel.

John E. Darges, Laura Weidensager

Department of Mathematics, Emory University · Department of Mathematics, Simon Fraser University

Kernel Method Approximation Stochastic Interpolants Black Box

Aug 7, 2026math.PR

Limit Points of Reflow with Minibatch Optimal Transport

Rectified flows, also called flow matching or stochastic interpolants, are generative models that learn a time-dependent vector field steering a probability curve between two probability distributions, usually referred to as latent and target distributions. Reflow accelerates inference by iteratively straightening the trajectories induced by this vector field. We study the asymptotic behavior of this iteration and characterize its limit points. First, we define weak rectified couplings which always exist. Next, when rectified flow updates are alternated with minibatch optimal transport steps of fixed batch size, we show that any limit is

N

-cyclically monotone, where

N

is the batch size. Such

N

-cyclically monotone couplings enjoy favorable structural and stability properties such as rectifiability and straightness. Finally, restricting velocities to gradient fields and assuming additional support conditions, we prove that reflow limits coincide with the optimal transport map between the endpoint distributions.

Antonin Chambolle, Johannes Hertrich

Université Paris Dauphine - PSL & Inria · ENS Paris

Rectified Flow Stochastic Interpolants Transport

Aug 4, 2026cs.LG

Benign interpolation and Occam's razor

Contemporary deep learning methods generalize well even when they fit their training data perfectly, a phenomenon known as benign interpolation. This phenomenon cannot be accounted for by classical statistical learning theory and has prompted a range of attempted new explanations in the statistics and machine learning literature. A common feature of these new proposals is an appeal to a simplicity preference among interpolating models, often presented as a form of Occam's razor. We clarify this debate for a philosophical audience and argue that this new appeal to simplicity creates an explanatory gap. The classical theory offers theorems which connect the simplicity of model classes to good generalization, thus underwriting methodological simplicity norms. The new accounts instead appeal to properties of individual models, which they interpret as a kind of simplicity. Lacking a provable connection to generalization, it is the name "simplicity" that does the work a theorem used to do, making a substantive and unargued assumption look like the application of a familiar methodological principle.

Tom F. Sterkenburg, Daniel A. Herrmann, Jan-Willem Romeijn

Stochastic Interpolants Interpretability Interpolation Benign Methodology

Jul 18, 2026stat.ML

Semi-Supervised Conditional Generative Learning through Stochastic Interpolation and Sufficient Representations

Conditional generative modeling remains a challenging problem in semi-supervised settings where labeled data is scarce but unlabeled samples are abundant. To effectively leverage structural information embedded within the unlabeled dataset and compensate for sparse conditioning signals, we propose a semi-supervised framework combining conditional stochastic interpolation with low-dimensional latent representations. RepG decomposes generation into two stages: label-dependent latent sampling and high-dimensional reconstruction. This isolates the supervised learning of conditional dependencies to a low-dimensional space, requiring few labels while utilizing the abundant unlabeled data purely for reconstruction. Theoretically, we establish an error decomposition showing that the Kullback-Leibler divergence of RepG comprises stage-wise estimation errors and a structural bias quantified by conditional mutual information. For deep neural network estimators, we derive non-asymptotic convergence rates proving that RepG significantly improves sample complexity. By confining the supervised estimation burden to the low intrinsic dimension of the latent representation, RepG achieves a strictly faster convergence rate. Complemented by a minimax lower bound, our theoretical results demonstrate that this method effectively mitigates the curse of dimensionality inherent in direct ambient-space generative modeling.

Changyu Liu, Yuling Jiao, Jian Huang

Institute for Mathematics and Artificial Intelligence, Wuhan University, Wuhan, China. · School of Artificial Intelligence, Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan, China. · Departments of Data Science and Artificial Intelligence, and Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China.

Semi-Supervised Learning Generative Architectures Stochastic Interpolants Generative Models

Jun 7, 2026cs.LG

A Theoretical Analysis of Memory and Overfitting Phenomena in Stochastic Interpolation Models

This paper provides a theoretical account of memorization in stochastic interpolation models. By leveraging closed-form expressions for the optimal velocity field and the associated score function, we show that, in the continuous-time oracle setting, both deterministic and stochastic generation processes recover training samples. Under Euler discretization, generated samples remain centered around training samples, with deviations controlled by the step size. We further analyze generation in the presence of estimation errors and show that accumulated estimation errors control the endpoint deviation from the training set. These results imply that the generated sample admits a representation as a training sample perturbed by three controlled terms: a discretization-induced bound, an estimation-error-induced bound, and stochastic Gaussian noise. Based on this characterization, we provide theoretical definitions of overfitting and underfitting in generative models. Synthetic simulations support our theoretical findings.

Yunchen Li, Shaohui Lin, Zhou Yu

East China Normal University

Stochastic Interpolants Overfitting Generative Models Memorization Time Discretization Training Data+1

Jun 5, 2026cs.LG

Second-Order Path Kernel Interpolation Formulas in Machine Learning

Understanding how training data shape neural network predictions is a central problem in modern learning theory. In 2020, Pedro Domingos proposed an interpolation formula valid for every model learned by deterministic gradient descent. It expresses the model's prediction as an integral, along the optimization path, of a data-dependent kernel that aligns the model's gradients at the test and training data. Such a first-order characterization remains valid for models trained with batch-based stochastic optimization. In this paper, we develop second-order forms of these interpolation formulas. We show that the leading path-kernel interpolation is supplemented by a curvature-weighted interpolation term. For stochastic gradient descent, an additional sampling-induced component appears, coupling the curvature of the prediction with the covariance of mini-batch gradient noise. We also extend the representation to stochastic gradient descent with momentum, where the interpolation structure is preserved but with the weights modified by a memory-related factor. Moreover, we establish a concentration estimate for the terminal prediction, identifying the fluctuation scale around the expected second-order representation. Together, these results provide a refinement of the path-kernel interpretation of neural network prediction.

Jin Guo, Roy Y. He, Jean-Michel Morel

Department of Mathematics, City University of Hong Kong, Hong Kong · Division of Industrial Data Science, Lingnan University, Hong Kong

Stochastic Interpolants Stochastic Gradient Descent Interpolation

Jun 5, 2026cs.LG

Covariance Shrinkage via Stochastic Interpolation

We recast classical shrinkage of high-dimensional covariance estimators as empirical risk minimization over a parametric stochastic interpolant between a source and a target distribution. This formalism recovers known shrinkage estimators as special cases and reveals three distinct mechanisms for reducing statistical risk: (i) Scheduling: the interpolant schedule determines the class of admissible covariances, and hence the achievable risk. (ii) Flow maps and couplings: whereas naive constructions amount to assuming independence between the distributions, specific coupling structures (e.g., solutions of optimal transport problems) can lower the empirical risk. Moreover, non-linear flow maps realizing such couplings free the interpolant covariance from the eigenbasis of the empirical estimate, enabling eigenvector regularization. (iii) Early stopping: estimators defined by integrating a regressed vector field afford an additional bias-variance trade-off through approximation of the true interpolant distribution. We then propose a neural estimator of the interpolant, together with an upper bound on its quadratic risk in terms of the interpolant approximation error, and validate both on synthetic experiments. Finally, we apply the estimator to real neuroimaging data, demonstrating the additional regularization power this approach offers in practice.

Mathieu Chalvidal, Florentin Coeurdoux, Eric Vanden-Eijnden

Capital Fund Management 23 Rue de l’Universit´e, 75007 Paris

Covariance Stochastic Interpolants Empirical Risk Minimization Regularization Stochastic

Jun 4, 2026math.ST

How abundant are good interpolators?

Let

S

be the set of unit norm linear classifiers

θ\in \mathbb{R}^d

which correctly classify every point of a labeled dataset

(X_i,y_i)_{i=1}^n

,

X_i \in \mathbb{R}^d

,

y_i \in \{-1,+1\}

, with a possibly negative margin

κ

fixed in advance. Under two natural data-generating distributions of the

(X,y)

pairs -- a Gaussian mixture model and a logistic model with Gaussian features -- and in the proportional regime

n/d \to α

with small enough

α

, we establish a large deviation principle on the event that a point

θ

chosen uniformly at random from

S

achieves a given generalization error, with high probability over the choice of the data. The associated large deviation rate function is deterministic and describes the proportion, at the exponential scale in

d

, of interpolating classifiers having a given desired performance. As a consequence, we establish the following concentration phenomenon: all but an exponentially small fraction of interpolating classifiers have approximately the same generalization performance given by the unique maximizer of this rate function. We numerically compare this maximizer to the performance of empirical risk minimization by gradient descent and to the performance of a natural linear program, both finding a point in

S

, and deduce that in the overparametrized regime of small

α

, these efficient procedures outperform the vast majority of interpolators, pointing to their nontrivial benign overfitting in this setting.

August Y. Chen, Ahmed El Alaoui

Cornell University, Department of Computer Science. Ithaca, NY, USA. · Cornell University, Department of Statistics and Data Science. Ithaca, NY, USA.

Stochastic Interpolants Overparameterization Empirical Risk Minimization Interpolation Large Deviation Principle Gaussian Mixture Models

Jun 2, 2026math.OC

Nonlocal Mean Field Schrödinger Bridge with Learned Interactions

The Schrödinger Bridge Problem connects an initial distribution to a terminal one along a minimum-energy stochastic process. Its mean-field extension, the Mean-Field Schrödinger Bridge, governs interacting populations whose dynamics and costs depend on the collective distribution. When these interactions are nonlocal, their direct evaluation scales quadratically with the population size, making large ensembles intractable within FBSDE-based solvers. We replace these terms with neural surrogates in state and time, trained on empirical interaction values along sampled trajectories and embedded in a four-stage alternating scheme that updates the forward and backward potentials and the surrogates in turn, while preserving forward--backward consistency and the prescribed endpoint marginals. We derive Grönwall-type stability bounds quantifying how surrogate errors propagate to the generated trajectories under a small-gain condition. On crowd-navigation and high-dimensional opinion-dynamics benchmarks, the surrogates reproduce the trajectories obtained with exact evaluation at reduced training cost. The advantage is most significant when the interaction is a nonlinear functional of the measure, such as the normalized bounded-confidence drift, for which random-batch subsampling is biased and unstable whereas the learned surrogate remains accurate.

Daisuke Inoue, Dante Kalise, Mathieu Laurière

Department of Mathematics, Imperial College London, London SW7 2AZ, United Kingdom. · Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning, NYU-ECNU Institute of Mathematical Sciences, NYU Shanghai, Shanghai 200126, People’s Republic of China.

Schrödinger Bridges Mean-Field Limit Stochastic Processes Stochastic Interpolants Bridge

Jun 1, 2026cs.LG

Fast Generalization after Interpolation via Critically Damped Momentum Optimization

A central problem in machine learning is that models can achieve near-perfect training performance while generalizing substantially less well to unseen examples. This gap is especially acute in high-dimensional, low-sample regimes, where many interpolating solutions exist and optimization must implicitly select among minima with different generalization properties. Following recent theoretical advances on optimization dynamics near the interpolation threshold, we note that the two-regime structure of risk minimization, with loss minimization followed by complexity minimization, motivates a biphasic optimization schedule. We thus theoretically demonstrate that GROKtimizer, a biphasic strategy that combines rapid convergence to interpolation with Critically Damped Momentum (CDM)-based post-interpolation norm minimization, offers a natural solution for selecting low-norm interpolating solutions. Under a local quadratic model of the post-interpolation basin, GROKtimizer provides a quadratic speedup over classical gradient descent, with provable optimality among first-order optimizers. To showcase the applicability of our method, we evaluate GROKtimizer on several synthetic benchmarks common in the classical grokking literature and on various real-world datasets. Finally, we reconcile our findings with the flat-minima hypothesis, highlighting the importance of post-interpolation dynamics in the construction of high-quality, generalizing models.

Luca Muscarnera, Silas Ruhrberg Estévez, Yuanzhang Xiao +1

Stochastic Interpolants Gradient Descent Regularization Interpolation Grokking Momentum

May 28, 2026cs.LG

Midpoint Generative Models

We introduce Midpoint Generative Models (MGM), a principled framework for training one-step generative models. MGM is based on a simple symmetry of Flow Matching with linear interpolation: when the two endpoint distributions coincide, the corresponding drift field vanishes at the midpoint time,

t=1/2

. We show that the norm of this field defines a valid discrepancy between distributions, which we call the Midpoint Divergence. We extend this discrepancy beyond the midpoint by introducing randomly flipped interpolations and further generalize it by replacing deterministic linear Flow Matching interpolations with symmetric stochastic interpolants, yielding a generalized Midpoint Divergence. Finally, we derive a variational formulation of our generalized divergence, yielding a tractable objective for training a one-step generator. The resulting MGM algorithm offers an effective and theoretically grounded approach to generative modeling, achieving competitive performance against existing one-step generative modeling methods.

Daniil Shlenskii, Nikita Gushchin, Lev Novitskiy +2

1AXXX, Russia · 2Applied AI Institute, Russia · 3Kandinsky Lab, Russia

Generative Models Stochastic Interpolants Conditional Flow Matching Medium Divergence

May 28, 2026cs.LG

The Interplay Between Interpolation and Aggregation in Regression: Optimal Sample Complexity

This work investigates theoretically the interplay between interpolation and aggregation in regression. We establish that the

γ

-graph dimension characterizes learnability for a broad class of natural aggregation procedures. Furthermore, we prove that an extremely simple aggregation procedure, combining three interpolating hypotheses via the median, is optimal among all these aggregation procedures, and is strictly more powerful than proper learning. Finally, we show that some hypothesis classes are learnable only by aggregating infinitely many hypotheses or by using non-interpolating aggregation rules (which may predict outside the range of their inputs), and any finite interpolating aggregation fails to achieve even trivial performance.

Mikael Møller Høgsgaard, Kasper Green Larsen, Liang-Yu Zou

Department of Computer Science, Aarhus University, Aarhus, Denmark · Department of Statistics, University of Oxford, Oxford, United Kingdom

Stochastic Interpolants Learnability Optimal Sample Complexity Hypothesis Class Interpolation

May 26, 2026cs.LG

Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias

The successful training of neural networks hinges on the use of first order optimization methods, yet the theoretical characterization of these methods remains incomplete. This is especially true in settings with mild overparameterization. In this work, we study the gradient flow dynamics of two-layer ReLU networks from small initialization with orthogonal training data. We prove the limiting flow converges to a saddle-to-saddle jump process as the initialization scale tends to zero, revealing an incremental learning phenomenon in which a new neuron activates at each saddle. This analysis recovers the known result of Dana et al. (2025, arXiv:2502.16977) that the network interpolates the training data with high probability as soon as

m \gtrsim \log(n)

, where

m

is the network width and

n

is the number of training samples. This incremental process characterization also allows us to derive a novel implicit bias result: the learned interpolator has a squared

\ell_2

-norm scaling as

\sqrt{n}

, which is within a constant factor of the minimal

\ell_2

-norm interpolator. More broadly, our work provides the first rigorous proof of an incremental learning process for ReLU networks, whilst suggesting mildly overparameterized networks can converge to interpolating solutions whose complexity is of the same order as that of the optimal interpolator.

James Town, Etienne Boursier, Ben Lewis +2

University of Warwick · INRIA LMO, Université Paris-Saclay

Rectified Linear Unit Networks Stochastic Interpolants Class-Incremental Learning Orthogonality

May 26, 2026cs.LG

Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences

Learning an energy-based model from data samples is a central problem in machine learning. Many recent and popular methods, such as denoising score matching for training energy-based diffusion models, use stochastic interpolants to corrupt data samples at different noise levels indexed by a time variable. This defines a joint density over both the data space and time, and most methods learn its energy through either spatial or temporal differences. We identify distinct failure modes for both of these approaches. To solve them, we propose Spatiotemporal Noise-Contrastive Estimation (stNCE), a framework for learning the energy through joint spatiotemporal differences. stNCE unifies many existing methods and leads to new training objectives. Experiments on images and molecules demonstrate performance competitive with state-of-the-art density estimation methods.

Hanlin Yu, RuiKang OuYang, Partha Kaushik +3

University of Helsinki · University of Cambridge · Carnegie Mellon University +1

Diffusion Models Contrastive Learning Stochastic Interpolants Spatiotemporal

May 20, 2026cs.CV

Disentangling Generation and Regression in Stochastic Interpolants for Controllable Image Restoration

Recent advances in Image Restoration (IR) have been largely driven by generative methods such as Diffusion Models and Flow Matching, which excel in synthesizing realistic textures while suffering from slow multi-step inference and compromised pixel fidelity. In contrast, classical regression-based IR methods excel precisely in these aspects, offering single-step efficiency and high pixel-level reconstruction fidelity. To bridge this gap, we propose DiSI, a unified framework that Disentangles the underlying Stochastic Interpolant process into independent generation and regression components. This decoupling endows DiSI with remarkable versatility, enabling a continuous and controllable transition from a pure regression process to a fully generative one. Technically, we instantiate this framework with two specific sampling trajectories, accompanied by a unified sampler for high-quality, few-step inference on arbitrary trajectories. Furthermore, we design a dual-branch U-Net style transformer network in pixel space, using a dedicated branch to enhance conditional guidance while ensuring high throughput. Extensive experiments demonstrate that DiSI efficiently achieves competitive results on various IR tasks, while uniquely offering the inference-time flexibility to control the distortion-perception trade-off within a single model.

Yi Liu, Jia Ma, Wengen Li +3

Tongji University · Fudan University

Image Restoration Stochastic Interpolants U-Net

May 9, 2026cs.CV

Supersampling Stable Diffusion and Beyond: A Seamless, Training-Free Approach for Scaling Neural Networks Using Common Interpolation Methods

Stable Diffusion (SD) has evolved DDPM (Denoising Diffusion Probabilistic Model) based image generation significantly by denoising in latent space instead of feature space. This popularized DDPM-based image generation as the cost and compute barrier was significantly lowered. However, these models could only generate fixed-resolution images according to their training configuration. When we attempt to generate higher resolutions, the resulting images show object duplication artifacts consistently. To solve this problem without finetuning SD models, recent works have tried dilating the convolution kernels of the models and have achieved a great level of success. But dilated kernels are harder to fine-tune due to being zero-gapped. Apart from this, other methods, such as patched diffusion, could not solve the object-duplication problem efficiently. Hence, to overcome the limitations of dilated convolutions, we propose kernel interpolation of SD models for higher-resolution image generation. In this work, we show mathematically that interpolation can correctly scale convolution kernels if multiplied by a constant coefficient and achieve competitive empirical results in generating beyond-training-resolution images with Stable Diffusion using zero training. Furthermore, we demonstrate that our method enables interpolation of deep neural networks to adapt to higher-dimensional training data, with a worst-case performance drop of

2.6\%

in accuracy and F1-Score relative to the baseline. This shows the applicability of our method to be general, where we interpolate fully-connected layers, going beyond convolution layers. We also discuss how we can reduce the memory footprints of training neural networks, using our method up to at least

4\times

.

Md Abu Obaida Zishan, Jannatun Noor, Annajiat Alim Rasel

School of Data and Sciences BRAC University, Dhaka · Computing for Sustainability and Social Good (C2SG) Research Group Department of Computer Science and Engineering United Internation University, Dhaka

Denoising Diffusion Probabilistic Model Stochastic Interpolants Training-Free Neural Network

May 8, 2026cs.LG

Neural Operators as Efficient Function Interpolators

Neural operators (NOs) are designed to learn maps between infinite-dimensional function spaces. We propose a novel reframing of their use. By introducing an auxiliary base-space, any finite-dimensional function can be viewed as an operator acting by composition on functions of the base-space. Through a range of benchmarks on analytic functions of increasing complexity and dimensionality, we demonstrate that NOs can match or outperform standard multilayer perceptrons and Kolmogorov--Arnold Networks in accuracy while requiring significantly fewer parameters and training time. As a real-world application, we apply a two-dimensional Tensorized Fourier Neural Operator (TFNO) to the nuclear chart, learning a correction to state-of-the-art nuclear mass models as a partially observed residual field. A TFNO ensemble reaches a held-out root-mean-square error of 198.2 keV, placing it among the best recent neural-network approaches while retaining high parameter efficiency and short training times. More broadly, these results introduce NOs as a scalable framework for finite-dimensional function interpolation, from analytic benchmarks to structured scientific data.

Vasilis Niarchos, Angelos Sirbu, Sokratis Trifinopoulos

Department of Physics, CCTP and ITCP, University of Crete, 71303, Greece · Theoretical Physics Department, CERN, Geneva, Switzerland · Physik-Institut, Universität Zürich, 8057 Zürich, Switzerland

Fourier Neural Operators Neural Operators Nuclear Data Stochastic Interpolants Kolmogorov-Arnold Networks Interpolation

May 7, 2026eess.AS

Predictive-Generative Drift Decomposition for Speech Enhancement and Separation

We propose a plug-and-play framework for speech enhancement and separation that augments predictive methods with a generative speech prior. Our approach, termed Stochastic Interpolant Prior for Speech (SIPS), builds on stochastic interpolants and leverages their flexibility to bridge predictive and generative modeling. Specifically, we decompose the interpolation dynamics into a task-specific drift and a stochastic denoising component, allowing a predictive estimate to be integrated directly into the generative sampling process. This results in a mathematically grounded framework for combining strong pretrained predictors with the expressive power of generative models. To this end, we train a score model using only clean speech, yielding a degradation-agnostic prior that can be reused across tasks. During inference, the predictor provides a deterministic drift that steers the sampling process toward a task-consistent estimate, while the score model preserves perceptual naturalness. Unlike prior hybrid approaches, which typically rely on architecture-specific conditioning and are tied to particular predictors or degradation settings, SIPS provides a unified framework that generalizes across predictors and additive degradation tasks. We demonstrate its effectiveness for both speech enhancement and speech separation using recent predictors such as SEMamba and FlexIO. The proposed method consistently improves perceptual quality, achieving gains up +1.0 NISQA for speech separation.

Julius Richter, Yoshiki Masuyama, Christoph Boeddeker +3

MERL Cambridge, MA, USA

Speech Enhancement Speech Separation Speech Synthesis Separation Stochastic Interpolants Denoising Diffusion Probabilistic Model

Apr 28, 2026stat.ML

Adversarial Robustness of NTK Neural Networks

Deep learning models are widely deployed in safety-critical domains, but remain vulnerable to adversarial attacks. In this paper, we study the adversarial robustness of NTK neural networks in the context of nonparametric regression. We establish minimax optimal rates for adversarial regression in Sobolev spaces and then show that NTK neural networks, trained via gradient flow with early stopping, can achieve this optimal rate. However, in the overfitting regime, we prove that the minimum norm interpolant is vulnerable to adversarial perturbations.

Yuxuan Hou

1Qiuzhen College, Tsinghua University

Neural Tangent Kernel Neural Network Stochastic Interpolants Gradient Exponential Family

Feb 23, 2026cs.LG

Generative Modeling via Kernelized Stochastic Interpolants

We develop a kernel method for generative modeling within the stochastic interpolant framework, replacing neural network training with linear systems. The drift of the generative SDE is

\hat b_t(x) = \nablaφ(x)^\topη_t

, where

η_t \in \mathbb{R}^P

solves a

P\times P

system computable from data, with

P

independent of the data dimension

d

. Since estimates are inexact, the diffusion coefficient

D_t

affects sample quality; the optimal

D_t^*

from Girsanov diverges at

t=0

, but this poses no difficulty and we develop an integrator that handles it seamlessly. The framework accommodates diverse feature maps: scattering transforms, pretrained generative models, etc, enabling generation and model combination without neural network training. We demonstrate the approach on financial time series, turbulence, and image generation.

Florentin Coeurdoux, Etienne Lempereur, Nathanaël Cuvelle-Magar +2

Capital Fund Management, Paris, France · Département d’informatique, ENS, Université PSL, Paris, France · Collège de France, Paris, France +2

Stochastic Interpolants Stochastic Differential Equations Generative Models Kernel Method

Oct 1, 2025cs.LG

Multi-Marginal Flow Matching with Adversarially Learnt Interpolants

Learning the dynamics of a process given sampled observations at several time points is an important but difficult task in many scientific applications. When no ground-truth trajectories are available, but one has only snapshots of data taken at discrete time steps, the problem of modelling the dynamics, and thus inferring the underlying trajectories, can be solved by multi-marginal generalisations of flow matching algorithms. This paper proposes a novel flow matching method that overcomes the limitations of existing multi-marginal trajectory inference algorithms. Our proposed method, ALI-CFM, uses a GAN-inspired adversarial loss to fit neurally parametrised interpolant curves between source and target points such that the marginal distributions at intermediate time points are close to the observed distributions. The resulting interpolants are smooth trajectories that, as we show, are unique under mild assumptions. These interpolants are subsequently marginalised by a flow matching algorithm, yielding a trained vector field for the underlying dynamics. We showcase the versatility and scalability of our method by outperforming the existing baselines on spatial transcriptomics and cell tracking datasets, while performing on par with them on single-cell trajectory prediction. Code: https://github.com/mmacosha/adversarially-learned-interpolants.

Oskar Kviman, Kirill Tamogashev, Nicola Branchini +3

KTH · University of Edinburgh

Conditional Flow Matching Interpolation Stochastic Interpolants

Jun 20, 2024math.ST

Generalization error of min-norm interpolators in transfer learning

This paper establishes the generalization error of pooled min-

\ell_2

-norm interpolation in transfer learning, where data from diverse distributions are available. Min-norm interpolators arise naturally as implicit regularized limits of modern machine learning algorithms. Prior work has characterized their out-of-distribution risk when samples from the test distribution are unavailable during training. In many applications, however, limited test samples may be available at training time, yet properties of min-norm interpolation in this regime remain poorly understood. We address this gap by characterizing the bias and variance of pooled min-

\ell_2

-norm interpolation under both covariate shift and model shift. Our results yield several important implications. In certain cases under model shift, we show that adding data always hurts when the signal-to-noise ratio (SNR) is low. At higher SNR levels, transfer learning is beneficial provided the shift-to-signal ratio falls below a threshold that we characterize explicitly. Under covariate shift, we find that when the source sample size is small relative to the dimension, greater heterogeneity between domains reduces risk, and vice versa. While our model shift results are initially established for Gaussian designs, we extend them to more general designs through a universality argument. To illustrate the broader applicability of our technical tools beyond interpolation learning, we characterize the risk of a bias-corrected estimator that uses the pooled interpolator as an initialization and corrects the resulting bias with target data. On the technical side, we develop a novel anisotropic local law and a Lindeberg-swapping argument, yielding tools that may be of independent interest in random matrix theory and universality analysis. Finally, we supplement our theory with simulations demonstrating the finite-sample efficacy of our results.

Yanke Song, Kenneth Gu, Sohom Bhattacharya +1

Department of Statistics, Harvard University · Department of Statistics, Stanford University · Department of Statistics, University of Florida

Stochastic Interpolants Empirical Risk Minimization Transfer Learning Covariate Shift Finite-Sample