Stochastic Differential Equations

Sampling in score-based diffusion models can be performed by solving either a reverse-time stochastic differential equation (SDE) parameterized by an arbitrary stochasticity function or a probability flow ODE, corresponding to setting this stochasticity function to zero. In this work, we investigate the effect of this stochasticity on the generation process through the evolution of Kullback-Leibler (KL) divergences, obtaining general KL divergence bounds and a novel analysis of the impact of the time-profile of the score error on model performance. For exact score functions, stochasticity has a contractive effect, decreasing KL divergence along the sampling trajectory. For approximate scores, however, a trade-off arises between correcting accumulated errors and amplifying current score errors, meaning stochasticity can either improve or degrade generation performance. Theoretical considerations indicate that the gain from stochasticity depends on the time-localization of the trained model error. We test this in experiments on both toy and benchmark data sets, also comparing the KL divergence evolution with the obtained bounds. We also present a fully analytical example, where all the relevant quantities can be computed, and the optimal stochasticity function can be characterized via an optimal control analysis.

We propose a framework for generative modeling of continuous-time processes from irregularly and asynchronously recorded data. It is based on the matching of generators and accommodates discontinuous trajectories. Analytical formulas for diffusion and jump bridges yield a family of reference generators that a neural network is trained to match. The key ingredient is that, for our constructed jump bridge, a parametrization of the jump kernel densities by scaled Gaussians admits closed-form expressions for the Kullback-Leibler divergence, allowing simulation-free training.

We derive first-order Pontryagin optimality conditions for stochastic optimal control with deterministic controls for systems modeled by rough differential equations (RDE) driven by Gaussian rough paths. This Pontryagin Maximum Principle (PMP) applies to systems following stochastic differential equations (SDE) driven by Brownian motion, yet it does not rely on forward-backward SDEs and involves the same Hamiltonian as the deterministic PMP. The proof consists of first deriving various integrable error bounds for solutions to nonlinear and linear RDEs by leveraging recent results on Gaussian rough paths. The PMP then follows using standard techniques based on needle-like variations. As an application, we propose the first indirect shooting method for nonlinear stochastic optimal control and show that it converges 10x faster than a direct method on a stabilization task.

Motivated by dynamic risk measures and conditional $g$-expectations, in this work we propose a numerical method to approximate the solution operator given by a Backward Stochastic Differential Equation (BSDE). The main ingredients for this are the Wiener chaos decomposition and the classical Euler scheme for BSDEs. We show convergence of this scheme under very mild assumptions, and provide a rate of convergence in more restrictive cases. We then implement it using neural networks, and we present several numerical examples where we can check the accuracy of the method.

Neural operator (NO) architectures learn nonlinear maps between infinite-dimensional function spaces and are widely used to accelerate simulation and enable data-driven model discovery. While universality results ensure expressivity, they do not address complexity: for broad operator classes described only through regularity (e.g.\ uniform continuity or $C^r$-regularity), information-theoretic lower bounds imply that minimax-optimal NO approximation rates scale exponentially in the reciprocal accuracy $1/\varepsilon$. This has shifted the focus of NO theory toward identifying additional problem-specific structure, beyond regularity, under which suitably tailored NO architectures can leverage to unlock polynomial scaling in $1/\varepsilon$. We exhibit the first polynomial-scaling regime for NO approximations of solution operators in stochastic analysis; by identifying structured families of non-Markovian BSDEs with randomized terminal condition parameterized by the Sobolev-regular terminal condition and by Sobolev-regular additive nonlinear perturbations of the generator. We prove that their solution operator can be approximated (uniformly over the family) by a tailored NO whose number of trainable parameters grows polynomially in $1/\varepsilon$. We unlock this polynomial scaling regime by informing the NO's inductive bias by factoring out the singular part of the associated semilinear elliptic PDE Green's function and by incorporating the Doléans--Dade exponential of the BSDE's common non-Markovian factor into the NO's decoding layers. As a byproduct, we extend polynomial-scaling guarantees from families of linear elliptic PDEs on regular domains to the semilinear setting.

Classical particle methods based on propagation of chaos (PoC) have been developed for solving mean-field stochastic differential equations and their associated nonlinear Fokker--Planck equations. However, direct PoC implementations are difficult to apply to high-dimensional problems because they require simulating and storing large numbers of interacting particles, often with high particle-particle interaction costs. Motivated by these limitations, we build on the recently proposed sequential propagation of chaos (SPoC) framework, which replaces the fully interacting particle system in PoC with a sequential interaction mechanism. Based on this structure, we present DeepSPoC, a neural particle method that embeds a neural density representation into the sequential particle dynamics. DeepSPoC simulates particles batch by batch, while the neural network represents the evolving empirical law and is substituted into the coefficients of the mean-field SDE, thereby replacing direct particle-particle interactions with particle-network interactions. In DeepSPoC, a recently developed normalizing flow model called KRnet is used to approximate the empirical measure of particles. Compared with direct particle implementations, DeepSPoC substantially reduces memory consumption and evaluates interaction terms more efficiently, thereby improving scalability for high-dimensional problems. We apply DeepSPoC to a wide range of mean-field equations and verify its effectiveness and computational advantages.

Irregular sampling intervals and missing values in real-world time series data present challenges for conventional methods that assume consistent intervals and complete data. Neural Ordinary Differential Equations (Neural ODEs) offer an alternative approach, utilizing neural networks combined with ODE solvers to learn continuous latent representations through parameterized vector fields. Neural Stochastic Differential Equations (Neural SDEs) extend Neural ODEs by incorporating a diffusion term, although this addition is not trivial, particularly when addressing irregular intervals and missing values. Consequently, careful design of drift and diffusion functions is crucial for maintaining stability and enhancing performance, while incautious choices can result in adverse properties such as the absence of strong solutions, stochastic destabilization, or unstable Euler discretizations, significantly affecting Neural SDEs' performance. In this study, we propose three stable classes of Neural SDEs: Langevin-type SDE, Linear Noise SDE, and Geometric SDE. Then, we rigorously demonstrate their robustness in maintaining excellent performance under distribution shift, while effectively preventing overfitting. To assess the effectiveness of our approach, we conduct extensive experiments on four benchmark datasets for interpolation, forecasting, and classification tasks, and analyze the robustness of our methods with 30 public datasets under different missing rates. Our results demonstrate the efficacy of the proposed method in handling real-world irregular time series data.

In this article, we introduce a deep learning based approximation algorithm for SPDEs. Our approach employs neural networks to approximate the solutions of SPDEs along given realizations of the driving noise process. If applied to a set of simulated noise trajectories, it yields empirical distributions of SPDE solutions, from which functionals like the mean and variance can be estimated. We test the performance of the method on stochastic heat equations with additive and multiplicative noise as well as stochastic Black-Scholes equations with multiplicative noise and Zakai equations from nonlinear filtering theory. In all cases, the proposed algorithm yields accurate results with short runtimes in up to 100 space dimensions.