Identifying Drift, Diffusion, and Causal Structure from Temporal Snapshots
Authors: Vincent Guan, Joseph Janssen, Hossein Rahmani, Andrew Warren, Stephen Zhang, Elina Robeva, Geoffrey Schiebinger
Organizations: Department of Mathematics University of British Columbia, Vancouver, Canada · Department of Earth, Ocean and Atmospheric Sciences University of British Columbia, Vancouver, Canada · School of Mathematics and Statistics University of Melbourne, Melbourne, Australia
Stochastic differential equations (SDEs) are a fundamental tool for modelling dynamic processes, including gene regulatory networks (GRNs), contaminant transport, financial markets, and image generation. However, learning the underlying SDE from data is a challenging task, especially if individual trajectories are not observable. Motivated by burgeoning research in single-cell datasets, we present the first comprehensive approach for jointly identifying the drift and diffusion of an SDE from its temporal marginals. Assuming linear drift and additive diffusion, we show that non-identifiability can only arise if the initial distribution possesses generalized rotational symmetries. We further prove that even if this condition holds, the drift and diffusion can almost always be recovered from the marginals. Additionally, we show that the causal graph of any SDE with additive diffusion can be recovered from the identified SDE parameters. To complement this theory, we adapt entropy-regularized optimal transport to handle anisotropic diffusion and introduce APPEX (Alternating Projection Parameter Estimation from X0), an iterative algorithm designed to estimate the drift, diffusion, and causal graph of an additive noise SDE, solely from temporal marginals, and demonstrate its effectiveness on simulated data from linear additive noise SDEs.
Figures & tables
Figure 1: Outline of our theoretical (a) and algorithmic (b) contributions. In (b), our algorithm Alternating Projection Parameter Estimation from X0 alternates between Anisotropic Entropy-regularized Optimal Transport (AEOT) and Maximum Likelihood Estimation (MLE).
Figure 2: We visualize four distributions that satisfy auto-rotational invariance. The first three distributions obey strong auto-rotational invariance (Definition 2 ), while the Gaussian pancake is weakly auto-rotationally invariant (Definition 4 ).
Figure 3: Visualization of our parameter estimation algorithm APPEX on a toy example . Given observed temporal marginals from an underlying SDE (left), APPEX alternates between trajectory inference (middle) and MLE parameter estimation (right) in order to find the SDE parameters that best represent the temporal marginal observations.
Figure 4: The mean absolute error for estimates of A and H using APPEX is shown per iteration for all three pairs of SDEs from Section B .
d
A Estimation
GG ⊤ Estimation
WOT
APPEX
WOT
APPEX
3
0.351±0.04
0.237±0.04
0.793±0.205
0.147±0.030
4
0.730±0.067
0.328±0.041
1.549±0.439
0.415±0.070
5
0.912±0.060
0.602±0.195
2.174±0.702
0.362±0.039
6
1.43±0.170
0.358±0.020
18.010±8.636
0.256±0.046
7
1.480±0.132
0.360±0.015
6.807±1.724
0.345±0.037
Table 1: Mean absolute error (MAE) of estimated drift and diffusion for dimensions d . Each box reports the mean ± the standard error.
d
A Estimation
GG ⊤ Estimation
WOT
APPEX
WOT
APPEX
3
0.996±0.001
0.998±0.001
0.837±0.048
0.985±0.005
4
0.943±0.015
0.987±0.005
0.729±0.039
0.865±0.031
5
0.921±0.016
0.952±0.030
0.728±0.040
0.909±0.018
6
0.794±0.040
0.986±0.001
0.530±0.056
0.961±0.007
7
0.792±0.029
0.988±0.001
0.595±0.037
0.946±0.012
Table 2: Correlation between estimated and true drift and diffusion for dimensions d . Each box reports the mean ± the standard error.
Figure 5: The correlation between the estimated and true SDE parameters is plotted per iteration across 10 random linear additive noise SDEs for dimensions 3 and 10 . Each point reports the mean of the 10 runs, while the bars represent the standard error.
d
p=0.1
p=0.25
p=0.5
WOT
APPEX
WOT
APPEX
WOT
APPEX
3
0.40±0.40
0.00±0.00
1.80±0.76
0.30±0.30
1.40±0.65
0.00±0.00
5
3.80±2.13
0.10±0.10
3.56±1.70
0.00±0.00
4.40±1.14
0.60±0.22
10
38.3±8.53
0.50±0.40
48.6±6.55
0.80±0.39
37.4±4.66
3.30±0.70
Table 3: Average Structural Hamming Distance (lower is better) with varying dimensions d and random edge probabilities p . Each box reports the mean ± the standard error.
d
Hamming distance for drift edges
Hamming distance for latent confounders
WOT
APPEX
WOT
APPEX
3
1.20±0.51
0.7±0.40
1.00±0.30
0.00±0.00
5
4.80±1.47
1.60±0.87
3.50±0.64
0.10±0.10
10
38.00±6.30
3.00±0.77
23.80±3.86
2.90±1.39
Table 4: Average Structural Hamming Distance for directed edges, and latent confounders (lower is better) with varying dimensions d and random edge probability p=0.25 . Each box reports the mean ± the standard error.
Figure 6: The true and estimated causal graphs by WOT and APPEX for four random SDEs of varying dimensions are illustrated. Red edges i→j represent negative edge weights, such that X(i) negatively regulates X(j) . Green edges i→j represent positive edge weights, such that X(i) positively regulates X(j) . Dashed edges represent the effect of a latent confounder Ui,j on observed variables i,j through correlated diffusion.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Marginals are plotted at various times for SDE pairs following various non-auto-rotationally invariant initial distributions. Importantly, these initializations enable previously non-identifiable systems to be distinguished.
Figure 8: We plot APPEX’s drift and diffusion estimates across 10 random seeds for the Ornstein–Uhlenbeck SDE dXt=−Xtdt+dWt from the non-identifiable setting, where marginals are sampled from peq=N(0,21) .
Figure 9: Average mean squared error (MSE) of A and GG⊤ as the number of observations per marginal increases across 50 random 2d systems. Note that this is a log-log plot and the dotted line in the APPEX plot shows that the convergence follows a power law relationship with exponent −0.125 . The number of observed marginals is fixed at 20 .
Figure 10: Average mean squared error (MSE) of A and GG⊤ as the number of observations per marginal increases across 70 random 12d systems. In this log-log plot, the dotted line in the APPEX plot indicates that the convergence follows a power law relationship with exponent −1 . The number of observed marginals is fixed at 20 .
Causal representation learning for time series has developed strong identifiability results in discrete-time latent causal models, but identifiability in continuous-time latent stochastic differential equation (SDE) models remains largely open. We address this gap using environment-induced shifts in diffusion covariance. We study additive-noise latent SDEs observed through an unknown nonlinear diffeomorphism, with shared drift but environment-specific diffusion covariance. We show that two diagonal diffusion regimes with pairwise distinct coordinate-wise variance ratios identify the latent coordinates up to permutation, coordinate-wise scaling, and a possible constant shift, without any sparsity assumption on the drift. We first prove this result for linear Ornstein-Uhlenbeck systems and then extend it to general additive-noise latent SDEs. Under mild smoothness, the instantaneous drift-Jacobian causal graph is identifiable up to the same permutation. We propose a two-stage estimator for latent disentanglement and optional graph recovery; experiments on synthetic systems confirm the predicted identifiability boundary, and an application to Hardanger Bridge monitoring data illustrates the approach on real sensor trajectories.
Yuanyuan Wang, Wenjie Wang, Haoxuan Li +2
Mohamed bin Zayed University of Artificial Intelligence · The University of Melbourne · Peking University +1
Continuous-time stochastic differential equations (SDEs) provide a natural framework for modeling interacting dynamical systems. A central causal question is whether causal effects among observed variables can be identified from observational data drawn from the stationary distribution, given the underlying causal graph. For linear SDEs, existing identifiability results are largely restricted to fully observed systems whose causal graphs contain no directed 2-cycles. We first establish new graphical conditions for the generic identifiability of causal effects in systems that may contain directed 2-cycles, thereby allowing for reciprocal interactions. Next, we address the previously unexplored setting of latent variables by developing an instrumental-variable framework for identifying a target direct causal effect under partial observability.
Ana Leticia Garcez Vicente, Saber Salehkaleybar
Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Netherlands
We consider sparse multivariate stochastic systems that evolve in continuous time according to a causal mechanism and present methodology to recover the system's time-infinitesimal transition mechanism from mere cross-sectional data. This observational paradigm is motivated by applications such as gene expression analysis, where destructive experimental techniques may only allow recording data once over a cell's lifetime. Precisely, we assume the system follows a time-homogeneous diffusion process that has reached an equilibrium distribution at observation time. Further, we assume the causal mechanism is fully described by the diffusion drift, is acyclic, and its causal structure graph is known. In this setting, we prove that the full causal mechanism, i.e., the drift function, can be non-parametrically identified under a weak non-explosion criterion. We derive a non-parametric kernel estimator for this challenging inverse problem and prove its consistency. Moreover, we propose a cross-validation scheme for hyperparameter tuning, illustrate the behavior of our estimator in simulations, and we discuss connections with irreversible generative diffusion models and low-frequency sampled data.
Richard Schwank, Mathias Drton
School of Computation, Information and Technology, Technical University of Munich · 2Munich Center for Machine Learning