Learning hidden-state dynamics from unpaired snapshots with lift-project SDEs
Authors: Qi Wang, Uduak George, Gustaaf Jacobs
Organizations: Department of Aerospace Engineering, San Diego State University, San Diego, CA, USA · Department of Mathematics and Statistics, San Diego State University, San Diego, CA, USA
Many systems are observed only through unpaired snapshots, with population samples available at each time point but no correspondence between samples at different times. These snapshots determine the one-time distributions but leave the temporal coupling unspecified, requiring a modeling prior to connect successive times. We propose the lift-project stochastic differential equation (SDE), an autonomous SDE on the observed coordinate and a few hidden coordinates, trained only to match the snapshots. Most existing methods, including time-dependent SDEs, Schrödinger bridges and optimal transport, evolve the observed coordinate alone. For a scalar observation, the classical comparison theorem implies that all such models have non-negative lagged covariance, so on data whose lagged covariance turns negative, their error has a computable floor. The lift-project SDE is not bound by this floor. On two representative partially observed systems, the Duffing and pendulum oscillators, at comparable fit to the snapshots, the proposed method reduces the lagged-covariance error by factors of 2.4 and 6.6 relative to the strongest baseline and is the only model below the error floor; one hidden coordinate is already enough for such improvement. Furthermore, if a second experiment changes only the initial state of the hidden coordinate, no single model that evolves the observed coordinate alone can fit both experiments, whereas a single lift-project SDE can.
Figures & tables
Figure 1: The lift–project SDE recovers the temporal coupling of a partially observed system. (a) Unpaired snapshots: samples at different times cannot be paired. (b) Schematic of the method: lift the observable x to z=(x,h) and advance with an autonomous SDE, followed by a projection back to x . (c) Lagged covariance C(t,t+d) of the true Duffing process and of three models learned only from its snapshots, with their errors ϵC .
Figure 2: The same marginals admit different temporal couplings. Colored patches show the marginals ptk of the Duffing position x at the training times, scaled to equal width. Lines mark three paths of x in each process. (a) The true process; paths cross because the hidden velocity carries them. (b) The rank-preserving process B; paths never cross, yet its marginals are identical to those of the true process.
Figure 3: Training curves on (a) Duffing and (b) the pendulum: minibatch loss per snapshot time against iteration n ; thin lines are seeds and bold lines the seed median. Dotted lines mark where the rollout horizon reaches T . The lift–project loss falls by orders of magnitude for every hidden dimension, whereas obs-TH, which has no hidden state, stays far above it; the Duffing obs-TD curve starts low because it continues from a multiple-shooting solution.
Figure 4: At comparable marginal fit, the lift–project SDE is the only model below the error floor of observed-state models. (a, b) Lagged correlation r(4,4+d)=C(4,4+d)/C(4,4)C(4+d,4+d) : truth (dashed), lift–project seed median with its seed range shaded, and the seed median of each baseline. (c) Relative lagged-covariance error ϵC . (d) Relative lagged-rank-correlation error ϵρ . (e) Training-window marginal error W2train . Points: seeds; bars: seed medians; dashed lines: error floor for every diffusion that is Markovian in x (grey: the region such models can reach); dotted: noise floors.
Figure 5: Effect of the hidden dimension dh ( dh=0 is obs-TH). (a) Training-window marginal fit. (b) Relative lagged-covariance error ϵC inside the observation window. (c) Mean W2 at t=1.5T,2T,3T beyond the training window. Large markers joined by lines are seed medians; small markers are the 5 seeds.
Figure 6: Two Duffing experiments with the same law of x0 but different hidden velocities. (a) W2 between the marginals of the true process (A) and process B under μ0 and μ1 , with the noise floor. (b) Training marginal error with both experiments over that with one (worse experiment shown); dotted: equal fit. (c) ϵC under μ0 ; bars: seed medians, dots: seeds; dashed: error floor of observed-state models.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Duffing data (the true process); only the position x is observed. (a) Five paths of the true system in [0,T] , colored by the hidden velocity v ; grey: the 5 – 95% range of the marginals. (b) Phase portrait (x,v) of three of these paths, colored by time t ; circles mark t=0 . (c) Densities of x at the 16 training times (colored by tk ) and at the extrapolation times 1.5T , 2T , 3T (dashed, never used in training). (d) The training data as a model sees them: 300 of the 2000 unpaired samples at each tk , spread horizontally for visibility.
Figure 8: Pendulum data (the true process); only the angle x is observed. (a) Five paths of the true system in [0,T] , colored by the hidden angular velocity ω ; grey: the 5 – 95% range of the marginals. (b) Phase portrait (x,ω) of three of these paths, colored by time t ; circles mark t=0 . (c, d) As in Figure 7 .
Figure 9: Three Duffing processes with the same marginals. (a–c) Five sample paths of the true process A (the stochastic Duffing oscillator), process B (the rank-preserving process of Lemma 1 ) and process C (the Gaussian-copula process of Appendix D ); the color of a path encodes the rank of its initial value x0 , and the grey band is the 5 – 95% range of the shared marginals. Process-B paths never cross, so their colors stay ordered; process-A and process-C paths cross. (d–f) Lagged covariance C(t,t+d) of each process from 4000 paths, on one color scale centered at zero; solid line: C=0 ; dashed line: t+d=T , the edge of the ϵC grid.
Figure 10: Each model class selects a stable coupling across marginal-equivalent data. Models are placed by their relative lagged-covariance error to process A ( ϵCA ) and process B ( ϵCB ); large markers: seed medians; small markers: individual seeds; stars: processes. (a) Duffing, trained on process-A (filled) and process-B data (open). (b) Pendulum, process-A data. (c) Duffing, trained on A (filled), B (open) and C data (light); open star: process C, marginal-equivalent to A and B.
Figure 11: The two FUCCI datasets; top row GSE121265 (iPSC, 888 cells), bottom row GSE146773 (U2OS, 1146 cells). (a, d) The first two of the six principal components that form the observed state x , colored by the continuous FUCCI phase ψ (cyclic color scale). (b, e) The same cells colored by the six equal-count phase bins k that define the snapshots; open circles mark the held-out bin k=3 . (c, f) FUCCI reporter intensities, green channel IG against red channel IR , colored by ψ (GSE121265: background-adjusted log10 sums; GSE146773: rescaled log intensities, as provided by the authors). Axes show the central 99% of cells. The continuous phase ψ is shown here for illustration only; no model was trained on it.
Figure 12: FUCCI case study. (a) Held-out-bin marginal fit W2 for each class; filled: GSE121265, open: GSE146773. (b, c) Circular mean absolute error against the withheld FUCCI phase ψ for GSE121265 and GSE146773; filled: all cells, open: held-out-bin cells; dashed: centroid-polygon reference; dotted: chance ( π/2 ). (d) Supervised static readout from one transcriptome; filled: unseen cells of the training bins, open: held-out-bin cells; dashed: permuted labels. Points are seeds (a–c) or random splits (d); bars are means.