Continuous-time latent state-space models are widely used for irregularly observed time series, typically representing uncertainty through a random initial state followed by smooth ordinary differential equation (ODE) dynamics, or Brownian perturbations accumulated by a stochastic differential equation (SDE). While these constructions are natural for smooth or diffusive systems, they do not directly encode a finite set of unobserved shocks whose timing, magnitude, and effect on memory are themselves uncertain. We instead introduce latent jump models (LJMs): the latent state follows closed-form deterministic dynamics between Poisson-driven event times and undergoes a learned instantaneous jump at each event, with state-dependent intensity and mark distributions. This defines a time-homogeneous Markov process whose jumps are intrinsic state transitions rather than numerical integration steps. For training, we derive a tractable event-augmented path-space evidence lower bound (ELBO) and introduce Scan-Parallel Variational Inference (SPVI): an observation-conditioned variational posterior samples the complete latent event configuration before the state recurrence is evaluated using triangular affine scans. This enables fast and scalable training for LJMs in logarithmic temporal depth. Across synthetic stochastic systems and real irregular time series, LJMs remain competitive with continuous-time latent baselines in forecast accuracy, often with fewer latent updates than numerical solver evaluations, while training one to two orders of magnitude faster than the sequential baselines.
Figures & tables
Figure 1: Comparison across four event-driven and smooth systems. All models use identical history-encoder and emission architectures. Solver step sizes are coarsened for baselines at evaluation to approach LJM event budget; NFE labels show the number of dynamics evaluations. LJM shows better reconstruction and forecast quality at fewer NFE. Latent SDE diverges on the regular PDMP, while Latent ODE and Neural Stochastic Flow (solver-free) exhibit errors in several examples.
Figure 2: (a) Generative prior : the current latent state determines the intensity and mark distribution of subsequent events, requiring chronological simulation. (b) SPVI posterior : observations determine the initial-state distribution and event laws independently of the sampled latent state. The complete event configuration can therefore be sampled before constructing the trajectory through successive blockwise scans. Both processes share the closed-form flow Φδ and jump map Fθ .
LJM (ours)
Best baseline at a similar budget
System
CRPS ↓
RMSE ↓
NFE ↓
Method
CRPS ↓
RMSE ↓
NFE ×
Train time ×
Cyclic jumps
\underset{\text{\tiny(0.0032)}}{\mathbf{0.0679}}
\underset{\text{\tiny(0.0036)}}{\mathbf{0.1261}}
89
Latent ODE
\underset{\text{\tiny(0.0010)}}{0.0729}
\underset{\text{\tiny(0.0015)}}{0.1276}
2.0 ×
7 ×
Irregular PDMP
\underset{\text{\tiny(0.0012)}}{\mathbf{0.2244}}
\underset{\text{\tiny(0.0017)}}{\mathbf{0.5210}}
19
Latent SDE
\underset{\text{\tiny(0.0036)}}{0.2410}
\underset{\text{\tiny(0.0118)}}{0.5467}
2.6 ×
10 ×
Regular PDMP
\underset{\text{\tiny(0.0009)}}{\mathbf{0.2509}}
\underset{\text{\tiny(0.0022)}}{0.5995}
19
SDE Matching
\underset{\text{\tiny(0.0009)}}{0.2567}
\underset{\text{\tiny(0.0041)}}{\mathbf{0.5945}}
2.3 ×
1 ×
Damped rotation
\underset{\text{\tiny(0.0056)}}{0.1012}
\underset{\text{\tiny(0.0225)}}{0.1693}
205
NCDSSM-NL
\underset{\text{\tiny(0.0017)}}{\mathbf{0.0743}}
\underset{\text{\tiny(0.0027)}}{\mathbf{0.1274}}
1.7 ×
86 ×
Table 1: Synthetic forecasting benchmarks. The reported numbers are mean and range over three independent runs of the whole study. NFE is dynamics evaluations per context trajectory; in the columns marked × , a value K× means LJM is K times cheaper than the baseline.
Context
Method
CRPS /sh↓
RMSE (W) ↓
E-CRPS(kWh)↓
90% cov.
Width (W)
512
LJM (ours)
0.0495 ±0.0004
551.7 ±2.9
0.0630 ±0.0005
0.925 ±0.005
858 ±33
SDE Matching
0.0544 ±0.0011
566.9 ±1.2
0.0705 ±0.0007
0.876 ±0.017
804 ±120
Latent NSF †
0.0588 ±0.0018
601.5 ±21.3
0.0779 ±0.0031
0.877 ±0.012
918 ±322
1024
LJM (ours)
0.0492 ±0.0004
551.4 ±0.2
0.0628 ±0.0005
0.926 ±0.008
926 ±40
SDE Matching
0.0568 ±0.0006
573.7 ±2.0
0.0730 ±0.0010
0.883 ±0.003
937 ±161
Latent NSF †
0.0599 ±0.0015
589.3 ±9.2
0.0794 ±0.0030
0.872 ±0.013
740 ±135
Table 2: REFIT aggregate load forecasting using 256 predictive trajectories per test window. Context is the number of observed points. CRPS /sh normalises by the household scale. Energy CRPS scores total consumption over the 30-minute forecast horizon.
Model
Forecast MSE ↓
CRPS ↓
90% coverage
NFE ↓
Train time/fold ↓
LJM (ours)
0.360±0.069
0.258±0.024
0.837±0.030
461
7.5 min
NCDSSM-NL
0.352±0.055
0.305±0.013
0.968±0.006
6740
3.6 h
Latent SDE
0.414±0.078
0.306±0.040
0.592±0.127
1685
21 h
Latent ODE
0.451±0.107
0.371±0.042
0.284±0.033
6740
11.5 h
SDE Matching
0.433±0.106
0.259±0.030
0.958±0.018
1685
33 min
Latent NSF
0.829±0.162
0.477±0.058
0.605±0.049
422
19 min
Table 3: USHCN test forecasts: The numbers show mean ± sd over five station folds with 128 predictive paths.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Latent dimension / ordered block sizes
8 / (4,4)
Mark dimension / observation dimension
2 / 2
Training batch size / observations per window
512 / 52
Context / forecast observations
44 / 8
Predictive samples / evaluation batch size
128 / 16
Posterior rate range
(10−3,10)
Appendix
Table 4: Synthetic implementation settings. The KL weight takes values in the listed set; the event-regularization weight is swept separately.
Dataset
Event times
Jump / innovation
σ
Grid
Per-traj. randomness
Deterministic rotation
none
none
0.10
regular
θ , z0
Damped rotation
none
Gaussian, ς=0.02
0.10
regular
α,ω , z0
Cyclic jumps
every 5 steps
4 -cycle, fixed dirs.
0.10
regular
a1,a2 , c1,c2 , z0
Cyclic jumps, fixed decay
every 5 steps
4 -cycle, fixed dirs.
0.10
regular
c1,c2 , z0
Regular PDMP
Poisson, λ=0.3
W1tanh(⋅) , Eq. ( 28 )
0.15
regular
events, marks
Irregular PDMP
Poisson, λ=0.3
W1tanh(⋅) , Eq. ( 28 )
0.15
random
events, marks, grid
Appendix
Table 5: The six synthetic datasets. All have latent and observation dimension 2 and 52 observations per trajectory, of which 44 are context
Figure 3: Forecast quality against latent update cost on the six synthetic systems. Rows are systems, columns are CRPS, forecast RMSE, horizon CRPS, joint LPD of the central 90% predictive interval, and joint log predictive density per scalar. The horizontal axis is the number of dynamics-network evaluations per trajectory over the context window, on a logarithmic scale. LJM is shown at each of βsparse∈{0,0.03,0.1,0.3,1} ; points of one family are joined in order of cost.
RMSE
90% coverage
Future events/path
βsparse
Full
Floor
Full
Floor
Full
Cyclic jumps
0
0.124
1.312
0.88
0.37
50.6
0.03
0.126
1.225
0.88
0.38
35.2
0.1
0.126
0.808
0.88
0.39
22.7
0.3
0.127
0.473
0.88
0.56
14.7
Appendix
Table 6: Future-event suppression across all six synthetic systems and five event-count penalties. Full uses the learned prior intensity; Floor holds it at its positive minimum during forecasting. Context inference and model parameters are unchanged. Each condition uses 1,000 validation trajectories and 128 predictive paths. The last column is the mean accepted future event count under Full. Coverage is rounded to two decimal places.
βsparse
Nev↓
CRPS /sh↓
RMSE (W) ↓
Cov. → 0.90
Width (W)
Energy CRPS ↓
0
205.4
0.0491
550.3
0.923
830
0.0627
0.01
200.6
0.0496
549.8
0.914
858
0.0633
0.03
176.8
0.0508
563.8
0.929
864
0.0664
0.1
185.0
0.0507
559.8
0.928
876
0.0649
0.3
147.5
0.0517
563.5
0.919
855
0.0668
1
98.4
0.0516
565.4
0.926
920
0.0666
Appendix
Table 7: Event-count penalty sweep for the latent jump model on REFIT, 18 households, context 512, test split.
Forward (ms)
Forward + backward (ms)
Peak memory
Model
B
L
Seq.
Scan
Seq.
Scan
Speed-up
Seq.
Scan
Cyclic
16
1024†
985
5.0
3,025
24
125×
155 MB
174 MB
Cyclic
128
1024
1,019
10.9
3,053
32
96×
1.20 GB
1.30 GB
Cyclic
128
4096
4,120
41.8
12,506
93
134×
4.81 GB
5.20 GB
Appendix
Table 8: Matched-model posterior state construction on one NVIDIA A40 (float32; median times). Seq. is the chronological flow–gated-jump loop; Scan is the pure-PyTorch scan used in training, without a fused kernel. B is batch size and L is the processed proposal length. † marks training padding lengths. Speed-up refers to the forward–backward pass and are computed from unrounded timing measurements; timing entries are rounded. Peak-memory units are shown in each entry.
Time series driven by unobserved latent states frequently exhibit abrupt jump discontinuities whose timing and magnitude cannot be predicted from observed history alone. Classical jump-diffusion models offer a principled mathematical framework but assume rigid parametric forms, while recent neural jump models operate on fully observed trajectories without inferring the hidden states that govern the dynamics. We propose Deep ZakaiJ, a latent-state model for partially observed jump-diffusion systems that embeds the Zakai nonlinear filtering equation into a neural encoder--decoder architecture. The encoder recursively updates a belief over the latent state via Strang splitting into three interpretable substeps: prior propagation, diffusion innovation, and jump innovation, yielding a differentiable, first-order-accurate approximation of the exact filtering evolution. The decoder is a structured jump-diffusion model explicitly conditioned on the filtered belief, preserving the separation between continuous dynamics and discontinuous shocks. On synthetic, financial, and oceanographic datasets, Deep ZakaiJ improves distributional forecasts while remaining competitive in point accuracy, achieving calibrated predictive intervals and recovering interpretable latent structure in synthetic and qualitative case studies.
Yan Leng, Thibaut Mastrolia, Hao Wang
University of Texas at Austin · University of California, Berkeley
Irregular multivariate time series impose a trade-off for long-horizon forecasting: discrete methods can distort temporal structure via re-gridding, while continuous-time models often require sequential solvers prone to drift. To bridge this gap, we present Latent Laplace Diffusion (LLapDiff), a generative framework that models the target as a low-dimensional latent trajectory, enabling horizon-wide generation without step-by-step integration over physical time. We guide the reverse process utilizing a stable modal parameterization motivated by stochastic port-Hamiltonian dynamics, and parameterize its mean evolution in the Laplace domain via learnable complex-conjugate poles, enabling direct evaluation over irregular timestamps. We also link continuous dynamics to irregular observations through renewal-averaging analysis, which maps sampling gaps to effective event-domain poles and motivates a gap-aware history summarizer. Extensive experiments show that LLapDiff improves over baselines in long-horizon forecasting, and its continuous-time generative nature supports missing-value imputation by querying the same model at historical timestamps. Code is available at https://github.com/pixelhero98/LLapDiffusion.
Zinuo You, Jin Zheng, John Cartlidge
School of Computer Science, University of Bristol · School of Engineering Mathematics and Technology, University of Bristol
Modelling extreme events and heavy-tailed phenomena is central to building reliable predictive systems in domains such as finance, climate science, and safety-critical AI. While Lévy processes provide a natural mathematical framework for capturing jumps and heavy tails, Bayesian inference for Lévy-driven stochastic differential equations (SDEs) remains intractable with existing methods: Monte Carlo approaches are rigorous but lack scalability, whereas neural variational inference methods are efficient but rely on Gaussian assumptions that fail to capture discontinuities. We address this tension by introducing a neural exponential tilting framework for variational inference in Lévy-driven SDEs. Our approach constructs a flexible variational family by exponentially reweighting the Lévy measure using neural networks. This parametrization preserves the jump structure of the underlying process while remaining computationally tractable. To enable efficient inference, we develop a quadratic neural parametrization that yields closed-form normalization of the tilted measure, a conditional Gaussian representation for stable processes that facilitates simulation, and symmetry-aware Monte Carlo estimators for scalable optimization. Empirically, we demonstrate that the method accurately captures jump dynamics and yields reliable posterior inference in regimes where Gaussian-based variational approaches fail, on both synthetic and real-world datasets.
Yaman Kindap, Manfred Opper, Benjamin Dupuis +2
1Imperial College London, UK · 2Technical University of Berlin, Germany · 3INRIA, CNRS, Département d’Informatique de l’Ecole Normale Supérieure / PSL, France