Identifying the Predictable Drift of a Semimartingale from Marginal Laws

Jakub Marecek; Enrico Biffis; Abigail Langbridge; Robert Shorten

math.STSep 27, 2026

Identifying the Predictable Drift of a Semimartingale from Marginal Laws

Authors: Jakub Marecek, Enrico Biffis, Abigail Langbridge, Robert Shorten

Organizations: Department of Computer Science, Czech Technical University in Prague, Czechia · Department of Finance, Imperial College London, UK · Dyson School of Design Engineering, Imperial College London, UK

Abstract

A special semimartingale admits a unique decomposition $X=X_0+M+A$ into a local martingale $M$ and a predictable finite-variation part $A$ . We consider the identification of $A$ when $X$ is observed only through repeated cross-sections. The estimand is then the projection of the sampled predictable compensator onto the observable feature filtration, namely the current state together with whatever randomness is shared across the population, so that at a fixed diffusion coefficient the marginal flow identifies the drift only up to a Markovian projection. If the drift is an affine functional of an observed lag window, the joint problem is a convex quadratic programme whose solution is the pseudo-panel regression of econometrics. Our principal concern is the case, which we believe not to have been treated before, in which the drift is the output of a hidden linear dynamical system whose dynamics are themselves to be identified from the marginals. The joint problem is then a bilinear quadratically constrained programme, which we solve to certified global optimality by spatial branch and bound; with unpenalised state disturbances and a drift basis growing with the grid it is NP-hard already in latent dimension one, by reduction from $\ell^1$ rank-one matrix approximation, whereas the complexity of the deterministic system at fixed latent dimension remains open. A block-coordinate decomposition offers a cheaper alternative. For the estimator itself, we obtain rates at a fixed mesh, separated into Monte-Carlo, estimation and grid contributions.

Figures & tables

	$K_{\mathrm{tr}}=24$		$K_{\mathrm{tr}}=96$
method	median	range	median	range
ours, Non-Convex Setting (identified $(a,c,\alpha,\beta)$ )	$0.043$	$[0.008,\,0.051]$	$0.012$	$[0.002,\,0.013]$
FIR of $u$ , four lags, with $(1,\bar{x}_{k})$	$0.050$	$[0.045,\,0.074]$	$0.042$	$[0.042,\,0.047]$
PO-MFL-style AR(1) prior on the residual, run forward	$0.351$	$[0.275,\,0.740]$	$0.489$	$[0.295,\,0.526]$
persistence: last Waddington-OT field carried forward	$0.589$	$[0.452,\,0.893]$	$0.813$	$[0.496,\,1.016]$
Waddington-OT surrogate given the next snapshot	$0.073$	$[0.069,\,0.083]$	$0.080$	$[0.076,\,0.088]$

Table 1: Forecasting the drift from the observed input: each method is fitted upon the first

K_{\mathrm{tr}}

pairs and predicts

b_{k}

for the next

12

steps from

u

alone;

\|b_{k}-b_{k}^{\mathrm{true}}\|_{L^{2}(\mu_{k})}

averaged over those steps, median over 5 seeds with the range, at

N=14

M=600

( code/compare_regimes.py ).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

	$N$	$h/\sigma\sqrt{\Delta}$	clip (pairs)	clip (mass)	rel	icv	err
${\epsilon}=0.05$	$100$	$0.346$	$0.29$	$6.9\cdot 10^{-4}$	$3.3\cdot 10^{-4}$	$0.493$	$0.0634$
	$200$	$0.172$	$0.33$	$6.3\cdot 10^{-4}$	$3.3\cdot 10^{-4}$	$0.491$	$0.0612$
	$300$	$0.115$	$0.33$	$6.3\cdot 10^{-4}$	$3.2\cdot 10^{-4}$	$0.490$	$0.0597$
	$400$	$0.086$	$0.33$	$7.6\cdot 10^{-4}$	$3.1\cdot 10^{-4}$	$0.490$	$0.0597$
${\epsilon}={\epsilon}^{\star}$	$100$	$0.346$	$0.29$	$6.9\cdot 10^{-4}$	$1.4\cdot 10^{-4}$	$0.491$	$0.0625$
	$200$	$0.172$	$0.33$	$6.3\cdot 10^{-4}$	$1.6\cdot 10^{-4}$	$0.489$	$0.0611$

Appendix

Table 2: Medians over five seeds.

\mathrm{rel}

is the relative discrepancy between the block iteration and the closed form ( 30 );

\mathrm{err}

\|\hat{\Theta}-\Theta_{\star}\|

;

\mathrm{icv}

is the implied conditional variance of the coupling in units of

\sigma^{2}\Delta

; the clip is reported both as the fraction of pairs at which some atom is clipped and as the

\mu_{k}

-mass actually displaced.

	$N=14$		$N=100$		mean-disp.
method	median	range	median	range	gap
ours, Non-Convex Setting	$0.039$	$0.006$ – $0.048$	$0.020$	$0.003$ – $0.045$	$3\cdot 10^{-2}$
PO-MFL surrogate, data prior	$0.024$	$0.022$ – $0.033$	$0.026$	$0.022$ – $0.033$	$1\cdot 10^{-8}$
PO-MFL surrogate, oracle prior	$0.027$	$0.017$ – $0.047$	$0.023$	$0.022$ – $0.032$	$1\cdot 10^{-8}$
mean-field Langevin surrogate (path space)	$0.074$	$0.069$ – $0.077$	$0.058$	$0.056$ – $0.063$	$6\cdot 10^{-11}$
Waddington-OT surrogate, ${\epsilon}={\epsilon}^{\star}$	$0.074$	$0.069$ – $0.077$	$0.058$	$0.056$ – $0.063$	$4\cdot 10^{-16}$

Appendix

Table 3: Drift error

\|b_{k}-b_{k}^{\mathrm{true}}\|_{L^{2}(\mu_{k})}

, averaged over

k

; median over five seeds, with the range beneath. The last column is the largest deviation of the implied mean displacement from

\gamma_{k}

. The three comparators are the surrogates described above, not the published estimators.

	drift error		time (s)
method	median	range	median	range
ours, Non-Convex Setting	$0.020$	$[0.003,\,0.045]$	$31.5$	$[27.7,\,43.2]$
PO-MFL surrogate, data prior	$0.026$	$[0.022,\,0.033]$	$5.9$	$[5.1,\,7.5]$
PO-MFL surrogate, oracle prior	$0.023$	$[0.022,\,0.032]$	$6.3$	$[5.9,\,7.8]$
mean-field Langevin surrogate (path space)	$0.058$	$[0.056,\,0.063]$	$0.5$	$[0.5,\,0.6]$
Waddington-OT surrogate, ${\epsilon}={\epsilon}^{\star}$	$0.058$	$[0.056,\,0.063]$	$1.7$	$[1.7,\,2.7]$

Appendix

Table 4: Drift error

\|b_{k}-b_{k}^{\mathrm{true}}\|_{L^{2}(\mu_{k})}

averaged over

k

N=100

, and wall-clock seconds per fit: medians over five seeds, with ranges. The comparators are the surrogates of Appendix U , not the published estimators; PO-MFL is granted a prior upon the latent dynamics, which we must identify. Our Non-Convex time includes the

2000

-sweep E-step; the Convex Setting is the closed form ( 30 ).

$M$	$K$	Waddington-OT, ${\epsilon}={\epsilon}^{\star}$	PO-MFL surrogate, data prior	ours, Non-Convex Setting
$50$	$24$	$0.097$ $[0.082,0.101]$	$0.070$ $[0.053,0.076]$	$0.038$ $[0.029,0.046]$
$50$	$48$	$0.099$ $[0.090,0.111]$	$0.071$ $[0.068,0.084]$	$0.033$ $[0.027,0.058]$
$50$	$96$	$0.103$ $[0.097,0.112]$	$0.072$ $[0.067,0.081]$	$0.038$ $[0.015,0.062]$
$100$	$24$	$0.087$ $[0.079,0.091]$	$0.052$ $[0.047,0.055]$	$0.039$ $[0.024,0.053]$
$100$	$48$	$0.090$ $[0.079,0.092]$	$0.053$ $[0.046,0.058]$	$0.043$ $[0.023,0.048]$
$100$	$96$	$0.093$ $[0.089,0.098]$	$0.052$ $[0.050,0.060]$	$0.041$ $[0.004,0.047]$

Appendix

Table 5: Drift error

\|b_{k}-b_{k}^{\mathrm{true}}\|_{L^{2}(\mu_{k})}

, averaged over

k

, against the number of copies

M

and the number of pairs

K

, at

N=14

; medians over 5 seeds with the range in brackets. The surrogates are those of Table 3 ; the PO-MFL surrogate uses the prior estimated from observables. A per-step method sees each pair once and sits at the Monte-Carlo floor of one displacement field whatever

K

is; the parametric fit pools the

K

residuals. The script is code/compare_regimes.py .

	total variation		$W_{1}$
coupling	$N=14$	$N=100$	$N=14$	$N=100$
drift-constrained, ${\epsilon}=0.05$	$0.033$	$0.091$	$0.023$	$0.022$
drift-constrained, ${\epsilon}={\epsilon}^{\star}$	$0.034$	$0.091$	$0.024$	$0.022$
unconstrained, ${\epsilon}=0.05$	$0.173$	$0.203$	$0.168$	$0.167$
unconstrained, ${\epsilon}={\epsilon}^{\star}$	$0.135$	$0.134$	$0.094$	$0.065$

Appendix

Table 6: Fidelity of the row-conditional laws to the true transition kernel; medians over five seeds, at

{\epsilon}=0.05

and at

{\epsilon}^{\star}

	penalised objective ( 10 )			$\\|\hat{\Theta}-\Theta_{\star}\\|$		time (s)
seed	decomposition	global QCQP	bound	decomp.	QCQP	decomp.	QCQP
3	1.2642	1.0047	1.0047	0.064	0.053	1.5	0.3
4	1.3741	1.2287	1.2287	0.139	0.147	1.2	0.2
5	1.7346	1.5045	1.5045	0.119	0.116	1.6	0.4

Appendix

Table 7: The block-coordinate decomposition next to the globally solved complete problem ( 10 ) (

N=10

K=11

d=2

a\in\mathcal{B}_{\varrho}

in both; the QCQP has 1361 variables and 684 constraints). Columns 2–4: the penalised objective of ( 10 ) at the parameter returned by each method, and the lower bound certified by spatial branch and bound. Columns 5–6: distance of the returned parameter to the truth. The two methods optimise different criteria, so the objective ordering and the accuracy ordering need not agree.

Explore similar work

Jun 26, 2026cs.LG

Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts

Causal representation learning for time series has developed strong identifiability results in discrete-time latent causal models, but identifiability in continuous-time latent stochastic differential equation (SDE) models remains largely open. We address this gap using environment-induced shifts in diffusion covariance. We study additive-noise latent SDEs observed through an unknown nonlinear diffeomorphism, with shared drift but environment-specific diffusion covariance. We show that two diagonal diffusion regimes with pairwise distinct coordinate-wise variance ratios identify the latent coordinates up to permutation, coordinate-wise scaling, and a possible constant shift, without any sparsity assumption on the drift. We first prove this result for linear Ornstein-Uhlenbeck systems and then extend it to general additive-noise latent SDEs. Under mild smoothness, the instantaneous drift-Jacobian causal graph is identifiable up to the same permutation. We propose a two-stage estimator for latent disentanglement and optional graph recovery; experiments on synthetic systems confirm the predicted identifiability boundary, and an application to Hardanger Bridge monitoring data illustrates the approach on real sensor trajectories.

Yuanyuan Wang, Wenjie Wang, Haoxuan Li +2

Mohamed bin Zayed University of Artificial Intelligence · The University of Melbourne · Peking University +1

Identifiability Latent Dynamics Causal Representation Learning Stochastic Differential Equations Covariance

Ask PDF

Jun 1, 2026stat.ML

Error Bounds for a Diffusion Model-Based Drift Estimator

Parameter estimation in stochastic differential equations is a classical statistical problem of much importance in many scientific fields. Recent work of Tapia Costa et al. (2026) introduced a novel technique for estimating the drift when the diffusion parameter is known, using discrete samples from multiple trajectories. Their method treats drift estimation as a denoising problem, and leverages tools from (conditional) score-matching diffusion models. Although their experiments showed promising results across different drift classes, the question of theoretical guarantees for their estimator was left unanswered. In this note, we address this gap by exploiting techniques from diffusion model theory. More concretely, we derive an explicit risk bound for the time-averaged mean-squared error of said drift estimator. Our bound decomposes the risk into the (i) Euler-Maruyama discretization, (ii) score/denoiser approximation, (iii) noise initialization, and (iv) sampling variance, revealing the trade-offs between the different hyperparameters and sources of error in the estimator.

Ioar Casado-Telletxea, Omar Rivasplata

Basque Center for Applied Mathematics (BCAM) · Bilbao, Spain · Centre for AI Fundamentals & Department of Computer Science +1

Stochastic Differential Equations Score-Based Diffusion Model Diffusion Dynamics Time Discretization

Ask PDF

Sep 29, 2026stat.ML

Generative sequence modeling for infinite memory processes via predictive states

We consider estimating the one-step-ahead conditional distribution of a multivariate stochastic process. Many existing approaches rely on assumptions such as finite-range memory, sparsity, or additivity, which can be poorly suited to processes with long-range nonlinear interactions. However, without such structural assumptions, nonparametric estimation is challenging due to the curse of dimensionality. To address this challenge, we introduce a new estimation approach based on the predictive states of a process, possibly with infinite-range memory. We show that our estimator achieves fast convergence rates when the past history can be compressed into a low-dimensional statistic that is sufficient for predicting the future. Specifically, we show that the statistical complexity of the estimation problem is determined by the intrinsic dimension of the predictive state space. We establish guarantees for an instantiation of our method based on deep neural network estimators, and we support these theoretical results with experiments.

Michael Wieck-Sosa, Cosma Rohilla Shalizi

Department of Statistics & Data Science Carnegie Mellon University

Next-State Prediction Sequence Modeling Conditional Distribution

Ask PDF