Identifying the Predictable Drift of a Semimartingale from Marginal Laws

Jakub Marecek; Enrico Biffis; Abigail Langbridge; Robert Shorten

math.STSep 27, 2026

Identifying the Predictable Drift of a Semimartingale from Marginal Laws

Authors: Jakub Marecek, Enrico Biffis, Abigail Langbridge, Robert Shorten

Organizations: Department of Computer Science, Czech Technical University in Prague, Czechia · Department of Finance, Imperial College London, UK · Dyson School of Design Engineering, Imperial College London, UK

Abstract

A special semimartingale admits a unique decomposition $X=X_0+M+A$ into a local martingale $M$ and a predictable finite-variation part $A$ . We consider the identification of $A$ when $X$ is observed only through repeated cross-sections. The estimand is then the projection of the sampled predictable compensator onto the observable feature filtration, namely the current state together with whatever randomness is shared across the population, so that at a fixed diffusion coefficient the marginal flow identifies the drift only up to a Markovian projection. If the drift is an affine functional of an observed lag window, the joint problem is a convex quadratic programme whose solution is the pseudo-panel regression of econometrics. Our principal concern is the case, which we believe not to have been treated before, in which the drift is the output of a hidden linear dynamical system whose dynamics are themselves to be identified from the marginals. The joint problem is then a bilinear quadratically constrained programme, which we solve to certified global optimality by spatial branch and bound; with unpenalised state disturbances and a drift basis growing with the grid it is NP-hard already in latent dimension one, by reduction from $\ell^1$ rank-one matrix approximation, whereas the complexity of the deterministic system at fixed latent dimension remains open. A block-coordinate decomposition offers a cheaper alternative. For the estimator itself, we obtain rates at a fixed mesh, separated into Monte-Carlo, estimation and grid contributions.

Figures & tables

	$K_{\mathrm{tr}}=24$		$K_{\mathrm{tr}}=96$
method	median	range	median	range
ours, Non-Convex Setting (identified $(a,c,\alpha,\beta)$ )	$0.043$	$[0.008,\,0.051]$	$0.012$	$[0.002,\,0.013]$
FIR of $u$ , four lags, with $(1,\bar{x}_{k})$	$0.050$	$[0.045,\,0.074]$	$0.042$	$[0.042,\,0.047]$
PO-MFL-style AR(1) prior on the residual, run forward	$0.351$	$[0.275,\,0.740]$	$0.489$	$[0.295,\,0.526]$
persistence: last Waddington-OT field carried forward	$0.589$	$[0.452,\,0.893]$	$0.813$	$[0.496,\,1.016]$
Waddington-OT surrogate given the next snapshot	$0.073$	$[0.069,\,0.083]$	$0.080$	$[0.076,\,0.088]$

Table 1: Forecasting the drift from the observed input: each method is fitted upon the first

K_{\mathrm{tr}}

pairs and predicts

b_{k}

for the next

12

steps from

u

alone;

\|b_{k}-b_{k}^{\mathrm{true}}\|_{L^{2}(\mu_{k})}

averaged over those steps, median over 5 seeds with the range, at

N=14

M=600

( code/compare_regimes.py ).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

	$N$	$h/\sigma\sqrt{\Delta}$	clip (pairs)	clip (mass)	rel	icv	err
${\epsilon}=0.05$	$100$	$0.346$	$0.29$	$6.9\cdot 10^{-4}$	$3.3\cdot 10^{-4}$	$0.493$	$0.0634$
	$200$	$0.172$	$0.33$	$6.3\cdot 10^{-4}$	$3.3\cdot 10^{-4}$	$0.491$	$0.0612$
	$300$	$0.115$	$0.33$	$6.3\cdot 10^{-4}$	$3.2\cdot 10^{-4}$	$0.490$	$0.0597$
	$400$	$0.086$	$0.33$	$7.6\cdot 10^{-4}$	$3.1\cdot 10^{-4}$	$0.490$	$0.0597$
${\epsilon}={\epsilon}^{\star}$	$100$	$0.346$	$0.29$	$6.9\cdot 10^{-4}$	$1.4\cdot 10^{-4}$	$0.491$	$0.0625$
	$200$	$0.172$	$0.33$	$6.3\cdot 10^{-4}$	$1.6\cdot 10^{-4}$	$0.489$	$0.0611$

Appendix

Table 2: Medians over five seeds.

\mathrm{rel}

is the relative discrepancy between the block iteration and the closed form ( 30 );

\mathrm{err}

\|\hat{\Theta}-\Theta_{\star}\|

;

\mathrm{icv}

is the implied conditional variance of the coupling in units of

\sigma^{2}\Delta

; the clip is reported both as the fraction of pairs at which some atom is clipped and as the

\mu_{k}

-mass actually displaced.

	$N=14$		$N=100$		mean-disp.
method	median	range	median	range	gap
ours, Non-Convex Setting	$0.039$	$0.006$ – $0.048$	$0.020$	$0.003$ – $0.045$	$3\cdot 10^{-2}$
PO-MFL surrogate, data prior	$0.024$	$0.022$ – $0.033$	$0.026$	$0.022$ – $0.033$	$1\cdot 10^{-8}$
PO-MFL surrogate, oracle prior	$0.027$	$0.017$ – $0.047$	$0.023$	$0.022$ – $0.032$	$1\cdot 10^{-8}$
mean-field Langevin surrogate (path space)	$0.074$	$0.069$ – $0.077$	$0.058$	$0.056$ – $0.063$	$6\cdot 10^{-11}$
Waddington-OT surrogate, ${\epsilon}={\epsilon}^{\star}$	$0.074$	$0.069$ – $0.077$	$0.058$	$0.056$ – $0.063$	$4\cdot 10^{-16}$

Appendix

Table 3: Drift error

\|b_{k}-b_{k}^{\mathrm{true}}\|_{L^{2}(\mu_{k})}

, averaged over

k

; median over five seeds, with the range beneath. The last column is the largest deviation of the implied mean displacement from

\gamma_{k}

. The three comparators are the surrogates described above, not the published estimators.

	drift error		time (s)
method	median	range	median	range
ours, Non-Convex Setting	$0.020$	$[0.003,\,0.045]$	$31.5$	$[27.7,\,43.2]$
PO-MFL surrogate, data prior	$0.026$	$[0.022,\,0.033]$	$5.9$	$[5.1,\,7.5]$
PO-MFL surrogate, oracle prior	$0.023$	$[0.022,\,0.032]$	$6.3$	$[5.9,\,7.8]$
mean-field Langevin surrogate (path space)	$0.058$	$[0.056,\,0.063]$	$0.5$	$[0.5,\,0.6]$
Waddington-OT surrogate, ${\epsilon}={\epsilon}^{\star}$	$0.058$	$[0.056,\,0.063]$	$1.7$	$[1.7,\,2.7]$

Appendix

Table 4: Drift error

\|b_{k}-b_{k}^{\mathrm{true}}\|_{L^{2}(\mu_{k})}

averaged over

k

N=100

, and wall-clock seconds per fit: medians over five seeds, with ranges. The comparators are the surrogates of Appendix U , not the published estimators; PO-MFL is granted a prior upon the latent dynamics, which we must identify. Our Non-Convex time includes the

2000

-sweep E-step; the Convex Setting is the closed form ( 30 ).

$M$	$K$	Waddington-OT, ${\epsilon}={\epsilon}^{\star}$	PO-MFL surrogate, data prior	ours, Non-Convex Setting
$50$	$24$	$0.097$ $[0.082,0.101]$	$0.070$ $[0.053,0.076]$	$0.038$ $[0.029,0.046]$
$50$	$48$	$0.099$ $[0.090,0.111]$	$0.071$ $[0.068,0.084]$	$0.033$ $[0.027,0.058]$
$50$	$96$	$0.103$ $[0.097,0.112]$	$0.072$ $[0.067,0.081]$	$0.038$ $[0.015,0.062]$
$100$	$24$	$0.087$ $[0.079,0.091]$	$0.052$ $[0.047,0.055]$	$0.039$ $[0.024,0.053]$
$100$	$48$	$0.090$ $[0.079,0.092]$	$0.053$ $[0.046,0.058]$	$0.043$ $[0.023,0.048]$
$100$	$96$	$0.093$ $[0.089,0.098]$	$0.052$ $[0.050,0.060]$	$0.041$ $[0.004,0.047]$

Appendix

Table 5: Drift error

\|b_{k}-b_{k}^{\mathrm{true}}\|_{L^{2}(\mu_{k})}

, averaged over

k

, against the number of copies

M

and the number of pairs

K

, at

N=14

; medians over 5 seeds with the range in brackets. The surrogates are those of Table 3 ; the PO-MFL surrogate uses the prior estimated from observables. A per-step method sees each pair once and sits at the Monte-Carlo floor of one displacement field whatever

K

is; the parametric fit pools the

K

residuals. The script is code/compare_regimes.py .

	total variation		$W_{1}$
coupling	$N=14$	$N=100$	$N=14$	$N=100$
drift-constrained, ${\epsilon}=0.05$	$0.033$	$0.091$	$0.023$	$0.022$
drift-constrained, ${\epsilon}={\epsilon}^{\star}$	$0.034$	$0.091$	$0.024$	$0.022$
unconstrained, ${\epsilon}=0.05$	$0.173$	$0.203$	$0.168$	$0.167$
unconstrained, ${\epsilon}={\epsilon}^{\star}$	$0.135$	$0.134$	$0.094$	$0.065$

Appendix

Table 6: Fidelity of the row-conditional laws to the true transition kernel; medians over five seeds, at

{\epsilon}=0.05

and at

{\epsilon}^{\star}

	penalised objective ( 10 )			$\\|\hat{\Theta}-\Theta_{\star}\\|$		time (s)
seed	decomposition	global QCQP	bound	decomp.	QCQP	decomp.	QCQP
3	1.2642	1.0047	1.0047	0.064	0.053	1.5	0.3
4	1.3741	1.2287	1.2287	0.139	0.147	1.2	0.2
5	1.7346	1.5045	1.5045	0.119	0.116	1.6	0.4

Appendix

Table 7: The block-coordinate decomposition next to the globally solved complete problem ( 10 ) (

N=10

K=11

d=2

a\in\mathcal{B}_{\varrho}

in both; the QCQP has 1361 variables and 684 constraints). Columns 2–4: the penalised objective of ( 10 ) at the parameter returned by each method, and the lower bound certified by spatial branch and bound. Columns 5–6: distance of the returned parameter to the truth. The two methods optimise different criteria, so the objective ordering and the accuracy ordering need not agree.

Explore similar work

Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts
Jun 26, 2026Yuanyuan Wang, Wenjie Wang, Haoxuan Li +2Identifiability Latent Dynamics
Mohamed bin Zayed University of Artificial Intelligence · The University of Melbourne · Peking University +1
Error Bounds for a Diffusion Model-Based Drift Estimator
Jun 1, 2026Ioar Casado-Telletxea, Omar RivasplataStochastic Differential Equations Score-Based Diffusion Model
Basque Center for Applied Mathematics (BCAM) · Bilbao, Spain · Centre for AI Fundamentals & Department of Computer Science +1
Generative sequence modeling for infinite memory processes via predictive states
Sep 29, 2026Michael Wieck-Sosa, Cosma Rohilla ShaliziNext-State Prediction Sequence Modeling
Department of Statistics & Data Science Carnegie Mellon University

Jun 26, 2026, cs.LGJ/K move · Enter open · S save

Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts

Yuanyuan Wang, Wenjie Wang, Haoxuan Li, Mingming Gong+1

Mohamed bin Zayed University of Artificial Intelligence · The University of Melbourne · Peking University+1

Identifiability Latent Dynamics Causal Representation Learning Stochastic Differential Equations Covariance

Open paper AskPDF arXiv

Causal representation learning for time series has developed strong identifiability results in discrete-time latent causal models, but identifiability in continuous-time latent stochastic differential equation (SDE) models remains largely open. We address this gap using environment-induced shifts in diffusion covariance. We study additive-noise latent SDEs observed through an unknown nonlinear diffeomorphism, with shared drift but environment-specific diffusion covariance. We show that two diagonal diffusion regimes with pairwise distinct coordinate-wise variance ratios identify the latent coordinates up to permutation, coordinate-wise scaling, and a possible constant shift, without any sparsity assumption on the drift. We first prove this result for linear Ornstein-Uhlenbeck systems and then extend it to general additive-noise latent SDEs. Under mild smoothness, the instantaneous drift-Jacobian causal graph is identifiable up to the same permutation. We propose a two-stage estimator for latent disentanglement and optional graph recovery; experiments on synthetic systems confirm the predicted identifiability boundary, and an application to Hardanger Bridge monitoring data illustrates the approach on real sensor trajectories.

Abstract

Explore similar work

Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts

Error Bounds for a Diffusion Model-Based Drift Estimator

Generative sequence modeling for infinite memory processes via predictive states