cs.LGOct 19, 2026

Deep Pontryagin Learning for Continuous-Time Stochastic Control with Jumps

Authors: Jean-Loup Dupret, Patrick Cheridito

Organizations: Department of Mathematics and RiskLab, ETH Zurich, Rämistrasse 101, Zurich, 8092, Switzerland

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We develop a deep Pontryagin learning framework for high-dimensional continuous-time stochastic control problems that naturally accommodates jump-diffusions. In contrast to direct policy optimization, which repeatedly differentiates the objective through the entire time-discretized dynamics, our method separates adjoint learning from policy improvement by explicitly incorporating state-sensitivity information from the Pontryagin stochastic maximum principle. For a fixed Markov control, we first compute pathwise adjoint labels by backpropagating the reward through time along simulated state trajectories, and project these labels onto the Markov state through a supervised regression step. The control is then updated by enforcing the pointwise Hamiltonian stationarity condition. We prove that the resulting projected adjoint process satisfies a discrete-time Pontryagin principle and converges, as the time step tends to zero, to the solution of the corresponding continuous-time adjoint backward stochastic differential equation. Numerical experiments illustrate the accuracy, stability, and scalability of the approach.

Figures & tables

 Γ^i(m):=νˉM2∑j=1M2γ ⁣(ti,Xi(m),ζi,j(m),aisg⁡,(m)).\vskip−6.54413pt\widehat{\Gamma}_{i}^{(m)}:=\frac{\bar{\nu}}{M_{2}}\sum_{j=1}^{M_{2}}\gamma\!\left(t_{i},X_{i}^{(m)},\zeta_{i,j}^{(m)},a_{i}^{\operatorname{sg},(m)}\right).\vskip-6.54413pt 
Algorithm 1 Deep PMP with BPTT adjoint regression
Figure 5.1 : Distribution of the terminal cost 0.5(XTα−c)20.5(X^{\alpha}_{T}-c)^{2} under the learned control via Algorithm 1 (blue) and under the optimal control ( 5.2 ) (orange).
Figure 5.2 : Comparison of the mean state trajectory (E[Xtα])0≤t≤T(\mathbb{E}[X^{\alpha}_{t}])_{0\leq t\leq T} (left) and of a given state trajectory (Xtα)0≤t≤T(X^{\alpha}_{t})_{0\leq t\leq T} (right) under the learned control via Algorithm 1 (blue), via the DPO method (orange) and under the optimal control ( 5.2 ) (dashed green).
Figure 5.3 : Left: Cost functional J(α)J(\alpha) as a function of the training epoch kk , evaluated using the learned control from Algorithm 1 (blue), the DPO method (orange), and the closed-form optimal control ( 5.2 ) (dashed line). Right: MAEα\rm{MAE}_{\alpha} between the learned Deep PMP control (blue) and the learned DPO control (orange). Solid curves represent averages over 10 independent runs, and shaded regions indicate ±\pm one standard deviation.
Figure 5.4 : Left: Portfolio weights πi(t0,x0)\pi_{i}(t_{0},x_{0}) for t0=0,x0=1t_{0}=0,x_{0}=1 and the first 25 assets i=1,…,25i=1,\ldots,25 . Right: consumption rate c(t,x0)c(t,x_{0}) at x0=1x_{0}=1 in function of time t∈[0,1]t\in[0,1] . Comparison of the deep PMP (blue), the DPO (orange) and the reference solution ( 5.7 )–( 5.8 ) (dashed green).
Figure 5.5 : Left: Distribution of the reward in ( 5.6 ) under the learned deep PMP control (blue) and under the reference control ( 5.7 )–( 5.8 ) (orange). Right: Comparison of the mean state trajectory (E[Xtα])0≤t≤T(\mathbb{E}[X^{\alpha}_{t}])_{0\leq t\leq T} under the learned control via the deep PMP (blue), the DPO method (orange) and under the reference control (dashed green).
Figure 5.6 : Left: Reward functional J(α)J(\alpha) ( 5.6 ) as a function of the training epoch kk , evaluated using the learned control from Algorithm 1 (blue), the DPO method (orange), and the optimal control ( 5.7 )–( 5.8 ) (green dashed line). Right: MAEα\rm{MAE}_{\alpha} between the learned Deep PMP control (blue) and the learned DPO control (orange). Solid curves represent averages over 10 independent runs, and shaded regions indicate ±\pm one standard deviation
Figure 5.7 : Distribution of the quadratic cost in ( 5.14 ) under the learned control via the deep PMP algorithm 1 (blue) and under the Riccati control ( 5.15 ) (orange).
Figure 5.8 : Comparison of the mean state norm trajectory (E[∣Xtα∣])0≤t≤T(\mathbb{E}[|X^{\alpha}_{t}|])_{0\leq t\leq T} (left) and of a given state norm trajectory (∣Xtα∣)0≤t≤T(|X^{\alpha}_{t}|)_{0\leq t\leq T} (right) under the learned control via Algorithm 1 (blue), the DPO method (orange) and under the Riccati control ( 5.15 ) (right).
Figure 5.9 : Left: Cost functional J(α)J(\alpha) ( 5.14 ) as a function of the training epoch kk , evaluated using the learned control from Algorithm 1 (blue), the DPO method (orange), and the Riccati control ( 5.15 ) (dashed line). Right: MAEα\rm{MAE}_{\alpha} between the learned Deep PMP control (blue) and the learned DPO control (orange). Solid curves represent averages over 10 independent runs, and shaded regions indicate ±\pm one standard deviation
Figure 5.10 : Distribution of the objective reward in ( 5.19 ) under the learned PMP control via Algorithm 1 (blue) and under the optimal control ( 5.24 ) (orange).
Figure 5.11 : Comparison of the mean state norm trajectory (E[∣Xtα∣])0≤t≤T(\mathbb{E}[|X^{\alpha}_{t}|])_{0\leq t\leq T} (left plot) and the mean first state coordinate trajectory (E[Xtα,0])0≤t≤T(\mathbb{E}[X^{\alpha,0}_{t}])_{0\leq t\leq T} under the learned control via Algorithm 1 (blue), the DPO method (orange) and under the optimal control ( 5.24 ) (green), with X0(m)=0X_{0}^{(m)}=0 .
Figure 5.12 : Left: Reward functional J(α)J(\alpha) ( 5.19 ) as a function of the training epoch kk , evaluated using the learned control from Algorithm 1 (blue), the DPO method (orange), and the optimal control ( 5.24 ) (green dashed line). Right: MAEα\rm{MAE}_{\alpha} between the learned Deep PMP control (blue) and the learned DPO control (orange). Solid curves represent averages over 10 independent runs, and shaded regions indicate ±\pm one standard deviation
Figure 5.13 : Distribution of the terminal reward in ( 5.29 ) under the learned control via Algorithm 1 (blue) and under the optimal control ( 5.33 ) (orange).
Figure 5.14 : Comparison of the mean state norm trajectory (E[∣Xtα∣])0≤t≤T(\mathbb{E}[|X^{\alpha}_{t}|])_{0\leq t\leq T} (left plot) and the mean first state coordinate trajectory (E[Xtα,0∣])0≤t≤T(\mathbb{E}[X^{\alpha,0}_{t}|])_{0\leq t\leq T} under the learned control via Algorithm 1 (blue), the DPO method (orange) and under the optimal control ( 5.33 ) (green), with X0(m)=0X_{0}^{(m)}=0 .
Figure 5.15 : Left: Reward functional J(α)J(\alpha) ( 5.29 ) as a function of the training epoch kk , evaluated using the learned control from Algorithm 1 (blue), the DPO method (orange), and the optimal control ( 5.33 ) (green dashed line). Right: MAEα\rm{MAE}_{\alpha} between the learned Deep PMP control (blue) and the learned DPO control (orange). Solid curves represent averages over 10 independent runs, and shaded regions indicate ±\pm one standard deviation
More in this topic

Stochastic Optimal Control

Stochastic optimal control studies how to choose actions over time when system dynamics, observations, or outcomes are uncertain.

See all 101 papers

Explore similar work

CardsList

  1. A Continuous-Time Actor-Critic Framework for Jump-Diffusion Controls with Normalizing Flows

    Apr 7, 2026Liya Guo, Ruimeng Hu, Xu Yang +1Stochastic Optimal ControlActor-Critic Methods

    Department of Mathematics, Tsinghua University, Beijing 100084, China · Department of Statistics and Applied Probability, University of California, Santa Barbara, CA 93106-3080, USA · Department of Mathematics, University of California, Santa Barbara, CA 93106-3080, USA +1

  2. Adaptive Learning via Off-Model Training and Importance Sampling for Fully Non-Markovian Optimal Stochastic Control. Complete version

    Apr 14, 2026Dorival Leão, Alberto Ohashi, Simone Scotti +1Stochastic Optimal ControlAdaptive Learning

  3. Learning-Based Stochastic Optimal Control with Infinite-Horizon Probabilistic Constraints

    Aug 2, 2026Francesco Cordiano, Kanghui He, Bart De SchutterChance-Constrained OptimizationMarkov Decision Processes

    Delft Center for Systems and Control, Delft University of Technology, 2628 CD, Delft, The Netherlands · Department of Engineering Science, University of Oxford, OX1 3PJ Oxford, U.K.