Conservative Continuous-Time Treatment Optimization

Nora Schneider; Georg Manten; Niki Kilbertus

cs.LGMar 17, 2026

Conservative Continuous-Time Treatment Optimization

Authors: Nora Schneider, Georg Manten, Niki Kilbertus

Organizations: Technical University of Munich · Helmholtz Munich · Munich Center for Machine Learning

Abstract

We develop a conservative continuous-time stochastic control framework for treatment optimization from irregularly sampled patient trajectories. The unknown patient dynamics are modeled as a controlled stochastic differential equation with treatment as a continuous-time control. Naive model-based optimization can exploit model errors and propose out-of-support controls, so optimizing the estimated dynamics may not optimize the true dynamics. To limit extrapolation, we add a consistent signature-based MMD regularizer on path space that penalizes treatment plans whose induced trajectory distribution deviates from observed trajectories. The resulting objective minimizes a computable upper bound on the true cost. Experiments on benchmark datasets show improved robustness and performance compared to non-conservative baselines.

Figures & tables

Figure 1: Effect of increasing regularization strength

\lambda

. (Left) Improvement of the ground truth costs for

\lambda=\{1,10,100\}

relative to the ground truth costs with no regularization, i.e.,

\lambda=0

on three optimization problems across

15

different initial conditions (Cancer (Explicit), Cancer (Relative), Covid-19 (Target Trajectory). Boxplots show medians and interquartile ranges (IQR). Whiskers extend to the farthest point within

1.5\times\text{IQR}

from boxes. (Right) Example of predicted optimal treatment plans for

\lambda\in\{0,1,10,100\}

together with treatment plans seen during training and ground truth optimal treatment plan.

Table 1: Benchmarking the optimized treatment plans on all three tasks; Cancer (Explicit), Cancer (Relative) and Covid-19 (Target Trajectory).

Figure 2: Example patient trajectories under treatment plans optimized with different conservatism levels (

\lambda

). Tumor volume over time for two optimized treatment plans. Shaded regions indicate variability across simulated trajectories, the bold curve denotes the mean trajectory, and the dot marks the mean final tumor volume.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$I$	$I:=[t_{s},t_{f}]$ , the considered finite time horizon
$N$	the number of training samples
$\mathcal{D}$	the dataset $\mathcal{D}:=\{(z^{(i)},y^{(i)},u^{(i)})\}_{i\in[N]}$
$d_{z}$	dimension of the stochastic covariate process $Z:\Omega\times I\rightarrow\mathbb{R}^{d_{z}}$
$d_{y}$	dimension of the stochastic outcome process $Y:\Omega\times I\rightarrow\mathbb{R}^{d_{y}}$
$d_{u}$	dimension of the stochastic treatment process $U:\Omega\times I\rightarrow\mathbb{R}^{d_{u}}$

Appendix

Table 2: Summary of notation

Panel of Figure 3 — Figure 3: Effects of regularization strength parameter $\lambda$ .

Explore similar work

Interaction-Limited Safe Continuous-Time RL for Dynamical Medical Treatment
May 31, 2026Yuepeng Wang, Ken Kawano, Akifumi Wachi +13Reinforcement Learning Sequential Decision Making
Tokyo University of Agriculture and Technology · Institute of Science Tokyo · LY Corporation +6
Trajectory-Regularized Stochastic Optimal Control via KL Divergence
Jul 24, 2026Mintae Kim, Koushil SreenathStochastic Optimal Control
Hybrid Robotics, UC Berkeley, CA 94720, USA
Adaptive Learning via Off-Model Training and Importance Sampling for Fully Non-Markovian Optimal Stochastic Control. Complete version
Apr 14, 2026Dorival Leão, Alberto Ohashi, Simone Scotti +1Stochastic Optimal Control Adaptive Learning

May 31, 2026J/K move · Enter open · S save

Interaction-Limited Safe Continuous-Time RL for Dynamical Medical Treatment

Yuepeng Wang, Ken Kawano, Akifumi Wachi, Yongqi Zhou+12

Tokyo University of Agriculture and Technology · Institute of Science Tokyo · LY Corporation+6

Reinforcement Learning Sequential Decision Making Offline RL Continuous-Time RL Safe RL

Open paper AskPDF arXiv

Dynamic medical treatment requires deciding treatment intensity and intervention timing, while patient states evolve continuously and adverse events may occur between clinical interactions. Most existing treatment learning methods assume fixed schedules or enforce safety only at discrete decision points. We propose Interaction-Limited Safe Continuous-Time Reinforcement Learning, a framework that jointly optimizes treatment administration and clinical interaction timing under trajectory-level safety constraints. Our key idea is to reformulate the continuous time treatment problem as an option-based semi-Markov decision process, where each option specifies a continuous-time treatment policy and its duration. We develop a safety-tightening mechanism showing that suitably constructed constraints at interaction times guarantee safety over the full continuous-time trajectory with high probability. We further establish finite-sample guarantees for policy learning from logged treatment trajectories and introduce a practical data-driven conservative surrogate. Experiments show that the proposed adaptive interaction-timing mechanism improves both safety and treatment effectiveness over equidistant interaction schemes across different safe policy optimization methods.

Abstract

Explore similar work

Interaction-Limited Safe Continuous-Time RL for Dynamical Medical Treatment

Trajectory-Regularized Stochastic Optimal Control via KL Divergence

Adaptive Learning via Off-Model Training and Importance Sampling for Fully Non-Markovian Optimal Stochastic Control. Complete version