math.OCNov 8, 2026

CVaR Deep Optimal Stopping

Authors: Yechan Jeong, Youngjoon Hong

Organizations: Seoul National University

Abstract

Risk-sensitive optimal stopping is useful when poor stopping outcomes carry substantial costs, as in option exercise, sequential medical treatment, and safety-critical control. A natural objective is Conditional Value at Risk (CVaR) of the stopped reward. We develop a theoretical framework and algorithm for CVaR deep optimal stopping. The framework and algorithm address four difficulties in applying risk-sensitive reinforcement learning to optimal stopping: state augmentation, score-function pathology, blindness to success, and dependence on auxiliary networks. We isolate three structural properties of optimal stopping: single-point reward, action-independent dynamics, and sum-zero gradient. We exploit these properties to establish Markov optimality without state augmentation and derive a pathwise gradient estimator. Under bounded policy derivatives, the pathwise estimator has bounded second moments near deterministic policies, avoiding the divergence that can occur with score-function estimators. Building on this framework, we propose DIOS (Distribution-Informed Optimal Stopping) to address the remaining two difficulties. A per-time-step soft penalty combined with tail-level annealing mitigates blindness to success. Estimating the reward quantile directly from each batch removes the need for an auxiliary critic. For a fixed tail level, we establish a finite-time stationarity guarantee under regularity assumptions along the training iterates. Across the 24 main Bermudan max-call configurations, DIOS achieves the highest mean CVaR and the shortest recorded training time among the tested CVaR baselines.

Figures & tables

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Continuous-time Optimal Stopping through Deep Reinforcement Learning

    Jun 16, 2026Cosmin Borsa, Michael LudkovskiQuantitative FinanceSequential Decision Making

  2. Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity

    Feb 3, 2026Aneri Muni, Vincent Taboga, Esther Derman +2Markov Decision ProcessesQ-Learning

  3. Deep catastrophic-risk-aware reinforcement learning

    Oct 19, 2026Carlos Octavio Pérez Mendoza, Frédéric GodinRisk-Sensitive RLConditional Value-at-Risk