Options market makers absorb client order flow and must hedge the resulting inventory risk under trading frictions. We study whether an option hedging reinforcement learning (RL) agent trained on historical market data outperforms three alternatives in tail-risk protection: (i) traditional delta hedging, (ii) simple hedge-ratio rules, and (iii) the same agent trained in simulated markets. Most RL hedging agents are trained in model-generated markets that do not capture the crashes and volatility regimes present in real data. We introduce an empirical replay environment using S&P 500 option data from 1996 to 2017 and compare against a matched geometric Brownian motion (GBM) environment, a stochastic-volatility (SABR) simulator, and a hybrid that values historical paths with Black-Scholes, all sharing the same portfolio mechanics, stochastic arrivals, costs, and test period. We train a distributional Twin Delayed Deep Deterministic Policy Gradient (TD3) agent and evaluate it out of sample over 2018-2023. The agent trained on empirical data reduces Conditional Value-at-Risk (CVaR) tail loss by 3.1% compared to delta hedging and outperforms every hedge-ratio rule chosen during the training period and every alternative training environment. The reduction reaches 8.9% in volatile and falling markets, but reverses to a cost of 7-10% in rising markets. Empirical training induces a state-dependent short net delta: the policy under-hedges negative-delta books, most strongly after prices have fallen. This tilt disappears with GBM training and is weaker with SABR or hybrid training. These findings suggest that exposure to historical market dynamics can materially change a tail-risk hedging policy, although the evidence rests on a single test path and the edge is regime-conditional.
Figures & tables
Component
Empirical replay
Hybrid
GBM / SABR
Training paths
Historical S&P 500, 1996–2017
As empirical
Simulated on the same normalized price scale (SABR: stochastic volatility)
Option values and Greeks
SPX mid-quotes with OptionMetrics implied volatility and Greeks
Black–Scholes at a constant volatility of 0.19
Black–Scholes at the simulated volatility
Client contracts
20 – 60 -day contracts sampled from the historical SPX chain
As empirical
Near-the-money, 20 – 60 -day maturity
Table 1. Training-environment design (shared settings in Section 3.1 ).
Regime
Net δ
RL
Delta
Impr.
Impr. %
Q1 HV ↑
−3.4
−134.6
−122.5
−12.1
−9.9%
Q2 HV ↓
−7.1
−184.0
−201.9
+17.9
+8.9%
Q3 LV ↓
−7.5
−64.6
−67.6
+3.0
+4.4%
Q4 LV ↑
−5.0
−54.7
−50.9
−3.8
−7.4%
All
−5.7
−77.7
−80.2
+2.5
+3.1%
Table 2. Net delta and CVaR 95 by market regime (empirical TD3, five training runs pooled).
Rule (parameter)
CVaR 95
Q2
Q4
CVaR 99
Mean
Net δ
Constant ratio ( a=0.97 )
+1.1
+4.9
−0.0
+2.8
+0.21
−0.8
Sign rule ( k=0.02 )
+0.0
+2.1
−1.5
+0.9
−0.49
−2.7
Two-parameter ( 0.96 , 1.00 )
+0.8
+5.5
−1.4
+2.9
−0.35
−3.2
Short offset ( m=5 )
+1.1
+8.7
−2.7
+4.5
−0.81
−5.0
No-trade band ( b=2 )
−0.0
−0.1
+0.0
−0.0
+0.01
0.0
Band around 0.95δ ( b=5 )
+1.3
+7.5
−0.5
+3.9
+0.41
−1.4
Table 3. Hedge-ratio rules chosen on 1996–2017 and evaluated on 2018–2023: change vs. delta hedging.
Net δ by regime
CVaR 95 Impr. %
Training
Q1
Q2
Q3
Q4
All
Mean
Range
Empirical
−3.5
−7.1
−7.7
−5.4
−6.1
+3.3%
[+2.9,+3.8]
GBM
+0.4
+0.4
+0.5
+0.5
+0.5
−1.0%
[−1.5,−0.1]
GBM (oracle σ )
+0.0
+0.0
+0.1
+0.1
+0.1
+0.0%
[−1.2,+0.8]
SABR
−3.3
−4.9
−4.8
−4.1
−4.3
+1.1%
[+0.3,+1.7]
Hybrid
−2.6
−2.6
−3.0
−2.8
−2.8
+1.9%
[+1.5,+2.2]
Table 4. Training-environment comparison (TD3, CVaR 95 , three training runs each).
Figure 1. State-dependent net exposure (empirical TD3, CVaR 95 , five runs pooled): mean hedge ratio (left) and net delta (right) by client-book delta and price decile. Heatmaps showing the learned hedge ratio and net delta across client-book delta and price deciles.
Objective
Net δ
Impr.
Impr. %
Std.
Range
CVaR 95
−5.7
+2.5
+3.1%
0.4
[+2.0,+3.1]
VaR 95
−2.5
+0.5
+1.0%
0.1
[+0.4,+0.6]
VaR 99
−10.4
+0.4
+0.4%
1.5
[−0.6,+2.0]
Table 5. Robustness across risk objectives (each policy evaluated on its own objective).
Figure 2. Stress-period paths for empirical TD3 (CVaR 95 ): COVID-19 crash (Q2, top) and 2022 selloff (Q3, bottom). Price, net delta, and cumulative PnL for empirical TD3 and delta hedging during two held-out stress episodes.
Quantitative finance uses mathematical models, statistics, and machine learning to value assets, measure risk, and guide trading and portfolio decisions.
Concordia University, Department of Mathematics and Statistics, Montréal, Canada · Quantact Laboratory, Centre de Recherches Mathématiques, Montréal, Canada