Organizations: Department of Industrial Engineering and Operations Research, UC Berkeley, United States. · Stoa, United States. · Department of Cybersecurity, University at Albany, SUNY, United States.
Deploying AI systems requires deciding when to delegate tasks and when humans should intervene to monitor and mitigate risk induced by AI operations. These decisions are challenging when failures cluster: a hallucination or harmful output can trigger further errors, creating periods of elevated risk. We introduce a continuous-time framework for learning adaptive human oversight under such turbulent AI risk. Existing oversight and delegation formulations condition on history but do not model incident clustering, or its suppression by supervision effort, jointly with the delegation decision and this study fixes this gap. The self-exciting dynamics capture how risk events increase the likelihood of subsequent events, making their timing and history central to decision-making. We formulate a stochastic control problem combining human actions, monitoring effort, and switching between human-AI-assisted operation and full AI delegation, balancing operational rewards against oversight costs, and cascading AI-failures and induced uncertainty. Human participation is an endogenous component of risk management: the policy determines both when oversight is needed and how much effort to allocate. We study a relaxed switching formulation and propose Hawkes-PPO, a policy-gradient method that uses a bank of exponential filters of observed incident times. In a synthetic environment it attains a higher risk-adjusted objective than either fixed regime and approaches an approximate full-information oracle. We illustrate our results with numerical simulations by examining how cascade risks influence intervention and delegation, connecting reinforcement learning with adaptive human oversight of AI systems. In particular, we illustrate the benefit of our switching strategy and Hawkes-PPO algorithm to monitor the project efficiently along time, reducing turbulent risks occurrences and costs.
Figures & tables
Figure 1: Four-method comparison. Error bars are 90% confidence intervals. Higher objective and terminal state are better; lower realized volatility indicates less quadratic variation.
Figure 2: Vertical dashed lines mark regime switches. From left to right, top to bottom: paths of X , net running reward and terminal reward, intensity of state and action turbulence (Hawkes intensities), quadratic variation of the project, regime activation, optimal switching intensity, action of the agent human or AI, human effort.
Configuration
Human time (%)
Effort (Human)
Switches
Incidents
Burst (%)
Terminal quality
Objective
Panel A. AI initial regime
Baseline
46.4
0.896
1.46
5.40
45.9
1.81
-93.61
Excitation 0.5
0.9
0.893
0.27
5.59
45.9
1.17
-85.96
Excitation 1.5
73.6
0.842
1.22
5.13
38.6
1.88
-95.85
Human cost 0.75
76.0
0.731
1.15
5.05
36.7
1.82
-89.68
Human cost 1.25
13.7
0.973
1.01
6.20
57.4
1.30
-96.33
Table 1: Supervision and project outcomes across model configurations.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Numerical choice
Parameter
Numerical choice
(T,Δ)
(8,0.05)
(X0,R0,Z0)
(0,0,0)
(a,e,ν)
[0,1]2×[0,4]
ε
0.01
δ
0.18
(σ0,σ1)
(0.16,0.04)
f(x)
0.1x+0.05tanh(x−1)
g(x)
1.2x+0.6tanh(x−1)
(k0,ka,ke)
(3.25,0.75,1.25)
kAI
1.5
(η,κ,χ)
(50,0.30,0.20)
μ
(0.35,0.25)
Appendix
Table 2: Physical, AI and simulation parameters. Kernel rows/columns follow (S,A) . Learning settings are in the appendix.
Setting
Choice
Network and optimizer
Width 64, one gated DGM block, Adam, gradient norm cap 10.
Training
6,000 updates; batch 2,048; learning rates 10−3 for updates 1–3,000, 3⋅10−4 for 3,001–4,800, and 10−4 for 4,801–6,000.
Collocation and selection
t uniform on [0,8] ; x sampled 65% from [−4,6] and 35% from [−16,16] . Residual validation on 8,192 points every 250 updates; selected update 6,000.
Exported policy
Grid [0,8]×[−12,12] , spacings Δt=Δx=0.01 , bilinear interpolation; analytic affine-tail action outside the x -range.
Table 3: Project-only recommender training and deployment.
Setting
Choice
Regime networks
Actor 46−128−128−6 ; value 46−128−128−1 , independently for each regime.
Initialization
Orthogonal weights, hidden gain 2 ; actor output gain 0.01 , value output gain 1; log-standard-deviation bias −0.7 . Switching-mean bias atanh(−0.95) ; initial a,e≈0.5 , ν≈0.1 .
Parallel sampling
1,024 environments, exactly 512 AI starts and 512 Human starts. Each reset restores the assigned initial regime.
Rollout and update
32 decisions per environment; four shuffled epochs; minibatch 4,096; Adam; actor/value gradient norm caps 1 and 5.
Budget
480 rollouts, 15,728,640 transitions, 15,360 actor and 15,360 value updates. No replay buffer or target-network soft updates.
Initial learning rates
Actor 3⋅10−4 , value 5⋅10−4 ; common schedule below.
Appendix
Table 4: Hawkes-PPO training configuration.
Method
AI start
Human start
Oracle (HJB/DGM)
-93.262 [-93.508, -93.015]
-94.230 [-94.468, -93.991]
Hawkes-PPO
-93.608 [-93.861, -93.354]
-94.676 [-94.915, -94.437]
Pure AI
-98.127 [-98.537, -97.718]
—
Pure Human (HJB/DGM)
—
-96.027 [-96.191, -95.864]
Other RL methods:
PPO
-94.954 [-95.198, -94.710]
-97.219 [-97.407, -97.031]
Appendix
Table 5: Objective values and 90% Monte Carlo confidence intervals. Each reported entry uses 4096 paths; a dash indicates an incompatible initial regime for a fixed-regime benchmark. Higher is better.
Configuration
Human time (%)
Conditional effort
Switches / path
Incidents / path
Burst prob. (%)
Terminal quality
Objective
Panel A: AI initial regime
Baseline
46.436 [45.694,47.179]
0.896 [0.893,0.898]
1.457 [1.430,1.484]
5.396 [5.335,5.458]
45.874 [44.597,47.157]
1.811 [1.787,1.835]
-93.608 [-93.861,-93.354]
Excitation 0.5
0.946 [0.857,1.034]
0.893 [0.882,0.904]
0.271 [0.251,0.291]
5.595 [5.523,5.666]
45.923 [44.645,47.206]
1.168 [1.143,1.193]
-85.962 [-86.204,-85.720]
Excitation 1.5
73.607 [73.045,74.168]
0.842 [0.838,0.846]
1.222 [1.204,1.241]
5.127 [5.068,5.187]
38.647 [37.404,39.906]
1.881 [1.856,1.905]
-95.849 [-96.069,-95.628]
Human cost 0.75
75.998 [75.463,76.533]
0.731 [0.726,0.735]
1.151 [1.135,1.166]
5.054 [4.996,5.113]
36.670 [35.441,37.917]
1.817 [1.793,1.841]
-89.677 [-89.868,-89.487]
Human cost 1.25
13.716 [13.237,14.196]
0.973 [0.972,0.974]
1.013 [0.981,1.045]
6.202 [6.126,6.277]
57.446 [56.171,58.712]
1.297 [1.271,1.323]
-96.334 [-96.669,-96.000]
Appendix
Table 6: Supervision and project outcomes under the original numerical protocol. Means appear above conditional 90% evaluation-path Monte Carlo intervals.