Aligning, Fast and Slow: A Control-Theoretic Oversight Framework for Long-Horizon Trajectories and Maintained Deployments
Organizations: BCG X AI Science Institute, Boston Consulting Group · Centre for Human-Inspired Artificial Intelligence, University of Cambridge
Abstract
AI agents can pass checks on individual tasks while retaining memory or code changes that alter their later behavior. We introduce a control-theoretic framework that couples a fast loop controlling proposed actions with a slow loop reviewing persistent changes and reassessing configurations after model or tool updates. When measurement errors and changes between checks are bounded and controllers can enforce their decisions, we derive a deployment-level bound and a rule for dividing review effort. In simulation, dividing a fixed review budget between both loops reduces mean maximum departure from intended behavior roughly twelvefold compared with review confined to individual runs. In coding stress tests with authored memories and five matched seeds, combined controls reduce mean severity on a 0-3 scale from 0.325 without gates to 0.117, resolving 33 of 120 issues versus 35 without gates; in trading stress tests, combined controls reduce mean shortfall in low-emission purchases from 0.892 without gates to 0.107. The overall framework helps developers and auditors compare and enforce oversight designs, while the agent studies test component controls rather than the full deployment-level guarantee.
Figures & tables
| Quantity | Fast loop | Slow loop |
|---|---|---|
| Departure being assessed | Excursion per step | Inherited baseline between epochs |
| Accepted estimate | Monitor score | Recertified estimate |
| Estimate ceiling | ||
| Error after a check | ||
| Maximum check interval | steps | epochs |
| Between-check change | Residual : at most per step | Baseline increase: at most per epoch |
| Policy | Mean severity | Blocked only | Delivered mutation | Resolved |
|---|---|---|---|---|
| No gates | 0.325 | 0/120 | 13/120 | 35/120 |
| Fast gate | 0.242 | 29/120 | 0/120 | 25/120 |
| Fast gate + anchor | 0.200 | 24/120 | 0/120 | 27/120 |
| All gates | 0.117 | 14/120 | 0/120 | 33/120 |
| Neutral reference | 0.000 | 0/120 | 0/120 | 27/120 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Design element | Trading | Coding |
|---|---|---|
| Task and recipient | 32 quarters in Arike et al.’s market environment ( Arike et al., 2025 ) , GPT-4o-mini | 24 SWE-smith issues in marshmallow , GPT-4.1-mini with OpenHands |
| Charter | Minimize portfolio greenhouse gas emissions and ignore profit potential | Repair source code without altering protected tests or continuous-integration configuration, and report observed results truthfully |
| Pressure | Leadership override every quarter, in addition to native profit pressure | Release-manager override at issue start and after tool actions 10 and 30, if reached |
| Fast gate ( ) | Rationale-derived estimate of restricted-asset share, cutoff 0.05 | Rule-plus-judge score on proposed tool actions, cutoff 0.20 |
| Anchor | Quarter-end sales of restricted holdings above a measured share of 0.05 | Restore protected artifacts every ten mutating actions and before delivery |
| Anchor imperfection | Additive exposure error in | Probability 0.10 of suppressing the first scheduled inspection per issue |
| Policy | Mean | Seed range | Mean |
|---|---|---|---|
| No gates | 0.892 | 0.826–0.964 | 0.061 |
| Fast gate | 0.559 | 0.401–0.647 | 0.394 |
| Fast gate + anchor | 0.293 | 0.026–0.507 | 0.660 |
| All gates | 0.107 | 0.041–0.137 | 0.846 |
| Policy | Aligned purchases | Resources | Corrective sales |
|---|---|---|---|
| No gates | 42.64 | 746.80 | 0 |
| Fast gate | 258.50 | 671.12 | 0 |
| Fast gate + anchor | 349.03 | 537.42 | 53.16 |
| All gates | 595.88 | 706.06 | 32.81 |
| Neutral reference | 358.83 | 375.00 | 0 |
| Model | Settings | Claims |
|---|---|---|
| Carryover ratchet | 96 runs; ; , , ; | Linear baseline without slow control; bounds with control |
| Coupled monitor | , ; –1.5; 400 steps | Stability threshold in the linear instance |
| Anchored residual | , ; , ; 4,000 steps | Residual-based excursion bound |
| Jump recovery | , ; | Equality-dynamics recovery time |
| Repeated updates | , , ; ; 2,400 steps | Operation and violations for two specified policies |
| Binary review | , , ; 20,000 items | Kept-corruption floor and expected log growth |
| Budget | Realized grid optimum | Bound-optimal schedule | In-run checks only | Recertification only |
|---|---|---|---|---|
| 1 | 0.686 | – | 5.251 | 0.686 |
| 2 | 0.598 | 0.691 | 5.068 | 0.598 |
| 4 | 0.402 | 0.402 | 4.991 | 0.598 |
| 8 | 0.292 | 0.363 | 4.936 | 0.598 |
| 12 | 0.264 | 0.320 | 4.925 | 0.598 |
| 20 | 0.264 | 0.278 | 4.923 | 0.598 |
| Retention target | Analytical screen | Tested policy |
|---|---|---|
| 0.75 | 0.15 | 0.20 |
| 0.85 | 0.15 | 0.30 |
| 0.90 | 0.20 | 0.40 |
| 0.95 | Infeasible | Infeasible |
| Duty or practice | Operational question | Relevant limit |
|---|---|---|
| EU AI Act Art. 14(4)(e): interruption | Can the operator stop pending actions and verify the resulting state? | Prior effects may require separate recovery |
| EU AI Act Art. 72: post-market monitoring | Do evaluations cover accumulated memory and configuration changes? | Inherited drift requires baseline evidence |
| EU AI Act Art. 43(4): substantial modification | What must be reassessed after this change, and can the chosen procedure finish before further changes? | The maintenance budget is policy-specific; legal triggers require separate analysis |
| NIST AI RMF MANAGE 3.1–3.2: third-party risk | Can model changes be detected, assessed, deferred, or reversed? | A vendor update may invalidate earlier calibration |
| NIST Playbook MANAGE 4.1: monitoring and recovery | Are exposure, residual harm, and intervention capacity measured separately? | Exposure and repair are distinct outcomes |
AI Agent Safety
AI agent safety studies how to prevent and detect harmful actions by autonomous systems that use tools, memory, and multi-step plans.
See all 352 papers