cs.AISep 25, 2026
SaveContract monitoring: governing AI via separation of powers
Organizations: UPenn
Abstract
We propose an AI safety framework that binds worker agents to contracts specifying their permitted actions. We show how these contracts can be enforced and specified by assigning distinct responsibilities to monitor agents and judges, and asymmetric computational resources to monitors and workers. Our framework allows us to empirically measure statistical safety guarantees. The framework applies to a wide range of settings, including code security and escape-the-box scenarios.
Figures & tables
Figure 2: Training and deploying contract monitors with guarantees . Whenever a contract is specified by a report judge, we can train a monitor agent with self-play against an adversarial worker whose goal is to fool the monitor without being caught, and self-report (left) . Because of the validation phase, the trained monitor can be successfully deployed against workers trained with up to a certain bound on computational resources (right) .
| Framework | How compliance is pursued | Remaining limitation |
|---|---|---|
| Constitutional AI [ 8 ] | Worker is trained to complete the task while following written principles. | No independent enforcement: the worker may violate principles when pursuing task reward. |
| (Direct) monitoring [ 9 ] | External monitor assesses safety independently of the worker’s task performance. | Monitor evasion: satisfying the monitor’s judgment need not satisfy the intended safety boundary. |
| Contract monitoring (ours) | Fixed report judge specifies violations; a separate monitor is trained through self-play to find violation witnesses. | Contract misspecification remains. Evasion risk is bounded due to scaling laws for monitor self-play. |
Table 1: Conceptual progression in the separation of roles. Direct monitoring introduces oversight independent of the worker; contract monitoring further separates contract specification from enforcement and enables training against independently verifiable instances of monitor evasion.
| Framework | Burden of proof | Enforcement challenge | Complexity analogy |
|---|---|---|---|
| Verifiable AI [ 11 , 12 ] | Worker supplies evidence of safety or correctness. | Find accepted certificates; justify verifier soundness. | |
| Contract monitoring (ours) | Monitor supplies evidence of a contract violation. | Search for violations; justify report judge completeness. | |
| AI debate [ 13 ] | Proponent defends a claim against counterarguments. | Learned play may differ from intended quantified contract. |
Table 2: Complexity-theoretic comparison of proof obligations in complementary frameworks. Contract monitoring is to as verifiable AI is to and fixed-round AI debate is to levels of .
More in this topic
AI Agent Safety
AI agent safety studies how to prevent and detect harmful actions by autonomous systems that use tools, memory, and multi-step plans.
See all 330 papers