Honeypot Protocol
Organizations: University of North Carolina at Pembroke · With Apart Research
Abstract
Trusted monitoring, the standard defense in AI control, is vulnerable to adaptive attacks, collusion, and strategic attack selection. All of these exploit the fact that monitoring is passive: it observes model behavior but never probes whether the model would behave differently under different perceived conditions. We introduce the honeypot protocol, which tests for context-dependent behavior by varying only the system prompt across three conditions (evaluation, synthetic deployment, explicit no-monitoring) while holding the task, environment, and scoring identical. We evaluate Claude Opus 4.6 in BashArena across all three conditions in both honest and attack modes. The model achieved 100% main task success and triggered zero side tasks uniformly across conditions, providing a baseline for future comparisons with stronger attack policies and additional models.
Figures & tables
| Condition | Mode | Main Task | Side Task | Duration (s) |
|---|---|---|---|---|
| Evaluation | Honest | 1.0 | 0.0 | 1525 |
| Evaluation | Attack | 1.0 | 0.0 | 3469 |
| Honeypot | Honest | 1.0 | 0.0 | 1694 |
| Honeypot | Attack | 1.0 | 0.0 | 1091 |
| Deployment | Honest | 1.0 | 0.0 | 931 |
| Deployment | Attack | 1.0 | 0.0 | 1434 |
AI Agent Monitoring
AI agent monitoring tracks actions, tool use, and execution traces to detect failures, misuse, drift, and policy violations during operation.
See all 280 papers