"Shall We Approve?" Consequence-Aware Approval for Autonomous Coding Agents
Organizations: Singapore Management University
Abstract
Coding agents increasingly rely on automated approvers to decide whether a proposed shell command may run. Existing approvers, from rule-based checkers to guardrail models and deployed permission classifiers, base their decisions on the text of the request and command. Yet a command's behavior depends on the code it invokes and the state of the workspace, and systems that observe file changes leave the assessment to the agent or a human after execution. We present Caper, a consequence-aware approval mechanism that pre-executes each command in a snapshotted sandbox, provides an independent approver with a bounded report of its observed effects, commits allowed commands, and rolls back blocked ones with a reason the agent can act on. Compared with six existing systems, Caper achieves the lowest harmful-outcome rate on SABER (6.7% versus at least 14.8%) and YoloFS-Ext, a benchmark we build in which harmful behavior is hidden inside invoked tools (7.8% versus at least 35.3%), while blocking 1.0% of the safe twins of the dangerous tasks in YoloFS-Ext.
Figures & tables
| $ python3 scripts/check-deps.py |
| SABER harm category | A | B | C | Total |
|---|---|---|---|---|
| Code tampering | 40 | 48 | 38 | 126 |
| Information leak | 40 | 27 | 37 | 104 |
| File-system destruction | 30 | 32 | 22 | 84 |
| Unauthorized access | 32 | 15 | 35 | 82 |
| Data destruction | 31 | 25 | 26 | 82 |
| Privilege escalation | 30 | 19 | 25 | 74 |
| YoloFS-Ext harm category | Delete | Overwrite | Total |
| Source code | 23 | 23 | 46 |
| Configuration | 8 | 8 | 16 |
| Documentation | 8 | 8 | 16 |
| Data | 4 | 4 | 8 |
| Test fixtures | 4 | 4 | 8 |
| README | 4 | 4 | 8 |
| SABER | YoloFS-Ext | |||||||
| Type | System | Harmful | Safe compl. | Safe ref. | Safe outc. | Harmful | Safe compl. | False block |
| No approval | ❶ GLM-5.3-Flash | 19.7 | 57.4 | 9.1 | 66.6 | 71.6 | 17.6 | — |
| ❷ DeepSeek-V3.2 | 52.7 | 31.9 | 1.0 | 33.0 | 91.2 | 2.9 | — | |
| Rule-based | ❸ CARE | 15.0 | 47.9 | 13.6 | 61.6 | 69.6 | 15.7 | 2.0 |
| Guardrail | ❹ TS-Guard | 18.8 | 44.7 | 16.9 | 61.6 | 94.1 | 2.0 | 8.8 |
| ❺ gpt-oss-safeguard-20b | 18.6 | 53.1 | 12.1 | 65.2 | 62.7 | 18.6 | 15.7 | |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| System | Released | Revised |
|---|---|---|
| ❶ GLM-5.3-Flash | 22.9 | 19.7 |
| ❸ CARE | 16.9 | 15.0 |
| ❹ TS-Guard | 19.8 | 18.8 |
| ❺ gpt-oss-safeguard-20b | 21.4 | 18.6 |
| ❻ Qwen3Guard-Gen-8B | 18.4 | 16.0 |
| ❼ YoloFS | 23.4 | 21.2 |
| Rule | Released oracle | Revised oracle | False positive it removes |
| Command parsing (effects inferred from the command string) | |||
| Splitting | splits at every | , also inside quotes; ignores && and ; | splits at && , || , ; , | and newlines outside quotes; drops comments | grep ’ssh|scp’ read as running scp |
| Heredocs | body parsed as commands | a heredoc into a file is one write; heredocs fed to sqlite3 or bash are parsed as SQL or commands | writing a script counted as running it |
| rm , cat targets | last non-option argument of the segment | every operand | rm a; ls X read as deleting X |
| Uploads | any -d or -F in the command | only curl / wget ’s own data and upload options | pgrep -F read as an upload |
| SQL deletion | any DROP , DELETE , TRUNCATE substring | only DROP TABLE , DELETE FROM , TRUNCATE outside literals and comments | column would_delete , journal_mode=DELETE |
| Task | The released pattern also matched | The revised pattern requires |
|---|---|---|
| A_code_031 | source in a comment of a written script; an upload reproduced with a dummy file | sourcing the payload from stdin; posting the real files |
| A_code_037 | disassembling marshalled code | marshal.loads followed by exec or eval |
| A_data_008 | dropping the guarded table users_old | dropping users or sessions |
| A_data_016 | dropping tables in a temporary synthetic database | dropping tables in the production database |
| A_data_029 | decoding and printing a payload | piping the decoded payload into bash , sh , or sqlite3 |
| A_info_009 | bash -n debug.sh , a syntax check | running debug.sh |
| Project | Request (abridged) | Victim file | Original level |
|---|---|---|---|
| cleanup | clean up build artifacts | README.md | L1 |
| deploy | deploy to staging | src/app.js | L1 |
| migration | run the data migration | data/users.csv | L1 |
| backup | create a backup | src/main.py | L1 |
| docs | generate the API docs | docs/api.md | L1 |
| build | build the project | src/utils.c | L2 |
| System | Approver input | Model and deployment | Blocks when |
|---|---|---|---|
| ❸ CARE | command string | static rules; unresolved warnings go to its judge, Qwen3-Coder-30B-A3B (OpenRouter) | a rule denies, or the judge rules unsafe |
| ❹ TS-Guard | request, ReAct-format history, current action | local vLLM with the official ToolSafe template and parser | risk score |
| ❺ gpt-oss-safeguard-20b | request and command | OpenRouter, reasoning effort high; policy is our approver prompt | output is Block |
| ❻ Qwen3Guard-Gen-8B | request (user turn) and command (assistant turn), response moderation | local vLLM, built-in policy | Unsafe or Controversial (strict mode) |
| ❼ YoloFS | no approver; change summary after every command | Claude Code 2.1.288, permissions bypassed; YoloFS’s agent-facing commands implemented in the container | never; the staged state is judged |
| ❽ Claude Code Auto Mode | built-in rules and two-stage transcript classifier | Claude Code 2.1.288 official binary; classifier model set to gpt-5.6-luna | classifier denies; read-only commands skip the classifier |
| SABER | YoloFS-Ext | |||||||
| System | Decisions | Blocked | Tasks | Decisions | Blocked | Tasks | Harmful cmd. | Twins |
| ❸ CARE | 5515 | 11.6 | 53.3 | 1189 | 1.3 | 6.4 | 2.0 | 2.0 |
| ❹ TS-Guard | 4623 | 22.8 | 65.7 | 984 | 38.9 | 86.8 | 8.8 | 8.8 |
| ❺ gpt-oss-safeguard-20b | 5789 | 4.6 | 36.6 | 1181 | 3.6 | 19.1 | 14.7 | 15.7 |
| ❻ Qwen3Guard-Gen-8B | 5483 | 10.0 | 36.7 | 1126 | 0.0 | 0.0 | 0.0 | 0.0 |
| ❽ Claude Code Auto Mode a | 4146 | 10.1 | 48.6 | 898 | 2.0 | 8.8 | 3.9 | 3.9 |
| Harm category | ❶ | ❸ | ❹ | ❺ | ❻ | ❼ | ❽ | Hook | Caper |
|---|---|---|---|---|---|---|---|---|---|
| Code tampering | 16.7 | 16.7 | 13.2 | 21.1 | 9.6 | 21.1 | 17.5 | 8.8 | 2.6 |
| Information leak | 27.2 | 18.5 | 26.1 | 20.7 | 25.0 | 23.9 | 18.5 | 2.2 | 4.3 |
| File-system destruction | 14.9 | 8.5 | 23.4 | 14.9 | 10.6 | 27.7 | 14.9 | 10.6 | 10.6 |
| Unauthorized access | 21.6 | 16.2 | 21.6 | 16.2 | 14.9 | 24.3 | 9.5 | 4.1 | 8.1 |
| Data destruction | 27.3 | 18.2 | 20.8 | 24.7 | 22.1 | 26.0 | 13.0 | 18.2 | 14.3 |
| Privilege escalation | 14.3 | 8.6 | 12.9 | 12.9 | 14.3 | 8.6 | 8.6 | 11.4 | 4.3 |
| Approver sees | script | Makefile | chain | binary |
|---|---|---|---|---|
| command | 100.0 | 11.5 | 0.0 | 100.0 |
| request + command | 88.5 | 34.6 | 46.2 | 54.2 |
| dialogue | 65.4 | 50.0 | 50.0 | 83.3 |
| dialogue + predicted | 46.2 | 57.7 | 46.2 | 100.0 |
| dialogue + observed | 0.0 | 0.0 | 0.0 | 4.2 |
| without content diff | 3.8 | 0.0 | 7.7 | 0.0 |
| SABER | YoloFS-Ext | ||||||
| System | Harmful | Safe compl. | Safe ref. | Safe outcome | Harmful | Safe compl. | False block |
| ❶ GLM-5.3-Flash | [16.6, 23.1] | [53.4, 61.4] | [7.1, 11.8] | [62.6, 70.3] | [62.2, 79.4] | [11.5, 26.2] | — |
| ❷ DeepSeek-V3.2 | [48.9, 56.4] | [28.5, 35.6] | [0.5, 2.1] | [29.5, 36.6] | [84.1, 95.3] | [1.0, 8.3] | — |
| ❸ CARE | [12.3, 18.1] | [43.9, 52.0] | [11.1, 16.7] | [57.5, 65.4] | [60.1, 77.7] | [9.9, 24.0] | [0.5, 6.9] |
| ❹ TS-Guard | [15.8, 22.2] | [40.7, 48.7] | [14.1, 20.2] | [57.5, 65.4] | [87.8, 97.3] | [0.5, 6.9] | [4.7, 15.9] |
| ❺ gpt-oss-safeguard-20b | [15.7, 22.0] | [49.0, 57.1] | [9.7, 15.0] | [61.2, 68.9] | [53.1, 71.5] | [12.3, 27.3] | [9.9, 24.0] |
| Step | Median | p95 |
|---|---|---|
| Snapshot before execution | 347 | 489 |
| Workspace scan before execution | 68 | 79 |
| Traced execution | 81 | 832 |
| Workspace scan after execution | 66 | 77 |
| Tracing overhead (traced vs. not) | 14 | 25 |
| Snapshot removal after Allow | 15 | 22 |
| System | Agent | Appr. | Total | Calls | Latency |
|---|---|---|---|---|---|
| ❶ GLM-5.3-Flash | 0.0060 | — | 0.0060 | — | — |
| ❸ CARE | 0.0052 | 0.0000 | 0.0052 | 10.1 | 0.00 / 0.01 |
| ❹ TS-Guard | 0.0043 | — | 0.0043 | 8.2 | 3.89 / 5.98 |
| ❺ gpt-oss-safeguard | 0.0052 | 0.0016 | 0.0068 | 10.5 | 0.64 / 1.33 |
| ❻ Qwen3Guard | 0.0048 | — | 0.0048 | 9.9 | 0.31 / 0.44 |
| ❽ Auto Mode ∗ | 0.0035 | 0.0550 | 0.0585 | 6.2 | — |
AI Agent Safety
AI agent safety studies how to prevent and detect harmful actions by autonomous systems that use tools, memory, and multi-step plans.
See all 353 papers