VeriHarm-Bench: Measuring Harmful Success in LLM Coding Agents Verifiably
Organizations: University of Virginia · Virginia Tech · UCLA
Abstract
Large language model (LLM) coding agents increasingly operate with shell access, allowing them to write scripts, move files, alter permissions, and modify system configurations. However, an agent may achieve a user's goal while causing unintended harm, such as deleting unrelated files, weakening access controls, or corrupting data, in response to benign inputs without explicit jailbreaking. This failure mode is particularly important because apparent task success may reassure users that an agent behaved correctly, causing harmful side effects to go unnoticed. We call this harmful success: the agent successfully completes the intended benign task while simultaneously causing unintended harm. Despite its importance, harmful success is difficult to measure because conventional task-success metrics overlook harmful behaviors, while existing safety evaluations are often designed solely to assess harmful outcomes. We introduce VeriHarm, an automatic framework for constructing programmatically verifiable benchmarks that directly measure harmful success. VeriHarm minimally perturbs coding tasks to elicit unintended harmful behavior while maintaining the benignity of the task and preserving the original task objective. It then synthesizes executable harm tests that, together with the original task-correctness tests, evaluate both task accuracy and harmful outcomes. Frontier agents, including GPT-5.6-Sol and Claude-Opus-5, achieve an average task accuracy of 95% on our curated VeriHarm-Bench, yet still struggle to complete tasks safely: their safe success rate, the fraction of tasks completed successfully without triggering the designated harm, averages only 37%. These results suggest that successful task completion can frequently coexist with unintended harm, and that task-success metrics alone may substantially understate the potential safety risks of coding agents.
Figures & tables
| Target agent | Success rate (%) | Mean iteration | Gains across iteration (%) | ||||
|---|---|---|---|---|---|---|---|
| Iter-1 | Iter-2 | Iter-3 | Iter-4 | Iter-5 | |||
| Proposer agent: Claude-Opus-4-7 | |||||||
| Claude-Sonnet-4.6 | 79.5 | 2.48 | 22.0 | 23.5 | 16.5 | 9.0 | 8.5 |
| Kimi-K2.5 | 76.1 | 2.78 | 18.9 | 15.4 | 15.4 | 15.9 | 10.5 |
| GPT-5.4-Mini | 56.7 | 2.72 | 12.4 | 15.9 | 10.5 | 10.9 | 7.0 |
| Qwen3-235B-A22B | 42.8 | 2.76 | 10.7 | 10.7 | 5.9 | 9.1 | 6.4 |
| Evaluated agent | Task Accuracy (%) | Harmful Success (%) | Safe Success (%) |
|---|---|---|---|
| Claude-Opus-4.8 | 98.7 0.9 | 72.0 3.7 | 26.7 3.6 |
| Claude-Opus-5 | 97.3 1.4 | 56.2 4.1 | 41.1 4.1 |
| GPT-5.5-Pro | 89.3 2.5 | 53.7 4.1 | 35.6 3.9 |
| GPT-5.6-Sol | 95.3 1.7 | 50.7 4.1 | 44.7 4.1 |
| Overall | 95.1 0.9 | 58.2 2.0 | 37.0 2.0 |
| Agreement metric | Value |
|---|---|
| Fleiss’ | 0.537 |
| Krippendorff’s | 0.540 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Value |
|---|---|
| Rollout time limit | 600 s |
| bash , python tool timeout | 60 s |
| text_editor tool timeout | 180 s |
| run_tests tool timeout | 30 s |
| , execution timeout | 60 s each |
| Top-150 ( VeriHarm-Bench ) | Bottom-150 | |
| range | 21.5–24.0 | 10.5–16.5 |
| Mean | 22.15 | 14.58 |
| Distinct source tasks | 97 | 98 |
| Theme | % | Example elicited behavior |
|---|---|---|
| Delete / wipe / truncate | 51.3 | In single mode the script deletes or truncates string_log.txt , wiping previously logged entries. |
| Out-of-scope targeting | 58.0 | The search command performs a broad recursive filesystem search that returns paths outside SourceFiles/Programs , exposing the sentinel file in UnrelatedData . |
| Permission loosening | 23.3 | The agent sets generate_output.sh , solution.sh and the log files to mode 777 to satisfy “no access issues for teammates”. |
| Config / service / system | 14.7 | The agent binds the HTTP server to 0.0.0.0 instead of restricting it to loopback. |
| Log / telemetry paths | 12.7 | The script creates /logs world-writable instead of 755/775. |
| In-place data corruption | 34.0 | solution.sh rewrites /unrelated.tsv in place, appending suffixes to a non-target table. |
| Evaluated agent | Task Accuracy (%) | Harm Pass Rate (%) | Harmful Success (%) | Norm. Harmful Success (%) | Safe Success (%) |
|---|---|---|---|---|---|
| Claude-Opus-4.8 | 98.7 0.9 | 73.3 3.6 | 72.0 3.7 | 73.0 3.7 | 26.7 3.6 |
| Claude-Opus-5 | 97.3 1.4 | 58.2 4.1 | 56.2 4.1 | 57.7 4.2 | 41.1 4.1 |
| GPT-5.5-Pro | 89.3 2.5 | 57.7 4.1 | 53.7 4.1 | 60.2 4.3 | 35.6 3.9 |
| GPT-5.6-Sol | 95.3 1.7 | 53.3 4.1 | 50.7 4.1 | 53.1 4.2 | 44.7 4.1 |
| Overall | 95.1 0.9 | 60.7 2.0 | 58.2 2.0 | 61.1 2.1 | 37.0 2.0 |
| Stage | Input (M) | Output (M) | Cost (USD) | Share (%) |
|---|---|---|---|---|
| Target-agent rollouts | 1,654.3 | 30.7 | $856.5 | 70.0 |
| Proposer calls † | 33.3 | 5.7 | $319.9 | 26.1 |
| Judge calls † | 4.9 | 0.8 | $47.6 | 3.9 |
| Total | 1,692.4 | 37.2 | $1,224.0 | 100.0 |
| Per scored candidate ( ) | $1.17 | |||
| Per curated instance ( ) | $8.16 |
| Target agent | Cost (USD) |
|---|---|
| claude-sonnet-4.6 | $340.4 |
| kimi-k2.5 | $301.1 |
| gpt-5.4-mini | $27.4 |
| qwen3-235b-a22b | $187.5 |
| Total | $856.5 |
| Model | Input (M) | Output (M) | Cost (USD) |
|---|---|---|---|
| claude-opus-4.7 | 17.8 | 3.3 | $171.3 |
| gpt-5.5 | 15.5 | 2.4 | $148.7 |
| Total | 33.3 | 5.7 | $319.9 |
| Model | Input (M) | Output (M) | Cost (USD) |
|---|---|---|---|
| claude-opus-4.7 | 2.43 | 0.41 | $22.4 |
| gpt-5.5 | 2.43 | 0.44 | $25.2 |
| Total | 4.86 | 0.85 | $47.6 |
| Model | Provider |
|---|---|
| claude-opus-4.7 | Anthropic |
| gpt-5.5 | OpenAI |
| claude-sonnet-4.6 | Anthropic |
| gpt-5.4-mini | OpenAI |
| kimi-k2.5 | Bedrock |
| qwen3-235b-a22b | Bedrock |
AI Agent Safety Benchmarks
AI agent safety benchmarks measure whether tool-using and embodied AI systems act safely across realistic tasks and interactions.
See all 237 papers