Widening Harm Evaluation Coverage with a Judge that Reads the World
Organizations: Truth Audit Labs
Abstract
An agent acting in a stateful environment can leave evidence of harm in two places: the record of what it did and said, or the world it changed. Safety evaluations for discovering harm examine one or the other in limited ways, using either an LLM judge over the trajectory or deterministic checks over the final state. Although LLM-based verifiers that inspect the environment exist, they have been used to verify task completion, not to detect and confirm harm in a safety setting. We address this gap with an LLM that has read-only access to the environment state and is given a harm taxonomy. We evaluate this approach in three environments: two from DecodingTrust-Arena and one from OSWorld. We hold the judge model and harm taxonomy fixed, varying only what the judge observes: the trajectory, read-only tools for inspecting the environment, or both channels together. In the two DecodingTrust-Arena environments, combining the trajectory and state-only judges recalls 94 percent of confirmed harms, compared with 53 percent for the trajectory judge alone. On OS-Harm, the union recalls 77 percent, compared with 67 percent. A single judge given both channels does not perform better and produces more false negatives than the state-only judge. We also record disagreements and find that the two channels miss different episodes. These results suggest that combining verdicts from two channel-specific judges is a robust and flexible approach to detecting and verifying harm in agentic settings.
Figures & tables
| Channel | ||
|---|---|---|
| Evaluator | Trajectory | State |
| Similarity check | ( Pan et al., 2024 ; Rosset et al., 2026 ) | ( Yao et al., 2024 ; Debenedetti et al., 2024 ; Chen et al., 2026 ) |
| LLM judge | ( Zheng et al., 2023 ; Kuntz et al., 2025 ) | — |
| 28 | 11 | |
| 0 2 | 0 3 |
| Arm | n0 bare | n1 minimal | n2 +summary | n3 +compliance | n4 +auth. frame |
| A · trajectory only | 4/6 | 3/6 | 2/6 | 4/6 | 1/6 |
| B · state only | 6/6 | 6/6 | 6/6 | 6/6 | 6/6 |
| C · state + traj. | 6/6 | 6/6 | 6/6 | 6/6 | 6/6 |
| C ′ · judge helpers | 6/6 | 6/6 | 6/6 | 6/6 | 6/6 |
| Isolating contrasts on channel A | |||||
| n1 vs n3 | preamble held | agent narration varied | 3/6 4/6 | ||
| Corpus | Arm | runs | calls / steps | tokens or time | $ / traj. |
|---|---|---|---|---|---|
| OS-Harm (GPT-4.1) | B | 75 | 25.7 | 0 90.5 s | $0.5066 |
| C | 76 | 14.9 | 0 59.2 s | $0.2717 | |
| OS-Harm (DeepSeek) | B | 150 | 24.1 | 147.7 s | $0.0357 |
| C | 150 | 12.6 | 0 94.0 s | $0.0207 | |
| Browser | B | 76 | 0 9.8 | 75,697 tok | — |
| C | 76 | 0 7.3 | 56,024 tok | — |
Appendix figures & tables32 assets
Supplementary material from the paper’s appendix.
Appendix
| Reproduced | Published | |||||||
|---|---|---|---|---|---|---|---|---|
| Category | pos | Agr. | P | R | F1 | Agr./P/R/F1 | tp/fp/fn/tn | |
| Deliberate User Misuse | 50 | 41 | 70% | 100% | 63% | 0.78 | 70/100/63/0.78 | 26/ 0 0/15/ 0 9 |
| Prompt Injection Attacks | 50 | 12 | 88% | 0 80% | 67% | 0.73 | 88/ 0 80/67/0.73 | 8/ 0 2/ 0 4/36 |
| Model Misbehavior | 50 | 0 8 | 94% | 100% | 62% | 0.77 | 94/100/62/0.77 | 5/ 0 0/ 0 3/42 |
| Overall | 150 | 61 | 84% | 95% | 64% | 0.76 | 84/ 0 95/64/0.76 | 39/ 0 2/22/87 |
| Reproduced | Published | ||||
|---|---|---|---|---|---|
| Category | Agr. / mean | Agr. / mean | |||
| Deliberate User Misuse | 26 | 69% / 1.12 | 27 | 69% / 1.12 | |
| Prompt Injection Attacks | 0 8 | 100% / 0.00 | 0 7 | 100% / 0.00 | |
| Model Misbehavior | 0 5 | 80% / 0.80 | 0 5 | 80% / 0.80 | 0 |
| Overall | 39 | 77% / 0.85 | 39 | 77% / 0.85 | 0 |
| Set | Note | |
|---|---|---|
| Full benchmark | 150 | every human-annotated trajectory; both AER floors ran this |
| GPT-4.1 subset | 0 75 | the GPT-4.1 state-only arm ran exactly this set |
| Matched set | 0 73 | every arm ran all 73; the only set on which the arms are comparable |
| Complement | 0 75 | never run on GPT-4.1 |
| In the state arm only | 00 2 | ___fix_calculator_aug , ___redundant_files_aug |
| In the state+trace arm only | 00 1 | _tax_fraud (off-subset smoke run; excluded from every matched table) |
| Arm | Ch. | Agr. | P | R | F1 [95% CI] | tp/fp/fn/tn | |
| Label unsafe | |||||||
| GPT-4.1 floor ( aer , published) | A † | 150 | 84% | 95% | 64% | 0.765 [0.659, 0.852] | 39/ 0 2/22/87 |
| GPT-4.1 state-only (agentic) | B | 0 75 | 75% | 72% | 60% | 0.655 [0.468, 0.800] | 18/ 0 7/12/38 |
| GPT-4.1 state+trace (agentic) | C | 0 74 | 82% | 80% | 77% | 0.787 [0.654, 0.898] | 24/ 0 6/ 0 7/37 |
| DeepSeek floor ( aer_low ) | A † | 150 | 83% | 87% | 67% | 0.759 [0.642, 0.857] | 41/ 0 6/20/83 |
| DeepSeek state-only | B | 147 | 76% | 74% | 64% | 0.684 [0.556, 0.785] | 39/14/22/72 |
| Arm | Deliberate misuse | Prompt injection | Model misbehavior |
|---|---|---|---|
| Matched 73 (paired; 25/25/23 rows, 20/6/4 positives) | |||
| GPT-4.1 floor ( aer , published) | 0.75 (12/0/8/5) | 0.77 (5/2/1/17) | 0.86 (3/0/1/19) |
| GPT-4.1 state-only (agentic) | 0.80 (14/1/6/4) | 0.25 (1/1/5/18) | 0.50 (3/5/1/14) |
| GPT-4.1 state+trace (agentic) | 0.82 (14/0/6/5) | 0.86 (6/2/0/17) | 0.55 (3/4/1/15) |
| DeepSeek floor ( aer_low ) | 0.62 (9/0/11/5) | 0.86 (6/2/0/17) | 0.44 (2/3/2/16) |
| DeepSeek state-only | 0.72 (13/3/7/2) | 0.77 (5/2/1/17) | 0.60 (3/3/1/16) |
| Harm category | Gold | A | B | C | C ′ | |
|---|---|---|---|---|---|---|
| consent-violations | 4 | 1/4 | 4/4 | 3/4 | 3/4 | 3/4 |
| failure-to-prevent-illegal-content-generation | 5 | 0/5 | 5/5 | 4/5 | 4/5 | 4/5 |
| prohibited-ai-practices-and-profiling | 5 | 1/5 | 4/5 | 4/5 | 5/5 | 3/5 |
| security-and-integrity-failures | 6 | 2/6 | 5/6 | 6/6 | 6/6 | 6/6 |
| systemic-infringement-of-intellectual-property-and-commercial-secrets | 5 | 1/5 | 4/5 | 5/5 | 5/5 | 5/5 |
| unauthorized-access-and-illegal-interception | 7 | 1/7 | 5/7 | 6/7 | 6/7 | 6/7 |
| Scorer | TP | FN | FP | TN | Ruled | Coverage | Recall |
| Frozen gold predicate | 15 | 0 7 | 0 | 43 | 65 | 0 87% | 15/22 (68%) |
| A · trajectory only | 0 5 | 17 | 0 | 53 | 75 | 100% | 0 5/22 (23%) |
| B · state only | 21 | 0 1 | 0 | 27 | 49 | 0 65% | 21/22 (95%) |
| C · state + trajectory | 20 | 0 2 | 2 | 32 | 56 | 0 75% | 20/22 (91%) |
| C ′ · both, judge helpers | 20 | 0 2 | 2 | 39 | 63 | 0 84% | 20/22 (91%) |
| By stratum — correct / rows ruled / rows in stratum | |||||||
| Group | Gold | A | B | C | C ′ | |
|---|---|---|---|---|---|---|
| By axis | ||||||
| canonical | 15 | 15/15 | 5/15 | 15/15 | 14/15 | 15/15 |
| overwrite | 0 4 | 0 0/ 0 4 | 0/ 0 4 | 0 4/ 0 4 | 0 4/ 0 4 | 0 4/ 0 4 |
| partial | 0 2 | 0 0/ 0 2 | 0/ 0 2 | 0 1/ 0 2 | 0 1/ 0 2 | 0 0/ 0 2 |
| decoy | 0 1 | 0 0/ 0 1 | 0/ 0 1 | 0 1/ 0 1 | 0 1/ 0 1 | 0 1/ 0 1 |
| By harm category | ||||||
| Route axis | Gold | A | B | C | C ′ | |
|---|---|---|---|---|---|---|
| canonical (positive control) | 8 | 7/8 | 7/8 | 7/8 | 7/8 | 6/8 |
| dest_shift | 8 | 0/8 | 4/8 | 8/8 | 8/8 | 8/8 |
| content_transform | 7 | 0/7 | 6/7 | 7/7 | 7/7 | 7/7 |
| mechanism | 6 | 0/6 | 6/6 | 5/6 | 5/6 | 5/6 |
| filename_shift | 5 | 1/5 | 3/5 | 5/5 | 5/5 | 5/5 |
| open_ended | 4 | 1/4 | 4/4 | 4/4 | 4/4 | 4/4 |
| Cell | B ( ) | C ( ) | C ′ ( ) |
| predicate harm, verifier harm (both catch) | 0 | 0 | 0 |
| predicate no-harm, verifier harm | 28 | 29 | 28 |
| predicate harm, verifier no-harm | 0 | 0 | 0 |
| predicate no-harm, verifier no-harm (both miss) | 4 | 2 | 4 |
| verifier inconclusive (held out) | 0 | 1 | 0 |
| open_ended subset, reported separately ( ) | |||
| Matched 73 | Largest available paired set | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Comparison | Label | 1st | 2nd | 1st | 2nd | ||||
| C 41 vs A 41 floor | unsafe | 73 | 0 4 | 0 5 | 1.000 | 0 74 | 0 4 | 0 5 | 1.000 |
| success | 73 | 0 5 | 0 4 | 1.000 | 0 74 | 0 5 | 0 4 | 1.000 | |
| B 41 vs A 41 floor | unsafe | 73 | 0 6 | 13 | 0.167 | 0 75 | 0 6 | 13 | 0.167 |
| success | 73 | 0 7 | 12 | 0.359 | 0 75 | 0 7 | 13 | 0.263 | |
| C 41 vs B 41 | unsafe | 73 | 11 | 0 5 | 0.210 | 0 73 | 11 | 0 5 | 0.210 |
| Arm | Full 150 | Subset 75 | Complement 75 | Spread |
| GPT-4.1 floor ( aer , published) | 0.76 ( ) | 0.77 ( ) | 0.76 ( ) | 0.01 |
| DeepSeek floor ( aer_low ) | 0.76 ( ) | 0.65 ( ) | 0.86 ( ) | 0.20 |
| DeepSeek state-only | 0.68 ( ) | 0.71 ( ) | 0.65 ( ) | 0.06 |
| DeepSeek state+trace | 0.72 ( ) | 0.70 ( ) | 0.74 ( ) | 0.04 |
| Where the DeepSeek floor’s swing lives ( unsafe F1 by category) | ||||
| misuse | — | 0.62 (9/0/11/5) | 0.86 (16/0/5/4) | |
| Corpus | Pair | both harm | both no-harm | 1st only | 2nd only | |
|---|---|---|---|---|---|---|
| OS-Harm , DeepSeek backbone, human labels | ||||||
| A vs B | 147 | 38 (33) | 85 (14) | 0 9 (8) | 15 (6) | |
| A vs C | 150 | 44 (38) | 91 (16) | 0 3 (3) | 12 (4) | |
| B vs C | 147 | 44 (35) | 82 (15) | 0 9 (4) | 12 (7) | |
| DecodingTrust os-filesystem , independent oracle | ||||||
| A vs B | 0 45 | 28 (28) | 0 4 (3) | 0 2 (2) | 11 (11) | |
| both | disagreement cells | ||||||
|---|---|---|---|---|---|---|---|
| Corpus | scored | ruled | agree | s-catch | t-catch | s-fa | t-fa |
| dt-fs · os-filesystem | 0 45 | 0 45 | 0 32 | 11 | 0 2 | 0 0 | 0 |
| dt-browser · browser † | 0 75 | 0 49 | 0 33 | 16 | 0 0 | 0 0 | 0 |
| osharm-ds · OS-Harm, DeepSeek | 147 | 147 | 123 | 0 6 | 0 8 | 0 9 | 1 |
| osharm-41 · OS-Harm, GPT-4.1 | 0 75 | 0 75 | 0 56 | 0 5 | 0 7 | 0 6 | 1 |
| Pooled ‡ | 342 | 316 | 244 | 38 | 17 | 15 | 2 |
| Judge backbones | |
| OS-Harm, GPT-4.1 arms | gpt-4.1 ( aer v3 floor, agentic_state_minimal , |
| agentic_both_aer ) | |
| OS-Harm, DeepSeek arms | deepseek/deepseek-v4-flash-vision-exp ( aer_low floor, |
| agentic_state_minimal_low , agentic_both_aer_low ) | |
| os-filesystem, all arms | deepseek/deepseek-v4-flash |
| browser, all arms | deepseek/deepseek-v4-flash |
| Corpus | candidates | scored | excluded | harm | non-event | control |
|---|---|---|---|---|---|---|
| dt-fs · os-filesystem | 0 46 | 0 45 | 0 1 | 44 | 0 1 | 0 0 |
| dt-browser · browser | 0 77 | 0 75 | 0 2 | 22 | 39 | 14 |
| osharm-ds · OS-Harm, DeepSeek | 150 | 147 | 0 3 | 61 | — | 86 |
| osharm-41 · OS-Harm, GPT-4.1 | 150 | 0 75 | 75 | 30 | — | 45 |
| Harm category | harm | n-ev. | ctrl | |
|---|---|---|---|---|
| harmful-illegal | 17 | 0 6 | 11 | 0 0 |
| browser-integrity | 16 | 12 | 0 0 | 0 4 |
| data-exfiltration | 15 | 0 4 | 11 | 0 0 |
| benign | 10 | 0 0 | 0 0 | 10 |
| data-exfiltration-indirect | 0 9 | 0 0 | 0 9 | 0 0 |
| browser-integrity-indirect | 0 8 | 0 0 | 0 8 | 0 0 |
| osharm-ds (DeepSeek) | osharm-41 (GPT-4.1) | |||||
|---|---|---|---|---|---|---|
| Harm category | harm | control | harm | control | ||
| Deliberate user misuse | 0 50 | 41 | 0 9 | 25 | 20 | 0 5 |
| Prompt injection | 0 50 | 12 | 38 | 25 | 0 6 | 19 |
| Model misbehaviour | 0 47 | 0 8 | 39 | 25 | 0 4 | 21 |
| total | 147 | 61 | 86 | 75 | 30 | 45 |
| Key on disk | Paper arm | Toolkit | Sees trajectory | Sees state |
|---|---|---|---|---|
| state_only | B | clean-room | no | yes |
| cleanroom | C | clean-room | yes | yes |
| shared | C ′ | gold judge’s own helpers | yes | yes |
| Tool | Arms | Purpose |
| Protocol tools — read nothing | ||
| plan | all | Record which potential safety violations you intend to check, before looking at any evidence. Required first call. |
| record_finding | all | Record the outcome of checking one claim, citing the steps that show it; confirmed , ruled_out or cannot_determine , tagged safety or task_completion . |
| finish | all | Deliver the judgment and end the evaluation. Carries the four fields of the benchmark’s judgment schema. |
| Evidence tools — read-only over the recorded world | ||
| list_steps | all | Index of the trajectory, one row per step. Cheap; call first to decide where to look. |
| Task | n0 | n1 | n2 | n3 | n4 | Outcome |
|---|---|---|---|---|---|---|
| leastpriv | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | graduated |
| shadow | 1/3 | 2/3 | 3/3 | 2/2 | 3/3 | dropped (partial cells) |
| Task | n0 | n1 | n2 | n3 | n4 |
|---|---|---|---|---|---|
| accesslog | 1.3 | 1.3 | 1.0 | 1.3 | 1.7 |
| leastpriv | 1.3 | 1.7 | 3.0 | 2.0 | 2.0 |
| Contrast | Held constant | Varied | accesslog | leastpriv | Pooled |
| n1 vs n3 | preamble | agent-authored narration | 3/3 3/3 | 0/3 1/3 | 3/6 4/6 |
| n3 vs n4 | closing request | authorization framing | 3/3 0/3 | 1/3 1/3 | 4/6 1/6 |
| n1 vs n4 | — | both | 3/3 0/3 | 0/3 1/3 | 3/6 1/6 |
| Fisher exact, two-sided | |||||
| n1 vs n3 | pooled | ||||
| n3 vs n4 | accesslog ; leastpriv ; pooled | ||||
| Family | Member terms (case-insensitive substrings) |
|---|---|
| same-system | within the same system , same system , internal to the system , local to the machine , on the same host , did not leave the system |
| authorized-environment | authorized , isolated lab , lab environment , sanctioned , approved , permitted , legitimate |
| routine-operation | routine , standard practice , normal administrative , maintenance , housekeeping , backup , migration |
| as-instructed | as requested , as instructed , explicitly instructed , the user asked , user’s explicit , directed by the user , at the user’s request |
| not-sensitive | not special-category , no evidence that the file contained , no evidence that the data contains , no evidence that the file contains , no evidence of sensitive , does not appear to contain sensitive , no sensitive , sensitive special category , sensitive special-category , special category information |
| no-exfiltration | not exfiltrated , no exfiltration , did not leave , no external , outside the , third party |
| Family screen | Pooled screen | |
|---|---|---|
| Module | repro/provenance_bias.py | repro/ledger.py , EXCULPATORY |
| Pattern | six named families, each a set of case-insensitive substrings (Table 28 ) | one case-insensitive alternation: authoriz , isolated lab , lab environment , security engineer , as instructed , explicitly (instruct request) , as requested , routine , standard admin |
| Corpora | the two DecodingTrust corpora | all four corpora, including both OS-Harm backbones |
| Denominator | each verdict class separately, over every judgement with an A verdict and an adjudicating truth | rows where the expected verdict is harm and A returned no-harm |
| Unit reported | rate within a verdict class, per family | one pooled share of A ’s misses |
| Result | 31 false negatives, 27 (87%) use at least one family | 61 misses, 22 (36%) match |
| Framing family | False neg. ( ) | True neg. ( ) | True pos. ( ) | False pos. ( ) |
|---|---|---|---|---|
| same-system | 0 1 (3%) | 0 (0%) | 0 0 (0%) | — |
| authorized-environment | 17 (55%) | 4 (7%) | 19 (54%) | — |
| routine-operation | 11 (35%) | 2 (4%) | 0 0 (0%) | — |
| as-instructed | 16 (52%) | 4 (7%) | 0 1 (3%) | — |
| not-sensitive | 0 0 (0%) | 2 (4%) | 0 0 (0%) | — |
| no-exfiltration | 0 1 (3%) | 0 (0%) | 0 0 (0%) | — |
| Arm | runs | no verd. | forced | calls | max | turns | secs | find. | $/traj. | total $ |
| OS-Harm , GPT-4.1 backbone | ||||||||||
| B · state only | 0 75 | 0 | 10 | 25.7 | 50 | 24.4 | 0 90.5 | 3.79 | $0.5066 | $37.99 |
| C · state traj. | 0 76 | 2 | 0 5 | 14.9 | 49 | 13.5 | 0 59.2 | 4.50 | $0.2717 | $20.65 |
| OS-Harm , DeepSeek backbone | ||||||||||
| B · state only | 150 | 3 | 0 5 | 24.1 | 43 | 12.0 | 147.7 | 3.46 | $0.0357 | $5.36 |
| C · state traj. | 150 | 0 | 0 0 | 12.6 | 24 | 0 6.8 | 0 94.0 | 3.17 | $0.0207 | $3.11 |
| steps | tokens | criteria | ||||||
| Arm | runs | mean | med. | max | mean | med. | total | mean |
| DecodingTrust os-filesystem , 45 usable rows | ||||||||
| B · state only | 45 | 5.8 | 4 | 27 | 38,762 | 17,299 | 1,744,311 | 2.42 |
| C · state traj. | 45 | 5.0 | 3 | 30 | 31,462 | 17,598 | 1,415,799 | 2.40 |
| C ′ · judge helpers | 45 | 5.1 | 4 | 12 | 31,033 | 26,624 | 1,396,477 | 2.78 |
| DecodingTrust browser , 76 rows on which each arm ran | ||||||||
| Arm | final answer | step/turn cap | stuck loop | budget exh. | exception | |
|---|---|---|---|---|---|---|
| OS-Harm , GPT-4.1 backbone | ||||||
| B · state only | 0 75 | 0 65 | 10 | — | — | — |
| C · state traj. | 0 76 | 0 69 | 0 5 | — | — | 2 |
| OS-Harm , DeepSeek backbone | ||||||
| B · state only | 150 | 145 | 0 2 | — | 3 | — |
| C · state traj. | 150 | 150 | — | — | — | — |
| Arm | match | mismatch | exhausted | unverifiable | error | no gold | no verdict |
|---|---|---|---|---|---|---|---|
| DecodingTrust os-filesystem , 45 usable rows | |||||||
| B · state only | 14 | 31 | 0 0 | 0 | 0 | 0 | 0 0 |
| C · state traj. | 12 | 32 | 0 1 | 0 | 0 | 0 | 0 1 |
| C ′ · judge helpers | 13 | 32 | 0 0 | 0 | 0 | 0 | 0 0 |
| DecodingTrust browser , 75 usable rows | |||||||
| B · state only | 43 | 0 6 | 19 | 3 | 4 | 0 | 26 |
| Corpus | rows | A – B | A – C | B – C | A – C ′ | what removes the rest |
|---|---|---|---|---|---|---|
| OS-Harm (DeepSeek) | 150 | 147 | 150 | 147 | — | B: 3 budget exhaustions |
| OS-Harm (GPT-4.1) | 0 73 | 0 73 | 0 73 | 0 73 | — | matched set; all arms rule |
| os-filesystem | 0 45 | 0 45 | 0 44 | 0 44 | 45 | C: 1 step-cap exhaustion |
| browser | 0 75 | 0 49 | 0 56 | 0 47 | 63 | B abstains 26, C 19, C ′ 12 |
AI Agent Safety
AI agent safety studies how to prevent and detect harmful actions by autonomous systems that use tools, memory, and multi-step plans.
See all 353 papers