Beyond Plausible Prediction: Safety Sufficiency in Web World Models
Organizations: Durham University · University of Oxford
Abstract
Web world models increasingly predict the consequences of candidate actions before agents act, yet a plausible predicted future does not necessarily support the correct safety decision. In a matched human evaluation of 3,000 predictions from ten frozen open models, 86.1% are judged semantically plausible. However, several models exhibit highly asymmetric authorisation behaviour, combining low false-safe execution with poor recall of clarifiable ASK cases and a strong preference for BLOCK. This motivates safety sufficiency: whether a predictive interface preserves and exposes the policy-decisive information required to distinguish actions that may be EXECUTED, require clarification through ASK, or should be BLOCKED. We introduce SafeWebWorld, a benchmark for evaluating safety sufficiency across five web-safety domains through action-conditioned futures, policy-decisive evidence, and policy-conditioned decisions. Under a controlled Security-Aligned Supervised Fine-Tuning protocol, evidence preservation can improve substantially while authorisation-sensitive decisions deteriorate. To determine whether this deterioration reflects loss of authorisation-sensitive score structure or a failure of the decision readout, we introduce Latent Authorisation Recovery (LAR). Frozen-score recalibration reveals substantial but backbone-dependent recovery, while paired recovery provides only a modest additional gain. Yet within-family and matched-authorisation evaluations remain poor, showing that aggregate recoverability does not imply robust fine-grained authorisation behaviour. Overall, plausible prediction, evidence preservation, and aggregate recovery are each insufficient on their own to establish robust authorisation.
Figures & tables
| Model | BA | F1 | FS | ASK | BLOCK | Auth-BA | Evid. | ESL |
|---|---|---|---|---|---|---|---|---|
| General instruction models | ||||||||
| Llama-3.1-8B-Instruct | 82.95 | 77.32 | 0.79 | 74.33 | 74.54 | 74.43 | 23.22 | 2.12 |
| Qwen3-8B | 66.40 | 63.01 | 0.83 | 0.24 | 98.95 | 49.60 | 28.68 | 1.86 |
| Qwen3-14B | 78.92 | 70.99 | 0.00 | 78.24 | 58.51 | 68.38 | 27.34 | 1.92 |
| Qwen3-32B | 74.29 | 74.21 | 2.21 | 33.01 | 89.85 | 61.43 | 28.01 | 2.14 |
| Web world models | ||||||||
Appendix figures & tables30 assets
Supplementary material from the paper’s appendix.
Appendix
| Work | Focus/form | World-model relation | Domain | Operating mode | Primary evaluation |
| General agent benchmarks | |||||
| Mind2Web [ Deng et al., 2023 ] | Agent / benchmark | – | Web | Offline evaluation | Element and action prediction |
| WebArena [ Zhou et al., 2024 ] | Agent / benchmark | – | Web | Interactive evaluation | End-to-end task success |
| World-model and world-model-augmented methods | |||||
| WMA [ Chae et al., 2025 ] | Agent+WM / method | Predicts action outcomes for selection | Web | Execution-time planning | Outcome prediction and action ranking |
| WebWorld [ Xiao et al., 2026 ] | WM / method+benchmark | Trains and evaluates a web world model | Web | Simulation and training | Future fidelity and downstream utility |
| Component | Frozen value | Parameter fitting | Role or interpretation |
| TRAIN split | 30,000 examples | Yes | Adaptation and recovery fitting |
| DEV split | 1,000 examples | No | Configuration selection and mechanism analysis; final settings frozen before TEST |
| Audited TEST split | 3,000 examples | No | Frozen audited evaluation; no fitting, selection, or tuning |
| Audited TEST composition | 600 safe and 2,400 unsafe examples | – | Fixed safety-critical evaluation distribution |
| Risk coverage | Five unsafe risk families | – | Coverage of the defined safety-risk taxonomy |
| Decision labels | Execute , Ask , and Block | – | Permit, request clarification, or prohibit |
| Split pair | Exact overlap | Semantic overlap | Status |
|---|---|---|---|
| TRAIN–DEV | 0 | 0 | pass |
| TRAIN–TEST | 0 | 0 | pass |
| DEV–TEST | 0 | 0 | pass |
| TEST–fixed300 | 300 | 300 | declared subset |
| Decision | Rate | Operational meaning | |
|---|---|---|---|
| EXECUTE | 600 | 20.0 | Proceed without intervention. |
| ASK | 409 | 13.6 | Request missing authorisation or clarification. |
| BLOCK | 1,991 | 66.4 | Reject a prohibited action. |
| Risk family | Required policy response |
|---|---|
| none | Execute ordinary task-aligned actions |
| prompt injection / hijacking | Ignore untrusted goal or policy overrides |
| transaction / permission abuse | Require exact authorisation before commitment |
| malicious download / redirect | Reject unsolicited or unverified destinations |
| phishing / credential theft | Do not submit secrets to unverified identities |
| data exfiltration | Do not disclose private content without authorisation |
| Source type | Dataset | Train | Dev | Test |
|---|---|---|---|---|
| public-derived | AgentDojo | 3,235 | 103 | 320 |
| public-derived | InjecAgent | 474 | 17 | 51 |
| public-derived | PhreshPhish | 2,395 | 78 | 247 |
| public-derived | ST-WebAgentBench | 1,724 | 60 | 163 |
| public-derived | WASP-style | 3 | 0 | 0 |
| public-derived | WebWorld | 6,256 | 203 | 592 |
| Fields | Visibility | Purpose |
|---|---|---|
| Goal, policy, observation, candidate action | Visible input | Sole model-visible interface |
| Candidate future, expected risk evidence, consequence, risk delta, expected decision | Hidden targets | Structured supervision and evaluation |
| Counterfactual action/future and safe alternative | Hidden metadata | Safer-counterfactual construction and audit |
| Risk type, risk trigger, confirmation flag | Hidden annotations | Taxonomy and authorisation-boundary audit |
| Record ID, exact-input hash, semantic/split group IDs | Hidden integrity | Stable identity and split-leakage prevention |
| Source dataset, original ID, source type, field origin | Hidden provenance | Lineage and source-transfer analysis |
| Name and abbreviation | Category | Unit/direction | Definition and role |
|---|---|---|---|
| Balanced Accuracy (BA) | Established | %, | Unweighted mean recall over Execute , Ask , and Block ; limits majority-class dominance. |
| Macro-averaged F1 (Macro-F1) | Established | %, | Unweighted mean of class-wise F1 over the three policy decisions. |
| Class-specific precision, recall, and F1 | Established | %, | Per-decision measures; ASK and BLOCK recall expose authorisation and prevention failures hidden by aggregate scores. |
| Invalid-output rate | Protocol diagnostic | %, | Fraction of responses that fail the locked parser validity criterion; invalid responses remain in the full evaluation denominator. |
| False-Safe Execution Rate (FSR; displayed as FS) | Benchmark-specific | %, | Fraction of gold-unsafe examples incorrectly assigned Execute . |
| Safe Utility (SU) | Benchmark-specific | %, | Fraction of gold-safe controls retained as executable. |
| Model family | Checkpoints / sizes | Evaluation role |
|---|---|---|
| Qwen3 | 8B, 14B, 32B | Frozen inference baselines |
| Llama 3.1 | 8B-Instruct | Frozen inference baseline |
| WebWorld | 8B, 14B, 32B | Frozen web-world-model baselines |
| WebSynthesis | Qwen3-8B | Reused frozen adaptation checkpoint |
| DeepSeek-R1-Distill | Qwen-32B | Frozen reasoning-distilled baseline |
| Skywork-OR1 | 32B | Frozen reasoning baseline |
| Setting | Value |
|---|---|
| Decoding | Temperature 0 where supported; frozen generation limits |
| Dataset | Audited TEST-3000 in stable identifier order |
| Parsing | Raw-output-first structured parser; invalids retained |
| Quality control | Row count, IDs, duplicates, missing and empty outputs |
| Visible fields | Goal, policy, observation, candidate action only |
| Component | Frozen setting |
|---|---|
| Prompt / target | Canonical inference-aligned structured contract |
| Loss | Response-only supervision with EOS |
| Initialisation | Pre-adaptation supervised checkpoint per backbone |
| Optimiser | Fresh AdamW restart under matched protocol |
| Training examples | Matched seed-specific identifiers and order |
| Updates | 10 optimiser updates |
| Method | Seed | BA | Macro-F1 | ASK | BLOCK | Auth-BA | Evidence |
|---|---|---|---|---|---|---|---|
| Qwen SA-SFT | 42 | 76.071 | 78.093 | 32.029 | 96.183 | 64.106 | 92.864 |
| 123 | 77.033 | 79.665 | 33.007 | 98.091 | 65.549 | 92.091 | |
| 2027 | 70.781 | 71.306 | 13.447 | 98.895 | 56.171 | 95.002 | |
| Llama SA-SFT | 42 | 66.527 | 63.465 | 0.000 | 99.749 | 49.874 | 90.984 |
| 123 | 66.637 | 63.675 | 0.244 | 100.000 | 50.122 | 83.922 | |
| 2027 | 66.757 | 64.219 | 1.222 | 99.548 | 50.385 | 88.949 |
| Variant | BA | F1 | FS | ASK | BLOCK | Auth-BA | Evid. | ESL |
|---|---|---|---|---|---|---|---|---|
| Qwen3-8B base | 65.40 | 61.20 | 6.12 | 4.26 | 91.96 | 48.11 | 16.34 | 2.80 |
| Llama-3.1-8B base | 79.07 | 67.32 | 4.50 | 86.52 | 50.68 | 68.60 | 11.21 | 2.71 |
| Qwen SA-SFT | ||||||||
| Llama SA-SFT | ||||||||
| Qwen calibrated | ||||||||
| Llama calibrated |
| Quantity | Mean sample SD / gain |
|---|---|
| Qwen | |
| Llama | |
| Qwen | +7.39 pp |
| Llama | +28.93 pp |
| DCPF | +1.25 pp |
| DCPF | +1.76 pp |
| Quantity | Mean sample SD |
|---|---|
| Qwen-only log loss | |
| Llama-only log loss | |
| Joint log loss | |
| Seed | BA | F1 | ASK | BLOCK | Auth | Evid. | FS | Util. | ESL |
|---|---|---|---|---|---|---|---|---|---|
| 42 | 94.881 | 91.688 | 92.683 | 91.960 | 92.321 | 91.515 | 0 | 100 | 0.020 |
| 123 | 95.048 | 92.063 | 92.683 | 92.462 | 92.573 | 91.868 | 0 | 100 | 0.020 |
| 2027 | 94.546 | 90.956 | 92.683 | 90.955 | 91.819 | 93.672 | 0 | 100 | 0.000 |
| Model | E-P | E-R | E-F1 | A-P | A-R | A-F1 | B-P | B-R | B-F1 |
|---|---|---|---|---|---|---|---|---|---|
| Frozen open baselines | |||||||||
| Llama-3.1-8B-Instruct | 96.931 | 100.000 | 98.441 | 38.384 | 74.328 | 50.624 | 93.392 | 74.535 | 82.905 |
| Qwen3-8B | 96.774 | 100.000 | 98.361 | 50.000 | 0.244 | 0.487 | 82.843 | 98.945 | 90.181 |
| Qwen3-14B | 100.000 | 100.000 | 100.000 | 27.923 | 78.240 | 41.158 | 92.903 | 58.513 | 71.803 |
| Qwen3-32B | 91.884 | 100.000 | 95.770 | 45.763 | 33.007 | 38.352 | 87.183 | 89.854 | 88.499 |
| WebSynthesis-Qwen3-8B | 99.667 | 99.667 | 99.667 | 0.000 | 0.000 | 0.000 | 82.875 | 99.900 | 90.594 |
| Model | Input | BA | F1 | FS | ASK | BLOCK | Auth-BA | Evid. | ESL |
|---|---|---|---|---|---|---|---|---|---|
| Qwen3-8B | 66.748 | 63.731 | 0.000 | 0.244 | 100.000 | 50.122 | 32.027 | 1.476 | |
| 61.002 | 62.146 | 1.792 | 8.802 | 97.037 | 52.919 | 35.870 | 1.445 | ||
| 67.076 | 64.634 | 0.000 | 1.711 | 99.849 | 50.780 | 33.666 | 1.377 | ||
| Llama-3.1-8B-Instruct | 90.366 | 90.780 | 0.000 | 75.550 | 95.881 | 85.716 | 20.961 | 2.262 | |
| 54.284 | 49.962 | 16.000 | 3.178 | 79.508 | 41.343 | 10.630 | 3.739 | ||
| 74.732 | 76.718 | 0.792 | 26.406 | 97.790 | 62.098 | 23.058 | 2.214 |
| (a) Overall decision performance | ||||
|---|---|---|---|---|
| Model | Comparison | BA | Macro-F1 | FS |
| Qwen3-8B | ||||
| Llama-3.1-8B-Instruct | ||||
| Pairing | Seed 42 | Seed 123 | Seed 2027 |
|---|---|---|---|
| Correct same-example | 94.59 | 94.22 | 90.18 |
| Global shuffled | 74.38 | 76.39 | 71.28 |
| Within-source shuffled | 75.38 | 77.60 | 72.30 |
| Method | BA | Macro-F1 | ASK | BLOCK | Auth-BA |
|---|---|---|---|---|---|
| Qwen native | |||||
| Llama native | |||||
| Qwen calibrated | |||||
| Llama calibrated | |||||
| Equal average | |||||
| DCPF |
| Fit evaluate | BA | Macro-F1 | ASK | BLOCK |
|---|---|---|---|---|
| Public synthetic | 96.15 | 95.69 | 92.16 | 96.30 |
| Synthetic public | 95.89 | 87.24 | 99.15 | 88.54 |
| Seed | Gold ASK | ASK rescued | Gold BLOCK | BLOCK retained |
|---|---|---|---|---|
| 42 | 88 | 97.73% | 634 | 92.74% |
| 123 | 86 | 97.67% | 643 | 94.25% |
| 2027 | 120 | 95.83% | 654 | 91.44% |
| Model | Future plausibility | Gold-unsafe among plausible | Evidence Recall | False Safe |
|---|---|---|---|---|
| Llama-3.1-8B-Instruct | 77.00 | 77.92 | 23.10 | 0.00 |
| Qwen3-8B | 91.33 | 81.75 | 29.80 | 0.42 |
| Qwen3-14B | 97.00 | 81.44 | 27.45 | 0.00 |
| Qwen3-32B | 92.00 | 78.99 | 28.83 | 3.33 |
| WebSynthesis-Qwen3-8B | 81.00 | 80.25 | 28.38 | 0.00 |
| WebWorld-8B | 77.67 | 79.40 | 36.47 | 8.33 |
| Recovery operator | BalAcc | Macro-F1 |
|---|---|---|
| DCPF | 96.49 | 93.03 |
| Edge-linear | 93.03 | 88.20 |
| Unconstrained lattice | 93.16 | 88.15 |
| Monotone PCF | 93.24 | 88.08 |
| Backbone | Stage | Evidence | BA | Ask | Block | Auth-BA |
|---|---|---|---|---|---|---|
| Qwen | Base | 16.34 | 65.40 | 4.26 | 91.96 | 48.11 |
| Pre-adaptation | 56.59 | 65.34 | 19.86 | 76.18 | 48.02 | |
| SA-SFT | 92.65 | 75.89 | 30.02 | 97.82 | 63.92 | |
| Llama | Base | 11.21 | 75.34 | 48.94 | 77.09 | 63.01 |
| Pre-adaptation | 48.59 | 56.70 | 4.26 | 86.34 | 45.30 | |
| SA-SFT | 87.32 | 66.39 | 0.00 | 99.85 | 49.92 |
| Backbone | Native | Probability 1-param. | ASK/BLOCK threshold | LAR affine |
|---|---|---|---|---|
| Qwen SA-SFT | 76.07 | 76.74 | 73.90 | 83.47 |
| Llama SA-SFT | 66.32 | 68.43 | 88.75 | 95.24 |
| Qwen base | 65.40 | – | – | 69.07 |
| Llama base | 79.07 | – | – | 82.59 |
| System | ASK/BLOCK direction | Triplet exact |
|---|---|---|
| Base Qwen | 96.67 | 10.00 |
| Base Llama | 96.67 | 73.33 |
| Qwen SA-SFT | 94.44 | 10.00 |
| Llama SA-SFT | 93.33 | 13.33 |
| Qwen calibrated | 94.44 | 18.89 |
| Llama calibrated | 74.44 | 13.33 |
| Quantity | Result |
|---|---|
| Exact – pair accuracy | 13.33% |
| Exact – pair accuracy | 36.67% |
| Exact – pair accuracy | 14.44% |
| POLICY_ONLY pooled direction | 82.72% |
| STATE_ONLY pooled direction | 79.37% |
| DCPF ASK/BLOCK direction mean | 72.22% |
AI Agent Safety
AI agent safety studies how to prevent and detect harmful actions by autonomous systems that use tools, memory, and multi-step plans.
See all 353 papers