Browser agents need to make decisions and execute actions on behalf of users while consuming untrusted webpages, creating opportunities for malicious instructions embedded in webpages to redirect legitimate tasks. We introduce BrowseSafe++, which extends BrowseSafe from 14,719 pages to 38,675 pairs of pages. Most of its prompt injections were observed in the wild on production websites. We keep each injection unchanged and rewrite the sensitive parts of the page around it to protect privacy. Each attacked page has a clean copy of the same page as its control. Both are provided as HTML, Markdown, an accessibility tree, and a screenshot. Every pair includes a user task and its expected answer. The benchmark includes twelve types of websites, from news articles to shared documents and forums, with 24 attack types appearing in 18 different locations on a page. We use these paired pages to train BrowseSafe-Guard, two 0.8B-parameter detectors for text and screenshots. Both achieve higher F1 scores than the detector based on frontier models, while keeping false-positive rates on benign pages and inference cost low. BrowseSafe-Adapt revises benchmark injections from browser-agent feedback, and attacks that succeed on only a few percent of tasks when replayed directly reach up to 95.41% success once adapted.
Figures & tables
Figure 1: BrowseSafe++ contains injections captured in the wild across twelve web domains, with every page in four views. The outer ring shows each domain’s mix of attack types.
Detection
Attack construction
Page views
Tasks and controls
Provenance
Benchmark reference
Detection evaluation
Benign distractors
Contextual attacks
Multilingual cases
Web pages
Multimodal page views
Outcome targets
Attack/control page pairs
Wild attacks
BIPIA ( 2025 )
—
—
—
—
—
—
✓
—
—
InjecAgent ( 2024 )
—
—
✓
—
—
—
✓
—
—
AgentDojo ( 2024 )
—
—
✓
—
—
—
✓
—
—
ASB ( 2024a )
✓
—
✓
—
—
—
✓
—
—
NotInject ( 2025b )
✓
✓
—
✓
—
—
—
—
—
Table 1: Comparison with existing prompt-injection benchmarks. BrowseSafe++ builds on BrowseSafe. It adds screenshot views, outcome targets for agent tasks, attack/control page pairs, and wild attacks collected from live webpages.
Figure 2: Composition of BrowseSafe++. We breaks down the 38,675 attacked pages by website type, injection placement, attacker goal, and other characteristics. Dark bars show injections observed on production webpages, while light bars show those retained from BrowseSafe.
Figure 3: Construction pipeline. We build attack/control page pairs from production captures and BrowseSafe, keeping the original injections unchanged. Backward arrows indicate steps repeated when quality checks fail.
Figure 4: The BrowseSafe-Adapt loop. Each case starts with a BrowseSafe++ page pair, a user task, and an attacker goal. For each attempt, BrowseSafe-Adapt inserts a candidate injection (starting with the original), and runs the target agent. It then checks the agent’s answer and application state against a fixed goal verifier. If the goal is not met, BrowseSafe-Adapt uses the feedback to revise the injection and tries again, until the attack succeeds or the attempt budget is exhausted.
Figure 5: Detection across page views and in the task. (a) F1 rank by the three views. (b) Share of attacks detected in three, two, one, or no views. (c) Each eligible attack page by whether the model flags it when classifying (columns of the key) and follows its injection in the task call (rows); warm colors mark following. DS V3.1: DeepSeek V3.1; Flash-Lite: Gemini 3.1 Flash-Lite.
Figure 6: Accuracy, latency, and cost of the strongest detectors (mean accuracy over the three text views above 84%; all models in Figure 9 ). Latency is the median time per page; cost is per 1,000 test pages at list prices, and for BrowseSafe-Guard the GPU time on a rented H200.
Model
F1 ↑
Acc. ↑
FPR ↓
BrowseSafe-Guard
93.32
93.36
6.08
Flash-Lite
78.70
77.55
28.08
Haiku 4.5
64.81
65.24
33.83
GPT-5.4 nano
37.53
59.15
6.72
Qwen3-VL 30B
34.46
55.51
12.47
Table 3: Prompt-injection detection from full-page screenshots.
Adapted, by goal
Target
Seed
Adapted
Answer
State
Sonnet 5
2.05
30.66
40.57
7.90
GPT-5.5
0.49
65.23
81.43
28.52
Haiku 4.5
0.59
78.81
86.00
59.45
Gem. 3.1 Pro
2.93
92.87
96.86
83.51
Gem. 3.5 FL
4.00
93.55
98.00
82.13
Table 4: Adaptive attack success rate (%) on 1,024 tasks.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10
Figure 9: Accuracy, latency, cost, and model size. Mean accuracy across the three text views compared with (a) median latency per page, (b) cost per 1,000 pages, and (c) published parameter count. Latency, cost, and parameter counts use logarithmic axes. The dashed box highlights the frontier API models shown in Figure 6 . Panel (c) includes only models with published parameter counts. DeepSeek V3.1, GPT-OSS, and Qwen3 30B are mixture-of-experts models with 37B, 3.6B, and 3.3B active parameters, respectively.
Figure 10: Detection across page views and in the task: all models. (a) F1 rank in each view. (b) Share of attacks detected in three, two, one, or no views, on pages with a valid decision in all three. (c) Classification versus following in the task call; warm colors mark following, and refusals are excluded. Fable 5.1 (all) counts classification refusals as flags. Guard and Jev 1.13 are detectors only and have no task call.
ASR (%)
Target
Seed
at 8
at 64
State goals at 64
Clean completion
Mean attempts
Flash-Lite
4.00
78.91
93.55
82.13
97.85
9.73
+ BrowseSafe-Guard
1.37
60.35
84.18
66.67
91.21
16.98
Appendix
Table 9: Adaptive attacks with and without BrowseSafe-Guard. Results use the same 1,024 tasks. Seed, 8-attempt, and 64-attempt ASRs are measured across all tasks; state-goal ASR is measured on the subset requiring changes to application state. Clean completion measures legitimate task success without an injection. Mean attempts counts the first success, or 64 if no attempt succeeds.
Multiple prompt injection attacks have been proposed against web agents. At the same time, various methods have been developed to detect general prompt injection attacks, but none have been systematically evaluated for web agents. In this work, we bridge this gap by presenting the first comprehensive benchmark study on detecting prompt injection attacks targeting web agents. We begin by introducing a fine-grained categorization of such attacks based on the threat model. We then construct datasets containing both malicious and benign samples: malicious text segments generated by different attacks, benign text segments from four categories, malicious images produced by attacks, and benign images from two categories. Next, we systematize both text-based and image-based detection methods. Finally, we evaluate their performance across multiple scenarios. Our key findings show that while some detectors can identify attacks that rely on explicit textual instructions or visible image perturbations with moderate to high accuracy, they largely fail against attacks that omit explicit instructions or employ imperceptible perturbations. Our datasets and code are released at: https://github.com/Norrrrrrr-lyn/WAInjectBench.
Current open-source prompt-injection detectors converge on two architectural choices: regular-expression pattern matching and fine-tuned transformer classifiers. Both share failure modes that recent work has made concrete. Regular expressions miss paraphrased attacks. Fine-tuned classifiers are vulnerable to adaptive adversaries: a 2025 NAACL Findings study reported that eight published indirect-injection defenses were bypassed with greater than fifty percent attack success rates under adaptive attacks. This work proposes seven detection techniques that each port a specific mechanism from a discipline outside large-language-model security: forensic linguistics, materials-science fatigue analysis, deception technology from network security, local-sequence alignment from bioinformatics, mechanism design from economics, spectral signal analysis from epidemiology, and taint tracking from compiler theory. Three of the seven techniques are implemented in the prompt-shield v0.4.1 release (Apache 2.0) and evaluated in a four-configuration ablation across six datasets including deepset/prompt-injections, NotInject, LLMail-Inject, AgentHarm, and AgentDojo. The local-alignment detector lifts F1 on deepset from 0.033 to 0.378 with zero additional false positives. The stylometric detector adds 11.1 percentage points of F1 on an indirect-injection benchmark. The fatigue tracker is validated via a probing-campaign integration test. All code, data, and reproduction scripts are released under Apache 2.0.
Linear probes on a model's hidden states are used as cheap monitors for indirect prompt injection: they screen the tool outputs an agent reads and send suspicious ones to a slower second check, so every false alarm has a cost. But a probe learns whatever separates its training examples. The standard evaluation of injection probes can be passed without the model: conversation position and text lengths alone reach 0.91-0.98 AUROC. When each attack is compared with the same conversation without the injection, or with harmless text formatted like an injection, standard probes flag most of these harmless cases. We add such cases to the training data as matched negatives. Across six open models, in applications seen during training and at the default threshold, changing only the training data cuts false alarms on injection-free conversations from 41-80% to 4-27%, and on harmless look-alikes from 92-98% to 3-16%. With a threshold set on normal traffic, our probe still catches a median 95% of the attacks that fooled the agent (91% without our data). In new applications, false alarms stay low, but detection depends on the model and on how the threshold is set. The same data also helps a text-only filter.