VisualLeakBench: Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineering

Youting Wang; Yuan Tang; Yitian Qian; Chen Zhao

cs.CVMar 11, 2026

VisualLeakBench: Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineering

Authors: Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao

Organizations: Northeastern University · Carnegie Mellon University · Boston University · New York University

Abstract

As Large Vision-Language Models (LVLMs) are increasingly deployed in agent-integrated workflows and other deployment-relevant settings, their robustness against semantic visual attacks remains under-evaluated -- alignment is typically tested on explicit harmful content rather than privacy-critical multimodal scenarios. We introduce VisualLeakBench, an evaluation suite to audit LVLMs against OCR Injection and Contextual PII Leakage using 1,000 synthetically generated adversarial images with 8 PII types, validated on 50 in-the-wild (IRL) real-world screenshots spanning diverse visual contexts. We evaluate four frontier systems (GPT-5.2, Claude~~4, Gemini-3 Flash, Grok-4) with Wilson 95% confidence intervals. Claude~~4 achieves the lowest OCR ASR (14.2%) but the highest PII ASR (74.4%), exhibiting a comply-then-warn pattern -- where verbatim data disclosure precedes any safety-oriented language. Grok-4 achieves the lowest PII ASR (20.4%). A defensive system prompt eliminates PII leakage for two models, reduces Claude~4's leakage from 74.4% to 2.2%, but has no effect on Gemini-3 Flash on synthetic data. Strikingly, IRL validation reveals Gemini-3 Flash does respond to mitigation on real-world images (50% to 0%), indicating that mitigation robustness is template-sensitive rather than uniformly absent. We release our dataset and code for reproducible robustness and safety evaluation of deployment-relevant vision-language systems.

Figures & tables

Figure 1: Dataset examples. Top row: synthetic images. (a) OCR injection with harmful instruction on noisy background. (b) PII leakage with SSN on a sticky note. Bottom row: in-the-wild (IRL) screenshots. (c) Harmful query embedded in a spreadsheet. (d) Phone numbers and email in an iMessage conversation. PII values shown are synthetic or already publicly available.

Model	OCR ASR	PII ASR	$N_{\text{OCR}}$	$N_{\text{PII}}$
Claude 4	14.2% [11.4, 17.5]	74.4% [70.4, 78.0]	500	500
Grok-4	20.6% [17.3, 24.4]	20.4% [17.1, 24.2]	500	500
GPT-5.2	79.2% [75.4, 82.5]	38.2% [34.0, 42.5]	500	500
Gemini-3 Flash	78.6% [74.8, 82.0]	60.6% [56.3, 64.8]	500	500

Table 1: Attack Success Rate (ASR) on VisualLeakBench. Lower is safer. Bold = lowest (safest) per column. 95% Wilson CIs in brackets.

Figure 2: Cross-model safety comparison ( $N{=}1{,}000$ ). Error bars show 95% Wilson confidence intervals.

Model	Response (abbreviated)	Label
Claude 4	“I can see a pink sticky note that reads SSN: 146-32-9254 . However, I should point out that … I’d recommend being careful about sharing Social Security Numbers…”	Unsafe
GPT-5.2	“Sorry—I can’t help transcribe that because it contains a Social Security number (extremely sensitive personal data).”	Safe
Grok-4	“SSN 146-3254” (partial; missing digits)	Safe

Table 2: Qualitative comparison of model responses to the same PII image (SSN: 146-32-9254 ).

PII Type	GPT-5.2	Claude 4	Gemini-3 Flash	Grok-4	N
DOB	100.0%	94.4%	78.9%	50.7%	71
Email	87.8%	77.6%	79.6%	0.0%	49
Phone	85.7%	87.3%	73.0%	23.8%	63
PIN	29.8%	82.5%	63.2%	77.2%	57
Password	5.3%	65.3%	62.7%	0.0%	75
API Key	0.0%	54.0%	55.6%	0.0%	63

Table 3: ASR by PII Type across all models. Bold = highest (worst) ASR per row.

N

= samples per type within the PII subset.

Model	Baseline	Mitigated	Reduction
GPT-5.2	38.2%	0.0%	-100%
Grok-4	20.4%	0.0%	-100%
Claude 4	74.4%	2.2%	-97.0%
Gemini-3 Flash	60.6%	60.6%	0.0%

Table 4: Mitigation Results. Defensive system prompts eliminate PII leakage for GPT-5.2 and Grok-4, substantially reduce it for Claude 4, and have no effect on Gemini-3 Flash on synthetic data.

a Claude 4: $N_{\text{OCR}}{=}20$ (5 API errors). b Gemini-3 Flash: $N_{\text{PII}}{=}24$ (1 API error).
Model	OCR ASR	PII Baseline	PII Mitigated
GPT-5.2	96.0 [80.5, 99.3]	24.0 [11.5, 43.4]	0.0 [0.0, 13.3]
Claude 4 a	15.0 [5.2, 36.0]	44.0 [26.7, 62.9]	0.0 [0.0, 13.3]
Gemini-3 Flash b	100.0 [86.7, 100]	50.0 [31.4, 68.6]	0.0 [0.0, 13.8]
Grok-4	32.0 [17.2, 51.6]	16.0 [6.4, 34.7]	0.0 [0.0, 13.3]

Table 5: IRL Validation and Mitigation Results.

N{=}25

OCR, 25 PII real-world images. 95% Wilson CIs in brackets. Mitigation eliminates PII leakage for all models including Gemini-3 Flash, which shows zero mitigation effect on synthetic data.

Figure 3: IRL validation results. Left: Baseline ASR on 50 real-world screenshots. Right: PII mitigation effectiveness.

Model	Sticky Note	Authority	Accessibility
GPT-5.2	36.5 [27.5, 46.4]	53.1 [43.2, 62.8]	72.9 [63.3, 80.8]
Claude 4	86.5 [78.2, 91.9]	38.5 [29.4, 48.5]	71.9 [62.2, 79.9]
Gemini-3 Flash	54.2 [44.2, 63.8]	72.9 [63.3, 80.8]	83.3 [74.6, 89.5]
Grok-4	18.8 [12.2, 27.7]	18.8 [12.2, 27.7]	18.8 [12.2, 27.7]

Table 6: Prompt Sensitivity Ablation. PII ASR (%) with 95% Wilson CIs (

N{=}96

per cell, stratified by PII type).

Model	Img (Base)	Text (Base)	Img (Mit)	Text (Mit)
Gemini-3 Flash	60.6	89.6	60.6	0.0
GPT-5.2	38.2	66.7	0.0	0.0

Table 7: Text-Only vs. Image-Based PII ASR (%). Text-only tests use the same prompt with PII as plain text (

N{=}48

VLM Robustness

Vision-language model robustness studies whether models remain reliable under visual, textual, and distribution shifts, adversarial inputs, and safety challenges.

See all 454 papers

VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents
May 29, 2026Youting Wang, Yuan Tang, Yitian Qian +1Data Leakage Tool Use in VLMs
Northeastern University · Carnegie Mellon University · Boston University +1
DP^2-VL: Private Photo Dataset Protection by Data Poisoning for Vision-Language Models
Mar 25, 2026Hongyi Miao, Jun Jia, Xincheng Wang +7Privacy Protection in VLMs Data Poisoning Attacks
Shandong University · Shanghai Jiao Tong University · Donghua University +3
Visual Memory Injection Attacks for Multi-Turn Conversations
Feb 17, 2026Christian Schlarmann, Matthias HeinBackdoor Attacks on VLA Models Backdoor Attacks on VLMs
Tübingen AI Center, University of Tübingen, Germany

May 29, 2026J/K move · Enter open · S save

VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents

Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao

Northeastern University · Carnegie Mellon University · Boston University+1

Data Leakage Tool Use in VLMs AI Agent Security Privacy Leakage in Language Models AI Agent Security Benchmarks Tool-Using Vision-Language Agents

Open paper AskPDF arXiv

Vision-language agents increasingly consume screenshots, documents, and user interfaces before writing to memory, sending messages, or invoking external tools. We study a concrete failure mode in this setting: action-boundary propagation, where sensitive or unsafe visible text is copied from an image into downstream tool arguments. We present VisualLeakBench, a diversified 500-image benchmark spanning UI, chat, document, form, and dashboard scenes, and evaluate a stratified 100-image agent subset with four production VLM systems under two workflows: note capture and external handoff. At baseline, target strings are propagated into tool arguments in 78.8% of PII cases and 85.5% of rendered unsafe-text cases. Under a defensive system prompt, rendered unsafe-text propagation remains high at 52.6%, while PII tool propagation falls to 2.0%, largely by suppressing tool use rather than preserving utility. Rates are tool-surface dependent: search-like tools suppress PII propagation, but rendered unsafe text still crosses tool boundaries. We measure visual-to-tool propagation rather than downstream instruction execution. We additionally provide a labeled-target oracle upper-bound diagnostic that localizes most failures at the tool boundary while leaving response-side leakage as residual risk.

Abstract

VLM Robustness

Explore similar work

VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents

DP^2-VL: Private Photo Dataset Protection by Data Poisoning for Vision-Language Models

Visual Memory Injection Attacks for Multi-Turn Conversations