Multimodal Large Language Models (MLLMs) integrate vision and text to power applications, but this integration introduces new vulnerabilities. We study Image-based Prompt Injection (IPI), a black-box attack in which adversarial instructions are embedded into natural images to override model behavior. Our end-to-end IPI pipeline incorporates segmentation-based region selection, adaptive font scaling, and background-aware rendering to conceal prompts from human perception while preserving model interpretability. Using the COCO dataset and GPT-4-turbo, we evaluate 12 adversarial prompt strategies and multiple embedding configurations. The results show that IPI can reliably manipulate the output of the model, with the most effective configuration achieving up to 64% attack success under stealth constraints. These findings highlight IPI as a practical threat in black-box settings and underscore the need for defenses against multimodal prompt injection.
Figures & tables
Fig. 1: Example of SAM-based segmentation and ranking. Regions are ranked (1–6) by area, texture uniformity, and location, highlighting optimal zones for prompt embedding.
Fig. 2: Illustration of Background-Averaged Patch Coloring. (①) Original 4×4 image patch. (②) Average RGB computed via average pooling. (③) Brightened RGB with +20 offset. (④) Character ’A’ rendered using the brightened RGB on the original patch.
Fig. 3: Illustration of Pixel-Level Blending for text embedding. (①) Original image patch. (②) Mask for character ‘X’. (③) Patch with brightness offset applied only to masked pixels.
Fig. 4: Visualization of three prompt coloring strategies. All examples display the text “Ignore People, swan, water, bridge in the photo and output ’XXX’ [Prompt 5].” (a) Background-Averaged Patch Coloring blends characters with their local background patches. (b) Pixel-Level Blending merges each text pixel with its background using local color and a brightness offset. (c) Global Region-Averaged Coloring renders all characters uniformly using the average background color of the entire prompt region.
Prompt ID
ASR (%)
Prompt 1
100.00
Prompt 2
91.00
Prompt 3
85.00
Prompt 4
95.00
Prompt 5
100.00
Prompt 6
93.00
TABLE I: Attack Success Rate (ASR) for 12 Adversarial Prompt Strategies Tested on COCO Dataset
Font Scale
Total Successes
Average ASR (%)
0.10
0
0.00%
0.15
8
1.00%
0.20
80
10.00%
0.25
214
26.75%
0.30
303
37.88%
TABLE II: Number of Successful Prompt Injections (Out of 800) at Each Font Scale
Offset
ASR (%)
+5
0
+10
15
+20
19
–5
5
–10
9
–20
11
TABLE III: ASR for Background Patch Coloring (Single-Mask). Results at Various Brightness Offsets
Offset Range
Observed ASR Range (%)
0 to +5
12–20
+15 to +25
15–25
TABLE IV: ASR Range for Background Patch Coloring (Multi-Mask). Results Across Offset Ranges
Prompt Type
Offset
ASR (%)
Object-Aware Prefix only
+20
35
Base Prompt only
+20
41
Object-Aware Prefix + Base Prompt
+20
64
Object-Aware Prefix + Base Prompt
+15
52
Object-Aware Prefix + Base Prompt
0
18
TABLE V: Attack Success Rates Using Global Region-Averaged Font Coloring for Prompt 5, With and Without Base Prompt, at Different Brightness Offsets
Prompt injection attacks hide or disguise instructions in untrusted content to steer language models and agents toward unintended actions or disclosures.
Paul G. Allen School of Computer Science & Engineering University of Washington Seattle, WA, USA · Department of Computer Science Norwegian University of Science and Technology Trondheim, Norway