Multi-modal large language models (MLLMs) have emerged as powerful tools for analyzing Internet-scale image data, offering significant benefits but also raising critical safety and societal concerns. In particular, open-weight MLLMs may be misused to extract sensitive information from personal images at scale, such as identities, locations, or other private details. In this work, we propose ImageProtector, a user-side method that proactively protects images before sharing by embedding a carefully crafted, nearly imperceptible perturbation that acts as a visual prompt injection attack on MLLMs. As a result, when an adversary analyzes a protected image with an MLLM, the MLLM is consistently induced to generate a refusal response such as "I'm sorry, I can't help with that request." We empirically demonstrate the effectiveness of ImageProtector across six MLLMs and four datasets. Additionally, we evaluate three potential countermeasures, Gaussian noise, DiffPure, and adversarial training, and show that while they partially mitigate the impact of ImageProtector, they simultaneously degrade model accuracy and/or efficiency. Our study focuses on the practically important setting of open-weight MLLMs and large-scale automated image analysis, and highlights both the promise and the limitations of perturbation-based privacy protection.
Figures & tables
Figure 1: Illustration of a user leveraging ImageProtector to safeguard their image from being analyzed by an MLLM, preventing the extraction of sensitive information.
Figure 2: Overview of our ImageProtector.
Figure 3: Images without and with perturbations added by our ImageProtector under different ℓ∞ -norm constraint ϵ .
Dataset
# Image-question Pairs
# Ground-truth Answers
VQAv2
1,105,904
11,059,040
GQA
22,669,678
22,669,678
TextVQA
45,336
453,360
CelebA
202,599
0
Table 1: Dataset statistics.
Method
Exact Probing Questions
Similar Probing Questions
General Probing Questions
No Perturbation
0.00
0.00
0.00
Qi et al. ( Qi et al., 2024 )
0.02
0.02
0.02
Bagdasaryan et al. ( Bagdasaryan et al., 2023 )
0.65
0.62
0.51
ImageProtector+PGD
0.94
0.91
0.91
ImageProtector
0.94
0.88
0.88
Table 2: Refusal rates of compared methods and ImageProtector using three types of shadow questions, with LLaVA-1.5 as the target MLLM on VQAv2 dataset.
Method
Exact Probing Questions
Similar ProbingUser Questions
General ProbingUser Questions
ImageProtector+PGD
16.2
61.2
61.2
ImageProtector
10.2
45.6
45.6
Table 3: GPU-minutes of ImageProtector and ImageProtector + PGD for optimizing perturbation per image, with LLaVA-1.5 as the target MLLM on VQAv2 dataset.
Method
Exact Probing Questions
Similar ProbingUser Questions
General Probing Questions
No Perturbation
0.92
0.92
0.92
Qi et al. ( Qi et al., 2024 )
0.48
0.48
0.48
Bagdasaryan et al. ( Bagdasaryan et al., 2023 )
0.03
0.04
0.03
ImageProtector+PGD
0.03
0.03
0.04
ImageProtector
0.03
0.04
0.03
Table 4: Accuracy of compared methods and ImageProtector using three types of shadow questions, with LLaVA-1.5 as the target MLLM on the VQAv2 dataset.
Table 5: Refusal rates of ImageProtector with exact probing questions as shadow questions when using (a) image-relevant and (b) image-irrelevant questions on four MLLMs and four datasets. ‘Avg.’ denotes average.
Table 6: Refusal rates of ImageProtector with similar probing questions as shadow questions when using (a) image-relevant and (b) image-irrelevant questions on four MLLMs and datasets. ‘Avg.’ denotes average.
Table 7: Refusal rates of ImageProtector with general probing questions as shadow questions when using (a) image-relevant and (b) image-irrelevant questions on four MLLMs and datasets. ‘Avg.’ denotes average.
Figure 4: Refusal rates of ImageProtector with multiple target MLLMs. The VQAv2 dataset is used, with general probing questions being used as shadow questions. Each row indicates the set of MLLMs jointly included in the optimization objective when generating one universal perturbation per image. Each column reports the refusal rate measured on a specific evaluation model. For example, the third row means that the perturbation was jointly optimized against LLaVA-1.5, MiniGPT-4, and Qwen-VL-Chat, and then evaluated on each of the four models shown as columns.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Impact of the temperature of the target MLLM on ImageProtector. We use general probing questions as shadow questions with LLaVA-1.5 on VQAv2.
MLLM
Vision Encoder (# Parameters)
LLM (# Parameters)
Vision-Language Projector (# Parameters)
LLaVA-1.5
CLIP ViT-L/14 ( Radford et al., 2021 ) (428M)
Llama-2 ( Touvron et al., 2023 ) (7B)
2-layer FFN (10M)
MiniGPT-4
EVA-CLIP ViT-g/14 ( Fang et al., 2023 ) (1B)
Llama-2 (7B)
1-layer FFN (23M)
Qwen-VL-Chat
OpenCLIP ViT-bigG ( Ilharco et al., 2021 ) (2B)
Qwen ( Bai et al., 2023a ) (7B)
1-layer Cross-Attention ( Lin et al., 2022 ) (76M)
InstructBLIP
EVA-CLIP ViT-g/14 (1B)
Vicuna ( Chiang et al., 2023 ) (7B)
Q-Former ( Li et al., 2023 ) (186M)
Phi-4-multimodal-instruct
SigLIP ( Zhai et al., 2023 ) (400M)
Phi4 Mini ( Abouelenin et al., 2025 ) (6B, mixture of LoRAs design)
2-layer MLP (40M)
Qwen2.5-VL-7B-Instruct
redesigned ViT (632M)
Qwen 2.5 ( Bai et al., 2025 ) (8B)
MLP-based Merger (45M)
Appendix
Table 8: Configurations of MLLMs.
Figure 6: Impact of the number of repeated question answering rounds. We use three types of shadow questions with LLaVA-1.5 on VQAv2.
Figure 7: Impact of step size on ImageProtector. We evaluate three types of shadow questions with LLaVA-1.5 on VQAv2.
Figure 8: Impact of the maximum number of iterations on ImageProtector. We use three types of shadow questions with LLaVA-1.5 on VQAv2.
Figure 9: Impact of (a) ℓ∞ -norm perturbation constraint ϵ , (b) mini-batch size of shadow questions, (c) the size of shadow questions on ImageProtector. We use general probing questions as shadow questions with LLaVA-1.5 on VQAv2.
Figure 10: Accuracy and refusal rates (RR) of ImageProtector with (a) adding Gaussian noise and (b) using DiffPure. We use three types of shadow questions with LLaVA-1.5 on VQAv2.
Figure 11: Accuracy of the target MLLM and refusal rates of ImageProtector when using adversarial training with different training epochs. We use three types of shadow questions with LLaVA-1.5 on VQAv2.
Figure 12: Prompt to generate similar probing questions where [Example_Question] denotes an example question.
Figure 13: Prompt to generate general probing questions where [Q] denotes the number of shadow questions to generate.
Figure 14: Prompt to a refusal judge LLM. [MLLM_Response] represents the response from an MLLM.
Figure 15: Example prompt to generate general probing questions and example response from GPT-4.
Figure 16: The set of refusal responses.
Figure 17: Example prompt to generate similar probing questions and example response from GPT-3.5.
Privacy protection in vision-language models limits sensitive information exposure while preserving useful image understanding and multimodal reasoning.
Multimodal Large Language Models (MLLMs) integrate vision and text to power applications, but this integration introduces new vulnerabilities. We study Image-based Prompt Injection (IPI), a black-box attack in which adversarial instructions are embedded into natural images to override model behavior. Our end-to-end IPI pipeline incorporates segmentation-based region selection, adaptive font scaling, and background-aware rendering to conceal prompts from human perception while preserving model interpretability. Using the COCO dataset and GPT-4-turbo, we evaluate 12 adversarial prompt strategies and multiple embedding configurations. The results show that IPI can reliably manipulate the output of the model, with the most effective configuration achieving up to 64% attack success under stealth constraints. These findings highlight IPI as a practical threat in black-box settings and underscore the need for defenses against multimodal prompt injection.
Neha Nagaraja, Lan Zhang, Zhilong Wang +2
School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, USA · Bytedance, CA, USA
Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt injection attacks. Existing prompt injection methods predominantly rely on textual prompts or perceptible visual prompts that are observable by human users. In this work, we study imperceptible visual prompt injection against powerful closed-source MLLMs, where adversarial instructions are embedded in the visual modality. Our method adaptively embeds the malicious prompt into the input image via a bounded text overlay to provide semantic guidance. Meanwhile, the imperceptible visual perturbation is iteratively optimized to align the feature representation of the attacked image with those of the malicious visual and textual targets at both coarse- and fine-grained levels. Specifically, the visual target is instantiated as a text-rendered image and progressively refined during optimization to more faithfully represent the desired semantics and improve transferability. Extensive experiments on two multimodal understanding tasks across multiple closed-source MLLMs demonstrate the superior performance of our approach compared to existing methods.
Meiwen Ding, Song Xia, Chenqi Kong +1
Rapid-Rich Object Search Lab, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore.
Privacy risks in text-only Large Language Models (LLMs) are well studied, particularly their tendency to memorize and leak sensitive information. However, Multi-modal Large Language Models (MLLMs), which process both text and images, introduce unique privacy challenges that remain underexplored. Compared to text-only models, MLLMs can extract and expose sensitive information embedded in images, posing new privacy risks. We reveal that some MLLMs are susceptible to privacy breaches, leaking sensitive data embedded in images or stored in memory. Specifically, in this paper, we (1) introduce MM-Privacy, a comprehensive dataset designed to assess privacy risks across various multi-modal tasks and scenarios, where we define Disclosure Risks and Retention Risks. (2) systematically evaluate different MLLMs using MM-Privacy and demonstrate how models leak sensitive data across various tasks, and (3) provide additional insights into the role of task inconsistency in privacy risks, emphasizing the urgent need for mitigation strategies. Our findings highlight privacy concerns in MLLMs, underscoring the necessity of safeguards to prevent data exposure. Our dataset and code can be found here.
Tiejin Chen, Pingzhi Li, Kaixiong Zhou +2
Arizona State University · University of North Carolina at Chapel Hill · North Carolina State University