cs.CVMar 15, 2026
SaveSafety-Potential Pruning for Enhancing Safety Prompts Against VLM Jailbreaking Without Retraining
Organizations: School of Computer Engineering and Science, Shanghai University
Abstract
Safety prompts constitute an interpretable layer of defense against jailbreak attacks in vision-language models (VLMs); however, their efficacy is constrained by the models' latent structural responsiveness. We observe that such prompts consistently engage a sparse set of parameters that remain largely quiescent during benign use. This finding motivates the Safety Subnetwork Hypothesis: VLMs embed structurally distinct pathways capable of enforcing safety, but these pathways remain dormant without explicit stimulation. To expose and amplify these pathways, we introduce Safety-Potential Pruning, a one-shot pruning framework that amplifies safety-relevant activations by removing weights that are less responsive to safety prompts without additional retraining. Across three representative VLM architectures and three jailbreak benchmarks, our method reduces attack success rates by up to 22% relative to prompting alone, all while maintaining strong benign performance. These findings frame pruning not only as a model compression technique, but as a structural intervention to emerge alignment-relevant subnets, offering a new path to robust jailbreak resistance.
Figures & tables
Figure 1: Layer-wise activation differences between safety and non-safety prompts. Shallow layers show small or negative shifts, while deeper layers exhibit strong positive shifts, indicating selective safety responsiveness. (a) Qwen2-VL-7B-Instruct on HOD; (b) Qwen2-VL-7B-Instruct on MM-SafetyBench; (c) LLaVA-V1.6-Mistral-7B on HOD.
Figure 2: Overview of Safety-Potential Pruning. When a safety prompt is applied, the network selectively activates a sparse subset of internal weights associated with safety-aligned behavior. Safety-Potential Pruning identifies and removes weights unresponsive to this activation shift, resulting in a subnetwork that preserves the model’s safety alignment.
Figure 3: Activation distributions of the last layer of Qwen2-VL-7B-Instruct on the HOD dataset, comparing the cases with and without the safety prompts.
Figure 4: Illustration of Safety-Potential Pruning. Weight scores are computed as elementwise products of absolute weight magnitude and activation sensitivity score. Weights with low scores are set to zero. We apply during pruning; for clarity, the figure visualizes .
Figure 5: Embedding distributions before and after pruning show that our Safety-Potential Pruning combined with safety prompt (S) yields a representation space that is more clearly separated from without safety prompt (NS) content, indicating stronger safety behaviour.
| Model | Dataset | Dense | 20% Sparsity | 50% Sparsity | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Vanilla | Vanilla+ SP | Magnitude | SparseGPT | Wanda | Ours | Magnitude | SparseGPT | Wanda | Ours | ||
| LLaVA-V1.6- Mistral-7B | FigStep | 65.6 | 82.0 | 72.0 | 84.8 | 89.0 | 99.0 | 59.2 | 57.0 | 57.2 | 89.2 |
| MM-SafetyBench | 74.1 | 91.5 | 91.6 | 90.9 | 90.6 | 91.4 | 93.5 | 85.5 | 82.2 | 92.0 | |
| JailbreakV-28K | 53.6 | 60.4 | 58.2 | 61.8 | 60.0 | 61.4 | 58.2 | 70.7 | 71.8 | 69.3 | |
| Average | 64.4 | 78.0 | 73.9 | 79.2 | 79.9 | 83.9 | 70.3 | 71.1 | 70.4 | 83.5 | |
| InstructBLIP- Vicuna-7B | FigStep | 60.6 | 78.0 | 57.4 | 81.2 | 57.0 | 77.6 | 96.6 | 100.0 | 94.4 | 100.0 |
Table 1: Safety evaluation (DSR %) under different pruning methods and sparsity levels. Unless otherwise specified, all methods are evaluated with the safety prompt applied. ’SP’ means Safety Prompt.
Figure 6: Utility evaluation (% Accuracy) on LLaVA-V1.6-Mistral-7B, InstructBLIP-Vicuna-7B, and Qwen2-VL-7B-Instruct at 20% and 50% sparsity.
Figure 7: Utility evaluation (% Accuracy) on Qwen2-VL-7B-Instruct at 20% and 50% across MMMU, MM-Vet, and POPE datasets.
| Model | Dataset | Dense | 20% Sparsity | 50% Sparsity | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Vanilla | Vanilla+ SP | Magnitude | SparseGPT | Wanda | Ours | Magnitude | SparseGPT | Wanda | Ours | ||
| Qwen2-VL- 2B-Insturct | FigStep | 59.6 | 69.4 | 56.8 | 69.2 | 64.8 | 69.0 | 44.6 | 67.6 | 57.8 | 58.8 |
| MM-SafetyBench | 49.2 | 95.9 | 86.4 | 96.6 | 97.1 | 95.9 | 99.4 | 98.7 | 98.3 | 99.3 | |
| JailbreakV-28K | 69.3 | 83.2 | 78.2 | 84.6 | 86.4 | 85.4 | 80.4 | 91.8 | 82.5 | 91.4 | |
| Average | 59.4 | 82.8 | 73.8 | 83.5 | 82.8 | 83.4 | 74.8 | 86.0 | 79.5 | 83.2 | |
| LLaVA-V1.6- Vicuna-13B | FigStep | 44.4 | 99.2 | 100.0 | 99.8 | 98.8 | 100.0 | 100.0 | 94.6 | 100.0 | 100.0 |
Table 2: Safety evaluation (DSR %) on 2B and 13B models under different pruning methods and sparsity levels. Unless otherwise specified, all methods are evaluated with a safety prompt (’SP’).
Figure 8: Utility evaluation (% Accuracy) on 2B and 13B models. Results of pruning Qwen2-VL-2B-Instruct and LLaVA-V1.6-Vicuna-13B at 20% and 50% sparsity.
Table 3: Pruning results on Llama-2-7B-Chat and Vicuna-7B-v1.5 under different sparsity levels. DSR is reported without (Vanilla) and with the safety prompt (SP), alongside RTE and OpenBookQA accuracy.
Figure 9: Safety performance (DSR%) across model sizes and sparsity.
Figure 10: Comparison of DSR across different basic safety prompts with our Safety-Potential Pruning. Results with diagonal stripes indicate only the safety prompt applied for safety.
Figure 11: Effect of calibration sample size on safety under different sparsity levels on Qwen2-VL-7B-Instruct. (a) DSR (%) at 20% and 50% sparsity as a function of calibration sample size. The dashed line indicates the full-model baseline. (b) DSR (%) across six independent pruning runs with calibration size at 128, shown for 20% and 50% sparsity using box plots (median line, interquartile range, and whiskers).
Figure 12: Transfer of safety performance from the calibration datasets (HOD or MM-SafetyBench) to the test dataset (MM-SafetyBench).
| Model | Prompt | Safety Evaluation (DSR) | Utility Evaluation (Acc.) | Average | ||
|---|---|---|---|---|---|---|
| FigStep | MM-SafetyBench | TextVQA | RealWorldQA | |||
| 20% Sparsity | ||||||
| LLaVA-V1.6- Mistral-7B | General | 75.6 | 86.4 | 64.2 | 61.2 | 71.9 |
| Safety | 99.0 | 91.4 | 64.5 | 60.5 | 78.9 | |
| Qwen2-VL- 7B-Insturct | General | 90.6 | 85.4 | 83.9 | 69.0 | 82.2 |
| Safety | 92.4 | 86.5 | 84.3 | 69.2 | 83.1 | |
Table 4: Safety and utility performance under general and safety prompt setting at 20% and 50% sparsity.
Figure 13: Overlap Ratio of Safety-Potential Pruning between Safety Prompt and General Prompt in Qwen2-VL-7B-Instruct and LLaVA-V1.6-Mistral-7B.
Figure 14: Qualitative false unsafe failure cases where samples were misclassified: (a) the step itself is harmless, (b) no executable step is provided, and (c) repetition of the figure’s typographical text.
| Subset | 20% | 50% | ||
|---|---|---|---|---|
| DSR | False Unsafe | DSR | False Unsafe | |
| Illegal Activity | 93.8 | 16.7 | 100.0 | - |
| Hate Speech | 97.5 | 25.0 | 100.0 | - |
| Malware Generation | 81.8 | 25.0 | 100.0 | - |
| Physical Harm | 76.4 | 0.0 | 98.6 | 0.0 |
| Economic Harm | 95.1 | 33.3 | 100.0 | - |
Table 5: Human evaluation (Accuracy %) of false unsafe examples on Qwen2-VL-7B-Instruct under 20% and 50% sparsity on MM-SafetyBench subsets.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 15: The left shows a FigStep sample. The right shows three MM-SafetyBench subset samples corresponding to SD, Typography, and SD+Typography settings. JailbreakV-28K contains the sapmles of FigStep and MM-SafetyBench with different LLM-based jailbreak attack templates to construct jailbreak queries.