Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMs

Ming Wen; Kun Yang; Xin Chen; Jingyu Zhang; Dingding Han; Shiwen Cui; Yuedong Xu

cs.LGFeb 28, 2026

Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMs

Authors: Ming Wen, Kun Yang, Xin Chen, Jingyu Zhang, Dingding Han, Shiwen Cui, Yuedong Xu

Organizations: Fudan University · Shanghai Innovation Institute · Ant Group · Zhejiang University ·

Abstract

Multimodal Large Language Models (MLLMs) pose critical safety challenges, as they are susceptible not only to adversarial attacks such as jailbreaking but also to inadvertently generating harmful content for benign users. While internal safety alignment via Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) is a primary mitigation strategy, current methods often face a safety-utility trade-off: they either refuse benign queries out of excessive caution or overlook latent risks in cross-modal interactions. To resolve this, we introduce Pragma-VL, an end-to-end alignment algorithm that enables MLLMs to pragmatically arbitrate between safety and helpfulness. First, we enhance visual risk perception with a novel cold-start SFT stage. This is achieved by applying risk-aware clustering to the visual encoder and using an interleaved dataset of risk descriptions and high-quality data. Second, we introduce a theoretically-guaranteed reward model that leverages synergistic learning. We train it with a novel data augmentation method that assigns dynamic weights based on the queries, enabling contextual arbitration between safety and helpfulness. Extensive experiments show that Pragma-VL effectively balances safety and helpfulness, outperforming baselines by 5% to 20% on most multimodal safety benchmarks while preserving its general capabilities in areas such as mathematics and knowledge reasoning.

Figures & tables

Figure 1: The dual failure modes of static safety policies in MLLMs. Our work aims to train a pragmatic model that dynamically arbitrates safety and helpfulness trade-off based on the context.

Figure 2: (a) Overview of Pragma-VL, which train the MLLM to perform context-aware dynamic arbitration, achieving a flexible balance between safety and helpfulness. (b) An illustration of our Contextual Data Augmentation Pipeline.

Figure 3: Pragma-VL Algorithm Pipeline.(a) MLLM Cold-Start (b) Prompt Regulated Reward

Architecture	Helpfulness Acc. $\uparrow$		Harmlessness Acc. $\uparrow$		Weighted Acc. $\uparrow$
	$\Delta\geq 2$	$\Delta\geq 4$	$\Delta\geq 2$	$\Delta\geq 4$	$\Delta\geq 2$	$\Delta\geq 4$
Single	–	–	–	–	79.1 $\pm$ 0.8	81.4 $\pm$ 0.6
Sequential	92.6 $\pm$ 0.5	96.5 $\pm$ 0.6	87.9 $\pm$ 0.4	98.2 $\pm$ 0.5	85.5 $\pm$ 0.7	86.8 $\pm$ 0.5
Parallel (Ours)	94.6 $\pm$ 0.4	98.2 $\pm$ 0.2	92.6 $\pm$ 0.5	98.2 $\pm$ 0.4	96.3 $\pm$ 0.4	98.7 $\pm$ 0.3

Table 1: Preference accuracy of different reward model architectures on the PragmaSafe validation set.

\Delta

refers to the labeled score difference between the chosen and rejected pair.

Qwen2.5-VL-7B
Model/Experiment	Beavertails-V(%)		SPA-VL(%		MM-SafetyBench(%)			SIUO(%)		MSSbench(%)
Model/Experiment	Help	Harmless	Help	Harmless	Help	Harmless	ASR $\downarrow$	Effective	Safety	Effective	Safety
Qwen2.5-VL-7B	50.00	50.00	50.00	50.00	50.00	50.00	48.75	92.17	38.78	98.48	36.53
Beavertails-V_harm	37.07	48.63	26.88	45.66	27.44	45.18	43.29	89.76	59.64	99.15	50.50
Beavertails-V_help	49.91	43.29	40.47	29.43	54.94	52.38	51.07	95.20	34.33	98.98	32.54
Beavertails-V_all	45.84	56.12	37.71	51.69	38.97	51.68	49.58	92.59	51.23	98.65	45.45

Table 2: Comprehensive evaluation results across multiple safety benchmarks. Help and Harm metrics are evaluated using Win Rate. For each model category (Qwen, Llava), the best-performing experiment in each column is highlighted in bold , the second-best is underlined , and the Pragma-VL experiment row is highlighted.

Model/Experiment	GQA(%)	ScienceQA(%)	Textvqa(%)	Vizwizqa(%)	Vqav2(%)	MathVista(%)
Qwen2.5-VL-7B	60.74	88.48	83.75	72.53	83.60	67.80
Beavertails-V_harm	56.25	85.93	78.32	64.26	80.31	51.80
Beavertails-V_help	59.57	86.06	82.84	68.85	81.97	48.40
SPA-VL	57.61	86.32	80.31	71.65	82.99	62.60
MM-RLHF	59.03	87.45	83.26	68.07	82.09	50.70
SFT	59.57	89.01	81.83	68.64	81.29	66.50

Table 3: Performance comparison on various general ability benchmarks. For each model category (Qwen, Llava), the best-performing experiment in each column is highlighted in bold , and the second-best is underlined . The Pragma-VL experiment row is highlighted for emphasis.

Figure 4: Ablation study of the Pragma-VL framework. Results consistently demonstrate that the full Pragma-VL framework outperforms its individual components, highlighting the synergistic effect of combining risk-aware pre-alignment with subsequent policy alignment.

Model/Experiment	Beavertails-V (%)		SPA-VL (%)		MM-SafetyBench (%)			SIUO (%)		MSSbench (%)
Model/Experiment	Help	Harmless	Help	Harmless	Help	Harmless	ASR $\downarrow$	Effective	Safety	Effective	Safety
Pre-RL Stage
EC	52.12	51.10	55.19	50.37	51.25	49.22	43.40	94.44	33.33	98.82	37.87
SFT	53.98	60.61	56.04	56.79	47.31	53.92	44.03	89.50	40.12	96.46	42.92
EC+SFT	58.70	65.53	70.45	65.28	48.09	55.01	41.13	88.62	48.79	97.31	43.09
RL Stage

Table 4: Ablation study on Qwen2.5-VL-7B. Abbreviations: EC (Encoder Clustering via Contrastive Learning), SFT (Supervised Fine-Tuning), and GRPO (Group Relative Policy Optimization).

Contextual Data Augmentation (Section 3.1 )
Symbol	Description
$\mathbf{W}_{\text{base}}$	The initial base weight vector derived from majority voting of annotations, $\mathbf{W}_{\text{base}}=[w_{h},w_{s}]$ .
$\mathbf{W}_{\text{final}}$	The final, variance-adjusted weight vector used for training.
$\sigma^{2}_{h},\sigma^{2}_{s}$	The variance of the helpfulness and harmlessness scores across 5 annotations, respectively.
$\mathcal{T}(\cdot)$	The targeting function that determines the direction of weight adjustment based on variance.
$\mathcal{N}(0,\sigma^{2})$	A Gaussian distribution with mean 0 and variance $\sigma^{2}$ .

Table 5: Summary of Mathematical Notations

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: (a) The distribution of items across all categories. (b) Score distributions for helpfulness, safety, and weighted metrics (top), with the corresponding word length distribution for each score bin (bottom).

Category	Original	Filtered	Retention(%)	Helpness Avg	Safety Avg	Help W Avg	Safety W Avg	Ans Len Avg
animal_abuse	9468	7502	79.24%	0.46	0.30	0.36	0.64	1255.21
dangerous_behavior	15726	12456	79.21%	0.77	0.88	0.33	0.67	1495.23
deception_in_personal_relationships	4524	3598	79.53%	0.57	0.31	0.46	0.54	1304.79
discriminatory_depictions	3546	2858	80.60%	1.13	0.24	0.62	0.38	1570.57
environmental_damage	14262	5922	41.52%	0.42	0.14	0.36	0.64	1687.79
false_information	4620	1498	32.42%	0.65	0.37	0.44	0.56	1503.37

Appendix

Table 6: Statistics of original and filtered samples for each safety category.

Figure 6: Safety-Dominant data example in PragmaSafe.

Figure 7: Helpfulness-Dominant data example in PragmaSafe.

Model/Experiment	Beavertails-V (%)		SPA-VL (%)		MM-SafetyBench (%)			SIUO (%)		MSSbench (%)
Model/Experiment	Help	Harmless	Help	Harmless	Help	Harmless	ASR $\downarrow$	Effective	Safety	Effective	Safety
Pre-RL Stage
EC	49.31	48.64	51.01	51.55	49.04	47.25	57.01	87.04	15.53	98.14	24.92
SFT	77.75	83.66	82.72	85.66	50.23	52.14	42.28	89.15	33.33	95.62	37.10
EC+SFT	83.36	86.24	90.04	89.77	49.82	54.05	38.51	88.02	39.15	97.13	39.90
RL Stage

Appendix

Table 7: Ablation study on Llava-1.5-7B. We compare the performance across the Pre-RL Stage (EC, SFT, EC+SFT) and the RL Stage (GRPO, SFT+GRPO, Pragma-VL).

Figure 8: Example before and after MLLM Cold-Start.

Figure 9: Visual Example of images with risk severity labels.

Figure 10: Helpfulness and Harmlessness Score of Beavertails-V Benchmark (categorized). (a) Comparison between Llava-1.5-7B and Llava with Pragma-VL (b) Comparison between Qwen2.5-VL-7B and Qwen with Pragma-VL

Figure 11: Attack Success Rate of MMSafety Benchmark (categorized). (a) Comparison between Llava-1.5-7B and Llava with Pragma-VL (b) Comparison between Qwen2.5-VL-7B and Qwen with Pragma-VL

Figure 12: Example before and after Pragma-VL Pipeline(Qwen2.5-VL-7B).

Figure 13: Example before and after Pragma-VL Pipeline(Llava-1.5-7B).

Figure 14: Visual example for three reward structure after GRPO on harm-dominant query (Qwen2.5-VL-7B).

Figure 15: Visual example for three reward structure after GRPO on help-dominant query (Qwen2.5-VL-7B).

Reward Arch.	Beavertails-V(%)		SPA-VL(%)		MM-Safety(%)			SIUO(%)
Reward Arch.	Help	Harmless	Help	Harmless	Help	Harmless	ASR $\downarrow$	Effective	Safety
Qwen2.5-VL-7B	50.00	50.00	50.00	50.00	50.00	50.00	48.75	92.17	38.78
par_grpo	62.65	67.91	87.17	87.92	52.74	58.99	31.66	95.21	63.47
seq_grpo	51.44	52.63	38.40	48.30	56.37	53.27	48.45	95.81	39.16
single_grpo	13.94	29.08	7.98	29.08	9.29	24.79	37.30	46.70	41.91

Appendix

Table 8: RL-Alignment performance comparison of different reward model architectures on the Qwen2.5-VL-7B backbone. Help and Harm are evaluated with Win Rate (%). par_grpo denotes parallel reward, seq_grpo denotes sequential reward, and single_grpo denotes single head reward.

Multimodal Large Language Models

Multimodal large language models combine language models with image, video, audio, and other inputs for grounded understanding and generation.

See all 1,091 papers

Towards Multi-modal Multi-turn Safety: From Agentic Interaction to Strategic Alignment
Jan 8, 2026Han Zhu, Jiale Chen, Chengkun Cai +8Language Model Safety Evaluation Multimodal Large Language Models
Hong Kong University of Science and Technology · University College London · AISpeech +1
When Reasoning Is Not Enough: Proactive Inquiry for Cross-Modal Safety in Multimodal Large Language Models
Oct 15, 2026Wei Cai, Jian Zhao, Yuchen Yuan +1Multimodal Multi-Turn Safety Alignment VLM Safety
Institute of Artificial Intelligence (TeleAI), China Telecom, P. R. China. · Peking University. · School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, P. R. China.
GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models
Feb 27, 2026Xingyu Zhu, Beier Zhu, Junfeng Fang +4VLM Safety Alignment VLM Safety
MoE Key Lab of BIPC, University of Science and Technology of China · Nanyang Technological University · National University of Singapore +1

Jan 8, 2026J/K move · Enter open · S save

Towards Multi-modal Multi-turn Safety: From Agentic Interaction to Strategic Alignment

Han Zhu, Jiale Chen, Chengkun Cai, Shengjie Sun+7

Hong Kong University of Science and Technology · University College London · AISpeech+1

Language Model Safety Evaluation Multimodal Large Language Models Multimodal Multi-Turn Safety Alignment LLM Safety Alignment Multimodal Model Evaluation MLLM Safety

Open paper AskPDF arXiv

Despite remarkable capability in multi-modal understanding, deploying Multi-modal Large Language Models (MLLMs) in open-ended conversational scenarios introduces safety risks that remain poorly addressed by existing alignment methods. Unlike simple malicious visual question and answer (VQA) pairs , multi-turn interactions enable adversaries to incrementally reconstruct harmful intent across dialogues, progressively bypassing safety constraints in ways that are difficult to detect at any individual turn. Meanwhile, conventional reinforcement learning from human feedback (RLHF) approaches are unsuitable for this situation: designed primarily for VQA tasks, they neither capture cross-turn risk dynamics nor scale efficiently without costly manual preference annotation. To close this gap, we introduce MINT-Safe, an open-source visual multi-turn training dataset comprising 11,270 multi-image dialogues and 500 refusal VQA pairs, constructed via multi-agent interaction with text-to-image (T2I) tool-call augmentation. Building on MINT-Safe, we propose TAD-Align, a dialogue safety alignment framework centered on a turn-aware dual-objective reward function. Rather than treating all dialogue turns uniformly, TAD-Align leverages rollout-based safety score variance to dynamically identify turns where the model exhibits inconsistent safety behavior, and adaptively up-weights these turns during optimization. Experiments on Qwen2.5-VL-7B-Instruct and LLaVA-NeXT-7B demonstrate reductions of over 10% in Attack Success Rate (ASR), alongside improvements of at least 8% in harmlessness and 13% in helpfulness on multi-modal multi-turn safety benchmarks, while preserving general model capabilities.

Abstract

Multimodal Large Language Models

Explore similar work

Towards Multi-modal Multi-turn Safety: From Agentic Interaction to Strategic Alignment

When Reasoning Is Not Enough: Proactive Inquiry for Cross-Modal Safety in Multimodal Large Language Models

GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models