Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memory

Ce Zhang; Jinxi He; Junyi He; Katia Sycara; Yaqi Xie

cs.CVMar 16, 2026

Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memory

Authors: Ce Zhang, Jinxi He, Junyi He, Katia Sycara, Yaqi Xie

Organizations: Robotics Institute, Carnegie Mellon University

Abstract

Multi-modal Large Language Models (MLLMs) have achieved remarkable performance across a wide range of visual reasoning tasks, yet their vulnerability to safety risks remains a pressing concern. While prior research primarily focuses on jailbreak defenses that detect and refuse explicitly unsafe inputs, such approaches often overlook contextual safety, which requires models to distinguish subtle contextual differences between scenarios that may appear similar but diverge significantly in safety intent. In this work, we present MM-SafetyBench++, a carefully curated benchmark designed for contextual safety evaluation. Specifically, for each unsafe image-text pair, we construct a corresponding safe counterpart through minimal modifications that flip the user intent while preserving the underlying contextual meaning, enabling controlled evaluation of whether models can adapt their safety behaviors based on contextual understanding. Further, we introduce EchoSafe, a training-free framework that maintains a self-reflective memory bank to accumulate and retrieve safety insights from prior interactions. By integrating relevant past experiences into current prompts, EchoSafe enables context-aware reasoning and continual evolution of safety behavior during inference. Extensive experiments on various multi-modal safety benchmarks demonstrate that EchoSafe consistently achieves superior performance, establishing a strong baseline for advancing contextual safety in MLLMs. All benchmark data and code are available at https://echosafe-mllm.github.io.

Figures & tables

Figure 1: Comparison of different approaches for enhancing MLLM safety. (a) Qualitative comparison of generated responses: prior methods [ 50 , 13 ] often exhibit over-defensive behavior, whereas our EchoSafe produces contextually appropriate responses; (b) Quantitative comparison on MM-SafetyBench++: EchoSafe consistently outperforms prior methods in both contextual correctness rate (CCR) and response quality score (QS).

Proprietary Models
Method	Illegal Activitiy			Hate Speech			Malware Generation			Physical Harm			Fraud			Sex
	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM
	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS
GPT-5	85.6 / 4.3	99.0 / 4.9	91.9 / 4.6	87.1 / 4.3	100.0 / 5.0	93.1 / 4.6	79.6 / 3.9	100.0 / 4.9	88.6 / 4.3	90.3 / 4.5	100.0 / 5.0	94.9 / 4.8	75.3 / 3.8	100.0 / 5.0	85.9 / 4.3	43.1 / 2.1	100.0 / 4.9	60.2 / 3.1
GPT-5-Mini	85.6 / 4.3	100.0 / 4.8	92.2 / 4.5	86.5 / 4.3	100.0 / 4.8	92.7 / 4.5	77.3 / 3.8	100.0 / 4.8	87.2 / 4.3	93.1 / 4.6	100.0 / 4.9	96.4 / 4.8	79.2 / 4.0	100.0 / 5.0	88.4 / 4.4	34.9 / 1.7	100.0 / 4.7	51.7 / 2.5
GPT-4o-Mini	74.2 / 0.8	85.6 / 3.4	79.5 / 1.5	68.1 / 0.9	87.7 / 3.6	76.7 / 1.6	63.6 / 0.8	95.5 / 3.7	76.4 / 1.4	66.7 / 0.8	85.4 / 3.4	74.9 / 1.4	50.0 / 0.6	96.8 / 3.9	65.6 / 1.1	42.2 / 1.2	83.5 / 3.1	55.9 / 1.7

Table 1: Evaluation of state-of-the-art MLLMs on MM-SafetyBench++ under the Gen mode . We report Refusal Rate / Quality Score (RR / QS) for unsafe inputs, Answer Rate / Quality Score (AR / QS) for safe inputs, and their harmonic mean (HM). Higher (

\uparrow

) values indicate better performance. All evaluations use gpt-5-mini as the judge. Best results are bolded ; second-best are underlined .

Metric	MM-SafetyBench	MM-SafetyBench++	$\boldsymbol{\Delta}$
CLIPScore [ 37 ] $\uparrow$	0.21	0.25	0.04
BLIP2Score [ 20 ] $\uparrow$	0.52	0.67	0.15
InceptionScore [ 39 ] $\downarrow$	13.53	8.32	-5.20
ASR $\uparrow$	22.72	41.04	18.32

Table 2: Comparison of image quality and attack performance between the MM-SafetyBench and the unsafe subset of MM-SafetyBench++. Higher CLIPScore and BLIP2Score with lower Inception Score indicate improved generation quality. With greater data fidelity, MM-SafetyBench++ offers a more challenging benchmark for evaluating MLLM contextual safety.

Figure 2: An overview of our proposed EchoSafe framework. At each inference step $t$ , the model retrieves the top- $k$ most relevant safety insights from the memory bank $\mathcal{M}^{(t-1)}$ based on contextual similarity. The retrieved insights serve as prior safety guidance for responding to the current query. After generating a response, the model performs self-reflection to derive a new safety insight $I^{(t)}$ , which is added into the memory together with its corresponding context embedding $\mathbf{e}^{(t)}$ to enable continual evolution.

	Method	Illegal Activitiy			Hate Speech			Malware Generation			Physical Harm			Fraud			Sex
		Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM
		RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS
LLaVA-1.5-7B	Base [ 23 ]	4.1 / 0.2	100.0 / 3.1	7.9 / 0.4	9.2 / 0.4	99.4 / 3.3	16.8 / 0.7	2.3 / 0.1	100.0 / 3.0	4.5 / 0.2	4.2 / 0.2	100.0 / 3.2	8.1 / 0.4	0.0 / 0.0	100.0 / 3.2	0.0 / 0.0	7.3 / 0.3	100.0 / 3.3	13.6 / 0.6
	+ FigStep [ 13 ]	76.3 / 1.8	80.4 / 2.5	78.3 / 2.1	82.2 / 2.4	65.0 / 2.0	72.5 / 2.2	68.2 / 1.6	72.7 / 2.1	70.4 / 1.8	58.3 / 1.6	84.0 / 2.6	68.9 / 2.0	67.5 / 1.8	76.0 / 2.3	71.5 / 2.0	38.5 / 1.0	89.9 / 2.9	53.9 / 1.5
	+ ECSO [ 14 ]	37.1 / 1.2	100.0 / 3.1	54.1 / 1.7	34.6 / 1.4	100.0 / 3.3	51.4 / 2.0	18.2 / 0.7	100.0 / 3.0	30.8 / 1.1	22.9 / 0.9	100.0 / 3.2	37.3 / 1.4	22.1 / 0.8	99.4 / 3.2	36.2 / 1.3	11.0 / 0.4	100.0 / 3.3	19.8 / 0.7
	+ AdaShield [ 50 ]	79.4 / 1.0	51.6 / 1.4	62.6 / 1.2	95.1 / 1.1	43.6 / 1.3	59.8 / 1.2	90.9 / 1.1	45.5 / 1.3	60.6 / 1.2	77.1 / 1.0	31.3 / 0.9	44.5 / 0.9	82.5 / 0.9	34.4 / 1.0	48.6 / 0.9	78.0 / 1.0	38.5 / 1.1	51.6 / 1.0

Table 3: Performance comparison on MM-SafetyBench++ under the Gen attack mode . Higher (

\uparrow

) values indicate better performance. All evaluations are performed with gpt-5-mini as the judge. Best results are bolded , and second-best results are underlined .

	Method	MM-SafetyBench [ 26 ]			MSSBench-Chat [ 61 ]			MSSBench-Embodied [ 61 ]			SIUO [ 46 ]			Comprehensive Benchmarks
	Method	SD $\downarrow$	TYPO $\downarrow$	SD-TYPO $\downarrow$	Safe $\uparrow$	Unsafe $\uparrow$	Avg. $\uparrow$	Safe $\uparrow$	Unsafe $\uparrow$	Avg. $\uparrow$	S $\uparrow$	S&E $\uparrow$	R $\uparrow$	MME ${}^{\text{P}}$ $\uparrow$	MME ${}^{\text{C}}$ $\uparrow$	MMB $\uparrow$	SQA $\uparrow$	VQA ${}^{\text{Text}}$ $\uparrow$
LLaVA-1.5-7B	Base [ 23 ]	20.76	66.08	57.99	97.50	6.50	52.00	100.00	0.79	50.39	17.37	16.17	8.38	1507.53	357.86	64.69	69.51	58.20
	+ FigStep [ 13 ]	15.09	5.97	38.71	98.50	5.50	52.00	100.00	0.26	50.13	36.53	16.77	9.58	1420.30	292.50	62.88	68.27	56.36
	+ ECSO [ 14 ]	23.41	16.08	41.57	98.00	5.33	51.67	100.00	0.25	50.13	16.77	14.97	7.19	1497.53	360.00	64.51	69.51	58.15
	+ AdaShield [ 50 ]	1.05	0.22	1.30	33.33	76.67	55.00	34.47	74.21	54.24	29.34	0.60	0.00	1398.34	314.64	59.87	67.03	56.15
	+ EchoSafe ( Ours )	0.37	0.46	1.10	62.33	59.17	60.75	64.47	64.47	64.47	32.93	13.41	8.48	1475.91	294.29	64.34	69.31	57.92

Table 4: Performance comparison on other safety benchmarks across three representative MLLMs . For MM-SafetyBench [ 26 ] , we report the average Attack Success Rate (ASR) across safety categories. For all other benchmarks, we report task-specific performance scores. All safety evaluations are conducted using gpt-5-mini as the judge. Best results are bolded , and second-best results are underlined .

Figure 3: Results on MM-SafetyBench++ using Qwen-2.5-VL with and without memory accumulation . Bar plots represent the contextual correctness rate, while circular markers indicate quality scores. $\Delta$ annotations above the bars highlight the relative gains achieved through memory accumulation across categories.

Figure 4: Efficiency comparison using Qwen-2.5-VL [ 2 ] . We present the average inference time, FLOPs (represented by bubble size), and average contextual correctness rate.

Figure A1 : Illustrative samples drawn from our MM-SafetyBench++. For each scenario, we show a paired unsafe and safe sample that differ only in the user intent while preserving similar visual contexts. The unsafe subset contains harmful requests ( e.g. , police impersonation, hate-speech content generation, DDoS development, invasion planning, client deception, or initiating sexually explicit conversations), whereas the safe subset provides benign alternatives aligned with the same contextual themes ( e.g. , identity verification, respectful communication, defensive cybersecurity training, defensive preparation, ethical client engagement, or healthy online discussions).

Proprietary Models
Method	Illegal Activity			Hate Speech			Malware Generation			Physical Harm			Fraud			Sex
	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM
	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS
GPT-5	100.0 / 5.0	99.0 / 4.9	99.5 / 5.0	97.6 / 4.9	100.0 / 4.9	99.0 / 4.9	97.7 / 4.9	100.0 / 4.9	98.9 / 4.9	97.6 / 4.9	100.0 / 4.9	99.0 / 4.9	100.0 / 4.9	99.1 / 4.9	99.4 / 4.9	73.4 / 3.6	100.0 / 4.9	84.6 / 4.2
GPT-4o-Mini	97.9 / 1.1	90.7 / 3.7	94.1 / 1.7	82.2 / 1.2	96.3 / 4.1	88.7 / 1.9	81.8 / 0.9	97.7 / 3.8	89.0 / 1.5	76.4 / 0.8	91.0 / 3.7	83.1 / 1.3	83.1 / 1.0	96.8 / 4.0	89.4 / 1.6	46.8 / 0.9	89.9 / 3.4	61.6 / 1.4
Open-Source Models

Table A1 : Evaluation of state-of-the-art MLLMs on MM-SafetyBench++ under the GenOCR mode . We report Refusal Rate / Quality Score (RR / QS) for unsafe inputs, Answer Rate / Quality Score (AR / QS) for safe inputs, and their harmonic mean (HM). Higher (

\uparrow

) values indicate better performance. All evaluations use gpt-5-mini as the judge.

Figure A2 : Agreement analysis of GPT-5-Mini as an automatic judge. Row-normalized confusion matrices comparing GPT-5-Mini with Gemini-2.5-Flash (left) and with human evaluations (right). The strong diagonal patterns indicate high agreement between judges. Most discrepancies occur between neighboring score levels, suggesting that disagreements are generally minor and correspond to borderline cases rather than systematic rating shifts. The corresponding Spearman correlations are $\rho=0.72$ (GPT-5-Mini vs. Gemini-2.5-Flash) and $\rho=0.74$ (GPT-5-Mini vs. Human), supporting the reliability of GPT-5-Mini as an automatic evaluator.

	Method	Illegal Activity			Hate Speech			Malware Generation			Physical Harm			Fraud			Sex
		Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM	Unsafe	Safe	HM
		RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS
LLaVA-1.5-7B	Base [ 23 ]	5.2 / 0.3	100.0 / 3.1	9.9 / 0.6	17.8 / 0.8	99.4 / 3.4	30.1 / 1.2	4.6 / 0.2	100.0 / 2.8	8.8 / 0.4	4.2 / 0.2	100.0 / 3.1	8.0 / 0.4	4.6 / 0.2	100.0 / 3.1	8.8 / 0.4	10.1 / 0.4	100.0 / 3.1	18.4 / 0.7
	+ FigStep [ 13 ]	75.3 / 2.2	84.5 / 2.7	79.5 / 2.4	77.3 / 2.4	86.5 / 2.8	81.7 / 2.6	68.2 / 1.8	97.7 / 2.7	79.7 / 2.1	50.7 / 1.6	92.4 / 3.0	65.5 / 2.0	56.5 / 1.8	81.8 / 2.6	66.7 / 2.1	33.0 / 0.9	92.7 / 2.8	48.6 / 1.3
	+ ECSO [ 14 ]	13.4 / 0.5	100.0 / 2.6	26.4 / 0.9	28.3 / 1.2	100.0 / 2.9	44.1 / 1.7	6.8 / 0.2	100.0 / 2.3	12.7 / 0.5	10.4 / 0.4	100.0 / 2.5	19.0 / 0.8	13.0 / 0.5	100.0 / 2.5	25.8 / 0.9	15.8 / 0.7	100.0 / 2.6	27.3 / 1.1
	+ AdaShield [ 50 ]	90.7 / 1.1	37.1 / 0.9	52.6 / 1.0	93.3 / 1.1	50.3 / 1.7	65.1 / 1.3	93.2 / 1.0	45.5 / 1.1	60.8 / 1.0	80.6 / 1.0	32.6 / 0.9	46.3 / 1.0	85.7 / 1.0	35.7 / 1.1	50.5 / 1.0	71.6 / 1.0	45.9 / 1.3	55.6 / 1.1

Table B2 : Performance comparison on MM-SafetyBench++ under the GenOCR attack mode . Higher (

\uparrow

) values indicate better performance. All evaluations are performed with gpt-5-mini as the judge. Best results are bolded , and second-best results are underlined .

Ablating the Embedding Model
Method	Illegal Activity			Hate Speech
	Unsafe	Safe	HM	Unsafe	Safe	HM
	RR / QS	AR / QS	CCR / QS	RR / QS	AR / QS	CCR / QS
CLIP-ViT-L/14	100.0 / 4.5	92.8 / 3.5	96.3 / 3.9	98.2 / 4.4	96.9 / 3.8	97.6 / 4.1
CLIP-ViT-B/16	99.0 / 4.3	87.6 / 3.4	92.9 / 3.8	96.8 / 3.8	93.2 / 3.7	95.0 / 3.7
CLIP-ViT-B/32	97.9 / 3.9	87.6 / 3.5	92.5 / 3.7	95.7 / 3.6	91.4 / 3.7	93.5 / 3.7

Table B3 : Ablation studies . Higher (

\uparrow

) values indicate better performance. All evaluations use gpt-5-mini as the judge.

Table B4: Ablation on retrieval strategy and memory storage format. Results are reported using Qwen-2.5-VL under the Gen attack mode. CCR

\uparrow

measures contextual compliance rate and QS

\uparrow

measures the safety quality score.

Method (CCR / QS)	Illegal Activity	Hate Speech	Malware Generation	Physical Harm
LLaVA-1.5-13B	6.0 / 0.2	14.8 / 0.5	4.4 / 0.1	9.3 / 0.4
+ EchoSafe (Ours)	81.3 / 2.8	91.4 / 2.8	80.1 / 2.8	88.6 / 3.2
Qwen-2.5-VL-32B	35.6 / 1.6	44.7 / 1.8	25.4 / 0.4	31.0 / 1.4
+ EchoSafe (Ours)	93.3 / 3.6	96.0 / 3.9	96.2 / 3.8	94.7 / 3.7

Table B5: Evaluation on larger MLLMs. CCR

\uparrow

denotes contextual compliance rate and QS

\uparrow

denotes safety quality score. EchoSafe consistently improves safety performance across multiple safety domains and model scales.

Figure B3 : Qualitative comparison of different defense approaches on a representative contextual-safety scenario (01-Illegal Activity) from MM-SafetyBench++. The left panel shows a harmful scenario in which the user query requests step-by-step guidance for engaging in an illegal activity (i.e., buying stolen personal data). The right panel shows the paired benign scenario, where the user instead asks for steps to report the activity. Quality Scores (0–5) reflect expert evaluation of the safety awareness or helpfulness of each response.

Figure B4 : Another qualitative comparison of different defense approaches on a representative contextual-safety scenario (06-Physical Harm) from MM-SafetyBench++. The left panel shows a harmful scenario where the user requests step-by-step guidance for carrying out espionage-related activities in a foreign country. The right panel shows the paired benign scenario, where the user instead asks how to protect their organization from such activities. Quality Scores (0–5) reflect expert evaluation of the safety awareness or helpfulness of each response.

Panel of Figure 18 — Figure D5 : System prompt for reflection-based safety insight generation.

Panel of Figure 19 — Figure D6 : Prompt for evaluating unsafe-query responses (RQ-5 safety rubric).

Panel of Figure 20 — Figure D7 : Prompt for evaluating safe-query responses (RQ-5 helpfulness rubric).

Multimodal Large Language Models

Multimodal large language models combine language models with image, video, audio, and other inputs for grounded understanding and generation.

See all 1,069 papers

Aug 8, 2026

MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs

While Multimodal Large Language Models (MLLMs) show remarkable advancements, their cross-modal capabilities introduce complex vulnerabilities that easily bypass unimodal filters. Existing benchmarks lack fine-grained intent-related annotations and rely on unidimensional metrics, hindering comprehensive robustness evaluation. To address this, we propose MME-Safety, a rigorously verified benchmark featuring a unique four-dimensional annotation schema that categorizes risk scenarios, harm severity, and modality-specific stealth levels. Furthermore, we introduce a hierarchical evaluation framework to assess fundamental response reliability, actual risk exposure, and the structural integrity of defensive behaviors. Extensive zero-shot evaluations across 17 state-of-the-art MLLMs provide a comprehensive safety profile of current multimodal systems. Our analysis systematically investigates cross-modal input configurations and uncovers safety implications associated with Chain-of-Thought (CoT) reasoning. These multifaceted findings underscore the urgent need for robust, reasoning-aware safety alignment in the multimodal landscape.

Yilian Shi, Yueming Lyu, Haoxiang Tan +5

Nanjing University

LLM Safety Benchmarks Multimodal Robustness Multimodal Large Language Models LLM Safety Evaluation

Ask PDF

Oct 15, 2026

When Reasoning Is Not Enough: Proactive Inquiry for Cross-Modal Safety in Multimodal Large Language Models

Multimodal large language models (MLLMs) can act unsafely when individually benign images and texts are interpreted jointly. Beyond implicit reasoning risk, where reasoning drifts toward harm, we identify intent-level risk, where cross-modal ambiguity silently commits the model to a hazardous premise that path-level optimization alone cannot correct. We propose SAFE-PRO, which uses a Risk-Conditioned Uncertainty (RCU) signal, combining predictive entropy with a learned cross-modal risk score, to decide when to ask safety-grounded clarifying questions and where to apply path optimization, under a reward that prioritizes safety and penalizes over-refusal. We also build SSUI-Dialog, a multimodal multi-turn dataset with dual-intent supervision, and theoretically analyze gradient variance and unsafe-completion risk under RCU. Across seven benchmarks and two base MLLMs, SAFE-PRO consistently improves safety over SRPO, roughly halves its over-refusal, and surpasses GPT-4o and Claude-3.5-Sonnet in average single-turn safety.

Wei Cai, Jian Zhao, Yuchen Yuan +1

Institute of Artificial Intelligence (TeleAI), China Telecom, P. R. China. · Peking University. · School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, P. R. China.

Multimodal Multi-Turn Safety Alignment VLM Safety LLM Safety MLLM Safety Multimodal Large Language Models Clarification Question Generation+1

Ask PDF

Apr 17, 2026

MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models

Existing multimodal safety benchmarks focus solely on visual inputs and cannot assess Omni Large Language Models (LLMs) that process vision, audio, and text. We introduce MCBench, a benchmark with 1196 scenarios spanning four safety categories that require integrating multiple modalities for accurate safety assessment. Each unsafe scenario is paired with a minimally different safe counterpart to assess model sensitivity. Our evaluations of state-of-the-art models reveal significant challenges. Omni LLMs struggle with subtle or non-physical risks but perform better when salient visual or acoustic cues are present. Analysis of reasoning traces shows that, although models can extract modality-specific information, they often fail to integrate these cues effectively for safety judgments. Our findings reveal that current Omni LLMs lack robust cross-modal reasoning in safety-critical settings, underscoring the need for improved architectures and training strategies for multimodal safety.

Manh Luong, Tamas Abraham, Junae Kim +6

Monash University, Australia · Defence Science and Technology Group, Australia

LLM Safety Benchmarks Language Model Safety Evaluation Multimodal Large Language Models Multimodal Model Evaluation

Ask PDF

Abstract

Multimodal Large Language Models

Explore similar work

MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs

When Reasoning Is Not Enough: Proactive Inquiry for Cross-Modal Safety in Multimodal Large Language Models

MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models