When Reasoning Is Not Enough: Proactive Inquiry for Cross-Modal Safety in Multimodal Large Language Models
Authors: Wei Cai, Jian Zhao, Yuchen Yuan, Xuelong Li
Organizations: Institute of Artificial Intelligence (TeleAI), China Telecom, P. R. China. · Peking University. · School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, P. R. China.
Multimodal large language models (MLLMs) can act unsafely when individually benign images and texts are interpreted jointly. Beyond implicit reasoning risk, where reasoning drifts toward harm, we identify intent-level risk, where cross-modal ambiguity silently commits the model to a hazardous premise that path-level optimization alone cannot correct. We propose SAFE-PRO, which uses a Risk-Conditioned Uncertainty (RCU) signal, combining predictive entropy with a learned cross-modal risk score, to decide when to ask safety-grounded clarifying questions and where to apply path optimization, under a reward that prioritizes safety and penalizes over-refusal. We also build SSUI-Dialog, a multimodal multi-turn dataset with dual-intent supervision, and theoretically analyze gradient variance and unsafe-completion risk under RCU. Across seven benchmarks and two base MLLMs, SAFE-PRO consistently improves safety over SRPO, roughly halves its over-refusal, and surpasses GPT-4o and Claude-3.5-Sonnet in average single-turn safety.
Figures & tables
Fig. 1: Two complementary failure modes in cross-modal safety. (a) Implicit reasoning risk : even when the request is fully specified, the model’s long reasoning chain may branch into unsafe trajectories. (b) Intent-level ambiguity : when the cross-modal input under-specifies the user’s goal, the model is forced to assume a premise, and a wrong assumption can deterministically commit reasoning to an unsafe path. SAFE-PRO addresses both via a two-layer defense unified by a risk-conditioned uncertainty signal.
Fig. 2: The construction pipeline of SSUI-Dialog. Stage 1 injects safety-critical premise ambiguity into SSUI queries; Stage 2 generates the corresponding clarification questions; Stage 3 simulates dual user replies (safe-intent and unsafe-intent); Stage 4 composes full dialogue trajectories; Stage 5 performs human verification; and Stage 6 unifies the verified trajectories into the released format. The stages reuse the multi-agent infrastructure of SSUI [ 15 ] with new ambiguity-injection and dual-simulation modules.
Fig. 3: Three-tier safety taxonomy (3 primary, 19 secondary, and 68 tertiary categories) that SSUI-Dialog inherits from SSUI; reproduced from [ 15 ] .
Fig. 4: Overview of the SAFE-PRO framework. A cross-modal input (I,T) enters the MLLM, which generates reasoning steps autoregressively. At each step sj , the Risk-Conditioned Uncertainty estimator u(sj) combines normalized predictive entropy with a learned cross-modal risk score. When u(sj)≥τhigh , the outer loop interrupts reasoning with a clarification question, and the user reply is re-injected into the trace. During training, steps with τlow≤u(sj)<τhigh receive risk-guided contrastive path optimization (inner loop). A composite reward with a dominant safety term jointly trains both loops.
USBBench
MLLMGuard
MSSBench
SafeBench
VLSBench
Average
Method
ASR ↓
ARR ↓
Avg ↓
PAR ↑
ASD ↓
Avg ↑
ASR ↓
SRI ↑
Avg ↑
Avg ↑
Avg ↓
Commercial MLLMs (zero-shot)
Claude-3.5-Sonnet
32.80
25.79
29.30
52.38
9.01
69.01
0.70
99.30
79.35
75.01
13.00
Gemini-1.5-Pro
64.45
11.33
37.89
38.12
21.94
61.94
2.60
97.10
50.21
61.84
20.80
GPT-4o
72.83
3.77
38.30
56.68
14.32
59.30
3.40
96.10
69.50
70.40
18.67
Base: LLaVA-NeXT-LLaMA3-8B
TABLE I: Cross-modal safety on five single-turn benchmarks, following the protocol of SRPO [ 15 ] . The commercial-MLLM, base-model, and + SRPO rows are taken from Table 1 of [ 15 ] ; the Avg ↑ of LLaVA-NeXT-LLaMA3-8B and the Avg ↓ of LLaVA-NeXT-LLaMA3-8B + SRPO are recomputed from the per-benchmark values (published as 41.12 and 14.37). Bold : best in each column.
Method
ASR ↓
Helpful ↑
CSHC ↑
TTR ↓
GPT-4o
28.72
0.71
0.46
2.85
Claude-3.5-Sonnet
24.18
0.74
0.51
2.73
SRPO
41.20
0.42
0.31
1.00
CollabLLM (mm.)
32.81
0.61
0.41
3.18
SAFE-PRO
15.83
0.78
0.68
1.62
TABLE II: Multi-turn evaluation on the SSUI-Dialog test set (base model: Qwen2.5-VL-7B). CSHC and TTR are defined in Section VI-A ; a non-interactive model has TTR =1 by construction, so TTR is not bolded.
Fig. 5: Qualitative case of intent-level cross-modal risk. The input pairs bleach (“84”) and an acid-based toilet cleaner (Harpic) with an under-specified instruction. (a) The baseline silently assumes a “mixing” intent and outputs instructions that would release toxic gas. (b) SAFE-PRO detects the safety-critical missing premise via a high RCU score ( u≥τhigh ) and proactively clarifies the user’s intent; conditioned on the reply, it proceeds to a safe and helpful answer.
Method
ORR ↓
MSSBench ↑
GPT-4o
21.3
59.30
Claude-3.5-Sonnet
34.8
69.01
Gemini-1.5-Pro
27.5
61.94
Qwen2.5-VL-7B
24.6
55.36
+ CollabLLM (mm.)
22.9
66.10
+ SRPO
38.4
73.89
TABLE III: Over-refusal on MOSSBench (ORR ↓ ) alongside MSSBench average accuracy (as in Table I ; the w/o Rover variant is evaluated only here). Trained methods use Qwen2.5-VL-7B. Bold : best in each column.
Variant
ASR ↓
CSHC ↑
TTR ↓
ORR ↓
SAFE-PRO (full)
15.83
0.68
1.62
19.70
RCU components
R only ( α=0 )
21.45
0.59
1.48
24.32
PE only ( β=0 )
30.82
0.42
3.05
31.85
Loop ablation
Outer only (no inner)
24.71
0.53
2.15
28.42
TABLE IV: Ablation on SAFE-PRO components (Qwen2.5-VL-7B). ASR, CSHC, and TTR are measured on SSUI-Dialog; ORR is measured on MOSSBench. Bold marks the best value in each of the ASR, CSHC, and ORR columns.
While Multimodal Large Language Models (MLLMs) are increasingly deployed in safety-critical domains, their reliability is threatened by multimodal implicit risks. Unlike explicit threats, these hazards emerge when individually benign text and neutral visual entities logically converge to induce unsafe outputs. Current detection methods fail to address this because they overlook the underlying risk activation mechanisms that govern cross-modal risk activation, leading to single-modality shortcut learning and hallucinated rationalizations. To bridge this gap, we first construct TriggerBench, the first dataset explicitly modeling risk compositionality (5,600 instances). By formally isolating Key Elements and Trigger Elements to build counterfactual contrastive pairs, TriggerBench eliminates risk residues and forces models to perform genuine logical deduction rather than superficial pattern matching, which provides a rigorous foundation for both large-scale training and fine-grained evaluation. Building on this, we propose a Step-Supervised Structured Reasoning training framework and employ it to train ThinkingGuard, a specialized guard model. Inspired by Situation Awareness theory, we decouple implicit risk identification into progressive cognitive stages, and utilize a step-reward Monte Carlo Tree Search algorithm to explore optimal reasoning trajectories, which are then distilled into the model through Dual-Constraint Preference Alignment. Extensive experiments across both standard and implicit safety benchmarks demonstrate that ThinkingGuard achieves strong performance. Project resources are available at https://github.com/FroggyChen/ThinkingGuard.
Ruochen Zhang, Yao Huang, Yitong Sun +5
Beihang University Beijing, China · Tsinghua University Beijing, China · Institute of Artificial Intelligence Beihang University Beijing, China State Key Laboratory of AI Safety
While Multimodal Large Language Models (MLLMs) show remarkable advancements, their cross-modal capabilities introduce complex vulnerabilities that easily bypass unimodal filters. Existing benchmarks lack fine-grained intent-related annotations and rely on unidimensional metrics, hindering comprehensive robustness evaluation. To address this, we propose MME-Safety, a rigorously verified benchmark featuring a unique four-dimensional annotation schema that categorizes risk scenarios, harm severity, and modality-specific stealth levels. Furthermore, we introduce a hierarchical evaluation framework to assess fundamental response reliability, actual risk exposure, and the structural integrity of defensive behaviors. Extensive zero-shot evaluations across 17 state-of-the-art MLLMs provide a comprehensive safety profile of current multimodal systems. Our analysis systematically investigates cross-modal input configurations and uncovers safety implications associated with Chain-of-Thought (CoT) reasoning. These multifaceted findings underscore the urgent need for robust, reasoning-aware safety alignment in the multimodal landscape.
While multimodal large language models (MLLMs) extend model capabilities beyond text, they also make safety alignment increasingly challenging. Multimodal safety alignment methods must address cross-modal jailbreaks, safety-awareness failures, and over-sensitive refusals. However, existing methods often rely on retraining or internal-state inspection, limiting their applicability to deployed closed-source MLLMs and motivating test-time safety alignment. We analyze this setting and identify two key obstacles, utility dominance and reasoning inertia, which cause models to overlook latent risks or follow malicious reasoning trajectories. Guided by these insights, we propose ReFrame, a training-free multimodal input reframing framework where two agents share a lightweight locally deployed MLLM: the evidence-generation agent constructs complementary risk and utility evidence, and the rewrite-and-routing agent converts it into a safe proxy prompt and image-routing decision before calling the downstream MLLM, without modifying it or accessing its internal information. Experiments across multiple MLLMs and benchmarks show that ReFrame improves jailbreak defense, safety awareness, and oversensitivity reduction while preserving multimodal utility.
Wenzheng Jiang, Xuankun Rong, Yuanzhao Zhai +2
College of Computer Science and Technology, National University of Defense Technology · State Key Laboratory of Complex & Critical Software Environment · School of Computer Science, Wuhan University