Organizations: College of Computer Science, Chongqing University, Chongqing, China · School of Economics and Business Administration, Chongqing University, Chongqing, China
Large Language Models (LLMs) remain vulnerable to jailbreak attacks and harmful-content prompts, particularly under adversarial distribution shifts where static alignment pipelines fail to generalize. Although introspective methods such as STAIR strengthen refusal consistency, they depend on fixed self-evaluation policies that cannot rapidly adapt to evolving manipulation strategies. To overcome these limitations, this paper introduces META-IR, a meta-introspective reasoning framework that enhances safety alignment through meta-learning-driven adaptation. It employs a dual-loop architecture: an inner introspective loop that performs structured risk analysis, self-assessment, and reflective correction via SI-MCTS, and an outer meta-learning loop that optimizes the introspection policy across diverse jailbreak task families to enable rapid generalization to unseen attacks. A dynamic jailbreak simulation module further exposes the model to progressively harder adversarial prompts, enabling continual self-improvement without human supervision. Experiments across multiple models (Llama-3.1-8B-Instruct and Qwen-2-7B-Instruct) and a broad suite of safety, helpfulness, generalization, and introspection metrics demonstrate that META-IR substantially improves adaptation speed, robustness, and reasoning stability compared with all baseline models. These results establish META-IR as an effective pathway toward building LLMs capable of sustained, adaptive safety alignment under dynamic and evolving threat environments.
Figures & tables
Fig. 1: Under adaptive jailbreaks, a standard aligned model is manipulated into unsafe output, whereas Meta-IR uses dual-loop introspective reasoning to detect hidden intent, correct intermediate reasoning, and produce a safe final response.
Fig. 2: Meta-IR has three stages: (1) structured introspective reasoning for prompt analysis, hidden-intent detection, risk evaluation, and safe trajectory generation; (2) dual-loop self-improvement, where SI-MCTS generates trajectories, builds safety-prioritized preference data, and meta-learns the introspection policy across jailbreak families; and (3) meta-adaptive decoding, where the learned reward model guides best-of-N and beam search to select a safe, helpful final answer.
Figure 3
LLAMA-3.1-8B-Instruct
Variant
REAR↑
SVR↓
MCR↓
JIR↓
MTA↑
META-IR-SFT
0.718
0.133
0.229
0.147
0.588
META-IR-1
0.793
0.139
0.211
0.130
0.634
META-IR-2
0.836
0.103
0.184
0.117
0.651
META-IR-3
0.902
0.061
0.162
0.094
0.702
Qwen-2-7B-Instruct
TABLE II: Ablation results: removing any Meta-IR component lowers REAR/MTA and increases SVR, MCR, and JIR.
Method
SelfGen
Meta
Search
Upd.
Adapt
Fail
IR
SFT [ 15 ]
×
×
×
Static
No
Yes
None
RLHF [ 12 ]
×
×
×
Static
Static
Yes
Low
DPO [ 13 ]
×
×
×
Static
Static
Yes
Low
SACPO [ 16 ]
✓
×
✓
Self
Static
Yes
Med
Self-Reward [ 20 ]
✓
×
×
Self
SelfG
Yes
Low
CoT [ 18 ]
×
×
×
Static
Static
Yes
Med
TABLE III: Alignment methods under adaptive jailbreaks: static, self-generated, and meta-adaptive safety strategies.
Red teaming is critical for uncovering vulnerabilities in Large Language Models (LLMs). While automated methods have improved scalability, existing approaches often rely on static heuristics or stochastic search, rendering them brittle against advanced safety alignment. To address this, we introduce Metis, a framework that reformulates jailbreaking as inference-time policy optimization within an adversarial Partially Observable Markov Decision Process (POMDP). Metis employs a self-evolving metacognitive loop to perform causal diagnosis of a target's defense logic and leverages structured feedback as a semantic gradient to refine its policy, offering enhanced interpretability through transparent reasoning traces. Extensive evaluations across 10 diverse models demonstrate that Metis achieves the strongest average Attack Success Rate (ASR) among compared methods at 89.2%, maintaining high efficacy on resilient frontier models (e.g., 76.0% on O1 and 78.0% on GPT-5-chat) where traditional baselines exhibit substantial performance degradation. By replacing redundant exploration with directed optimization, Metis reduces token costs by an average of 8.2x and up to 11.4x. Our analysis reveals that current defenses remain vulnerable to internally-steered, closed-loop reasoning trajectories under the tested settings, highlighting a critical need for next-generation defenses capable of reasoning about safety dynamically during inference.
Huilin Zhou, Jian Zhao, Yilu Zhong +7
University of Science and Technology of China · Institute of Artificial Intelligence (TeleAI), China Telecom
Large Language Models (LLMs) have achieved remarkable capabilities but remain vulnerable to jailbreak attacks that elicit harmful or unsafe outputs. Existing safety alignment approaches, including Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), often require substantial attack-specific supervision and computational resources, while remaining susceptible to shallow safety alignment and over-refusal. To address these challenges, we introduce SSRFT(Supervised Safe-Role Fine-Tuning), the first framework that reformulates safety alignment as the internalization of a predefined safe role. SSRFT constructs a Safe-Role Question-Answer (SRQA) dataset from psychometric questions, limited jailbreak prompts, and a safe-role description. Role-consistent responses are synthesized, validated, and expanded into diverse scenarios, enabling models to internalize safety-oriented values and principles rather than explicit refusal patterns. Experiments across multiple Base and Instruct models show that SSRFT achieves more robust and generalizable safety alignment than standard SFT. SSRFT shows substantially greater robustness to prefilling attacks and better generalization to unseen jailbreak domains, while reducing over-refusal on benign queries and preserving the model's general capabilities. These results establish safe-role internalization as an effective alternative to refusal-centric safety alignment. Warning: This paper contains examples of harmful and toxic language.
Jinghao Pang, Jitai Hao, Qiang Huang +2
Harbin Institute of Technology (Shenzhen) · Leiden University
Ensuring the safety of Large Language Models (LLMs) is critical for real-world deployment. However, current safety measures often fail to address implicit, domain-specific risks. To investigate this gap, we introduce a dataset of 3,000 annotated queries spanning education, finance, and management. Evaluations across 14 leading LLMs reveal a concerning vulnerability: an average jailbreak success rate of 57.8%. In response, we propose MENTOR, a metacognition-driven self-evolution framework. MENTOR performs metacognitive self-assessment, using strategies such as perspective-taking and consequential reasoning to uncover latent model misalignments. MENTOR couples single-pass rule-guided inference for routine requests with a selectively invoked metacognitive evolution cycle that revises residual unsafe responses, distills successful corrections into a dynamic rule graph, and compiles validated rules into activation-level steering signals for future inference. Experiments demonstrate that MENTOR substantially reduces attack success rates across all tested domains and outperforms existing safety alignment methods. The code and dataset for MENTOR are available at: https://anonymous.4open.science/r/MENTOR-Evo.
Liang Shan, Kaicheng Shen, Wen Wu +8
School of Computer Science and Technology, East China Normal University · Shanghai AI Lab, Shanghai Innovation Institute