META-IR: Meta-Introspective Reasoning for Adaptive Safety Alignment in LLM
Organizations: College of Computer Science, Chongqing University, Chongqing, China · School of Economics and Business Administration, Chongqing University, Chongqing, China
Abstract
Large Language Models (LLMs) remain vulnerable to jailbreak attacks and harmful-content prompts, particularly under adversarial distribution shifts where static alignment pipelines fail to generalize. Although introspective methods such as STAIR strengthen refusal consistency, they depend on fixed self-evaluation policies that cannot rapidly adapt to evolving manipulation strategies. To overcome these limitations, this paper introduces META-IR, a meta-introspective reasoning framework that enhances safety alignment through meta-learning-driven adaptation. It employs a dual-loop architecture: an inner introspective loop that performs structured risk analysis, self-assessment, and reflective correction via SI-MCTS, and an outer meta-learning loop that optimizes the introspection policy across diverse jailbreak task families to enable rapid generalization to unseen attacks. A dynamic jailbreak simulation module further exposes the model to progressively harder adversarial prompts, enabling continual self-improvement without human supervision. Experiments across multiple models (Llama-3.1-8B-Instruct and Qwen-2-7B-Instruct) and a broad suite of safety, helpfulness, generalization, and introspection metrics demonstrate that META-IR substantially improves adaptation speed, robustness, and reasoning stability compared with all baseline models. These results establish META-IR as an effective pathway toward building LLMs capable of sustained, adaptive safety alignment under dynamic and evolving threat environments.
Figures & tables
| LLAMA-3.1-8B-Instruct | |||||
|---|---|---|---|---|---|
| Variant | REAR↑ | SVR↓ | MCR↓ | JIR↓ | MTA↑ |
| META-IR-SFT | 0.718 | 0.133 | 0.229 | 0.147 | 0.588 |
| META-IR-1 | 0.793 | 0.139 | 0.211 | 0.130 | 0.634 |
| META-IR-2 | 0.836 | 0.103 | 0.184 | 0.117 | 0.651 |
| META-IR-3 | 0.902 | 0.061 | 0.162 | 0.094 | 0.702 |
| Qwen-2-7B-Instruct | |||||
| Method | SelfGen | Meta | Search | Upd. | Adapt | Fail | IR |
|---|---|---|---|---|---|---|---|
| SFT [ 15 ] | × | × | × | Static | No | Yes | None |
| RLHF [ 12 ] | × | × | × | Static | Static | Yes | Low |
| DPO [ 13 ] | × | × | × | Static | Static | Yes | Low |
| SACPO [ 16 ] | ✓ | × | ✓ | Self | Static | Yes | Med |
| Self-Reward [ 20 ] | ✓ | × | × | Self | SelfG | Yes | Low |
| CoT [ 18 ] | × | × | × | Static | Static | Yes | Med |
LLM Safety Alignment
Large language model safety alignment trains and evaluates systems to follow human policies while resisting harmful requests and misuse.
See all 364 papers