Large language models (LLMs) remain vulnerable to jailbreak prompts that are fluent and semantically coherent, and therefore difficult to detect with standard heuristics. A particularly challenging failure mode occurs when an attacker tries to hide the malicious goal of their request by manipulating its framing to induce compliance. Because these attacks maintain malicious intent through a flexible presentation, defenses that rely on structural artifacts or goal-specific signatures can fail. Motivated by this, we introduce a self-supervised framework for disentangling semantic factor pairs in LLM activations at inference. We instantiate the framework for goal and framing and construct GoalFrameBench, a corpus of prompts with controlled goal and framing variations, which we use to train Representation Disentanglement on Activations (ReDAct) module to extract disentangled representations in a frozen LLM. We then propose FrameShield, an anomaly detector operating on the framing representations, which improves model-agnostic detection across multiple LLM families with minimal computational overhead. Theoretical guarantees for ReDAct and extensive empirical validations show that its disentanglement effectively powers FrameShield. Finally, we use disentanglement as an interpretability probe, revealing distinct profiles for goal and framing signals and positioning semantic disentanglement as a building block for both LLM safety and mechanistic interpretability.
Figures & tables
Figure 1: A visual summary of our pipeline. ReDAct learns disentangled goal and framing representations from frozen LLM activations using the contrastive pairs in the GoalFrameBench corpus we construct. FrameShield can then detect jailbreak prompts via anomaly detection on the disentangled framing representation.
Figure 2: Visualization of how varying the framing of a prompt leads to a jailbreak.
Figure 3: Examples of contrastive prompts for goal and framing pairs in GoalFrameBench. The prompts are abbreviated here due to space limitations; the full examples are included in Appendix D .
Figure 4: ReDAct Architecture.
Llama-3-8B
Vicuna-7B
Qwen3-4B
Mistral-7B
vg
vf
vg
vf
vg
vf
vg
vf
Goal
0.37
0.18
0.37
0.19
0.30
0.20
0.37
0.19
Framing
0.18
0.22
0.17
0.26
0.18
0.29
0.12
0.23
Table 1: ANOVA effect size analysis for association of each of Goal and Framing with vg and vf in models from various families of LLMs. Each cell shows the η2 between the corresponding row and column. Results for other LLMs are reported in Appendix F .
Method
Llama3-8B
Llama2-7B
Vicuna-7B
Vicuna-13B
Mistral-7B
Qwen2-0.5B
Qwen2.5-7B
Qwen3-4B
(Acc / F1)
(Acc / F1)
(Acc / F1)
(Acc / F1)
(Acc / F1)
(Acc / F1)
(Acc / F1)
(Acc / F1)
JBShield
0.75 / 0.75
0.73 / 0.76
0.60 / 0.63
0.63 / 0.67
0.71 / 0.75
0.59 / 0.62
0.66 / 0.68
0.70 / 0.74
FrameShield-Last (Ours)
0.82 / 0.79
0.58 / 0.70
0.67 / 0.75
0.76 / 0.79
0.79 / 0.74
0.66 / 0.74
0.79 / 0.76
0.76 / 0.75
FrameShield-Crit (Ours)
0.76 / 0.69
0.80 / 0.79
0.81 / 0.79
0.81 / 0.82
0.76 / 0.69
0.74 / 0.65
0.94 / 0.93
0.89 / 0.87
Table 2: Performance comparison of FrameShield variants against JBShield on GoalFrameBench. Results show detection accuracy and F1 scores for several models from various families of LLM, with best and second-best accuracies bolded and underlined.
Method
Llama3-8B
Vicuna-7B
Qwen3-4B
Mistral-7B
(Acc / F1)
(Acc / F1)
(Acc / F1)
(Acc / F1)
LlamaGuard
0.60 / 0.75
0.75 / 0.85
–
0.74 / 0.85
SelfEx
0.16 / 0.26
–
–
0.46 / 0.63
GradSafe
0.37 / 0.54
0.03 / 0.06
–
0.05 / 0.10
JBShield
0.77 / 0.86
0.91 / 0.91
0.87 / 0.88
0.88 / 0.88
FrameShield-Last (Ours)
0.96 / 0.86
0.97 / 0.86
0.84 / 0.68
0.92 / 0.64
Table 3: Comparing FrameShield against four methods, including JBShield, on GoalFrameBench’s seed prompts from Chao et al. (2025) . Further LLM results are in Tables 5 and 6 . Best/second-best accuracy scores are bolded/underlined.
Figure 5: Strength of association between goal and framing and their corresponding representations learned by ReDAct across the second half of Llama2-7B layers (other LLMs in Figure 7 ). Blue and red bars show η2(G,vg) and η2(F,vf) , respectively.
Figure 6: The loss minimization of ReDAct’s training on different layers of Llama2-7B (additional examples reported in Figure 8 ).
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Llama2-7B
Vicuna-7B
Llama3-8B
Vicuna-13B
Mistral-7B
Qwen2-0.5B
Qwen2.5-7B
Qwen3-4B
vg
vf
vg
vf
vg
vf
vg
vf
vg
vf
vg
vf
vg
vf
vg
vf
Goal
0.41
0.19
0.37
0.19
0.37
0.18
0.39
0.19
0.26
0.26
0.44
0.20
0.31
0.19
0.30
0.20
Framing
0.14
0.22
0.17
0.26
0.18
0.22
0.14
0.23
0.23
0.25
0.14
0.22
0.24
0.27
0.18
0.29
Appendix
Table 4: ANOVA effect size analysis for association of each of Goal and Framing with vg and vf . Each cell shows the η2 between the corresponding row and column.
Method
Llama3-8B
Llama2-7B
Vicuna-7B
Vicuna-13B
(Acc / F1)
(Acc / F1)
(Acc / F1)
(Acc / F1)
LlamaGuard
0.60 / 0.75
0.53 / 0.69
0.75 / 0.85
0.76 / 0.86
SelfEx
0.16 / 0.26
0.27 / 0.30
–
–
GradSafe
0.37 / 0.54
0.62 / 0.77
0.03 / 0.06
–
JBShield
0.77 / 0.86
0.84 / 0.86
0.91 / 0.91
0.79 / 0.78
FrameShield-Last (Ours)
0.96 / 0.86
0.95 / 0.85
0.97 / 0.86
0.89 / 0.62
Appendix
Table 5: Comparing the performance of FrameShield against four other methods, including its closest counterpart, JBShield, using 4 LLMs from the Llama and Vicuna families. The performance is measured on GoalFrameBench’s harmful seed prompts from Chao et al. (2025) . Performance is measured by jailbreak detection accuracy and F1 score, and the best/second-best accuracy on each dataset and LLM is bold/underlined.
Method
Qwen2-0.5B
Qwen2.5-7B
Qwen3-4B
(Acc / F1)
(Acc / F1)
(Acc / F1)
JBShield
0.83 / 0.85
0.83 / 0.85
0.87 / 0.88
FrameShield-Last (ours)
0.96 / 0.92
0.81 / 0.73
0.84 / 0.68
FrameShield-Crit (ours)
0.97 / 0.94
0.83 / 0.64
0.85 / 0.71
Appendix
Table 6: As a supplementary comparison, we also compare JBShield and the two variations of FrameShield using 3 models from the Qwen family, on the GPF jailbreak prompts from Chao et al. (2025) , which is the source of harmful seed prompts for GoalFrameBench. The performance is measured by jailbreak detection (binary prediction) accuracy and F1 score. The best/second-best accuracy on each dataset is bold/underlined.
Method
Llama3-8B
Llama2-7B
Vicuna-7B
Vicuna-13B
Mistral 7B
Qwen2-0.5B
Qwen2.5-7B
Qwen3-4B
FrameShield-Last
ID
0.82 / 0.79
0.58 / 0.70
0.67 / 0.75
0.76 / 0.79
0.79 / 0.74
0.66 / 0.74
0.79 / 0.76
0.76 / 0.75
OOD
0.72 / 0.75
0.69 / 0.81
0.72 / 0.82
0.79 / 0.85
0.67 / 0.70
0.74 / 0.83
0.72 / 0.74
0.73 / 0.76
FrameShield-Crit
ID
0.76 / 0.69
0.80 / 0.79
0.81 / 0.79
0.81 / 0.82
0.76 / 0.69
0.74 / 0.65
0.94 / 0.93
0.89 / 0.87
OOD
0.83 / 0.69
0.72 / 0.66
0.73 / 0.64
0.59 / 0.60
0.83 / 0.69
0.80 / 0.63
0.65 / 0.64
0.75 / 0.72
Appendix
Table 7: Comparing in-distribution (ID) and out-of-distribution (OOD) performance of FrameShield for different LLMs. Small differences indicate strong out-of-distribution generalization. Each cell shows ”accuracy/F1 score” for jailbreak detection.
Figure 7: Strength of association between goal and framing and their corresponding representations learned by ReDAct across the second half of several LLMs. Blue and red bars show η2(G,vg) and η2(F,vf) , respectively.
Figure 8: Loss minimization during the training of ReDAct to converge at early, middle, and later layers, for different LLMs.
Jailbreak prompts are a practical and evolving threat to large language models (LLMs), particularly in agentic systems that execute tools over untrusted content. Many attacks exploit long-context hiding, semantic camouflage, and lightweight obfuscations that can evade single-pass guardrails. We present RLM-JB, an end-to-end jailbreak detection framework built on Recursive Language Models (RLMs), in which a root model orchestrates a bounded analysis program that transforms the input, queries worker models over covered segments, and aggregates evidence into an auditable decision. RLM-JB treats detection as a procedure rather than a one-shot classification: it normalizes and de-obfuscates suspicious inputs, chunks text to reduce context dilution and guarantee coverage, performs parallel chunk screening, and composes cross-chunk signals to recover split-payload attacks. On AutoDAN-style adversarial inputs, RLM-JB achieves high detection effectiveness across three LLM backends (ASR/Recall 92.5-98.0%) while maintaining very high precision (98.99-100%) and low false positive rates (0.0-2.0%), highlighting a practical sensitivity-specificity trade-off as the screening backend changes.
Multi-turn jailbreak attacks progressively erode LLM safety alignment across seemingly innocuous conversation turns, achieving success rates exceeding 90% against state-of-the-art models. Existing alignment-based and guardrail methods suffer from three key limitations: they require costly weight modification, evaluate each turn independently without modeling cumulative safety erosion, and detect attacks only after harmful content has been generated. To address these limitations, we first formulate the proactive early jailbreak detection problem with a new metric, detection lead, that measures how early an attack can be detected before the LLM complies. We then propose SAFEDREAM, a lightweight world-model-based framework that operates as an external module without modifying the LLM's weights. SAFEDREAM introduces three components: (1) a safety state world model that encodes LLM hidden states into a compact safety representation and predicts how it evolves across turns, (2) CUSUM detection that accumulates weak per-turn risk signals into reliable evidence, and (3) contrastive imagination that simultaneously rolls out attack and benign futures in latent space to issue early alarms before jailbreaks occur. On three multi-turn jailbreak benchmarks (XGuard-Train, SafeDialBench, SafeMTData) against 8 baselines, SAFEDREAM achieves the best detection timeliness across all benchmarks (1.06-1.20 turns before compliance) while maintaining competitive false positive rates and outperforming baselines in detection quality.
Bo Yan, Weikai Lin, Yada Zhu +1
University of Central Florida · University of Rochester · IBM Research
Large language models (LLMs) remain vulnerable to jailbreak attacks that conceal harmful intent within complex adversarial prompts. Existing defenses primarily rely on input perturbation or harmful-output suppression, but they rarely model where malicious intent resides, resulting in brittle protection and excessive over-refusal. We propose SENTINEL, a plug-and-play, generation-time jailbreak defense that reframes mitigation as an intent extraction problem. Our key insight is that instruction-tuned LLMs exhibit strong input--output semantic consistency: regardless of jailbreak complexity, generated outputs tend to align with the attacker's true intent. SENTINEL exploits this property by matching semantically aligned input--output regions to extract intention-revealing subsequences, scores these subsequences using refusal-direction projections to estimate harmfulness, and halts generation when necessary. Experiments on HarmBench across multiple LLMs show that SENTINEL reduces jailbreak success rates to close to 5% while maintaining low over-refusal. We further demonstrate robustness to adaptive attacks and provide a mechanistic interpretation: SENTINEL re-distributes jailbreak features from alignment blind spots to aligned regions.
Luoyu Chen, Weiqi Wang, Chenhan Zhang +2
University of Technology Sydney, Sydney, NSW, Australia · Xi’an Jiaotong University, Xi’an, Shaanxi, China · Southeast University, Nanjing, Jiangsu, China