We address jailbreaks, backdoors, and unlearning for large language models (LLMs). Unlike prior work, which trains LLMs based on their actions when given malign instructions, our method specifically trains the model to change how it interprets instructions. Our method, Latent Instruction Representation Alignment (LIRA), greatly improves generalization. We further boost generalization through an internally adversarial training algorithm. Our methods block over 99% of PEZ jailbreak attacks; remove a challenging insecure code backdoor; and achieve optimal forgetting on WMDP cyber with negligible loss of benign capabilities.
Figures & tables
Figure 1: (a) Standard safety training (brown) redirects malign requests into safe behavior at some point in the model, but the results can be brittle. Our approach (green) redirects the representation of the instruction to that of a nearby instruction that elicits harmless behavior, with much better generalization. (b) Inspecting instruction representations shows how LIRA works compared to standard methods. After a standard method (gradient difference—GD), the representation of malign instructions barely changes (red to purple) while representations for LIRA (green) are much more similar to representations of benign instructions (blue). (c) Sequence-Aware Gradients (SAG) encourage LIRA to focus on instruction representations by stopping some gradients (red) coming directly from response positions during safety training.
Questions + defender-installed toy backdoor trigger
Ordinary answers
Validation backdoor removed
Unlearning
n.a.
Request requiring undesired knowledge
n.a.
Fixed duration of N batches
Table 1: Hypothetical LIRA configurations applied to example tasks
Figure 2: AdLIRA iterates between (a) an aligning phase applying LIRA and (b) an attack phase that searches for new representations that bypass the defenses built in the aligning phase. (c) AdLIRA’s attack layers transform “toy” backdoor representations so that they are similar to unknown backdoor representations, allowing the aligning phase to remove backdoors without knowing the trigger.
Figure 3: (a) Unlearning : after applying LIRA, instructions to produce knowledge that should be forgotten have representations almost indistinguishable from those that produce normal knowledge, unlike prior work (RMU). (b) Classifier-guided LIRA uses a malignity classifier to train the aligning layers without paired benign/malign instructions.
Figure 4: (a) Jailbreak: PEZ AdLIRA reduces ASR to near 0% when defending against PEZ ( Wen et al., 2023 ) attacks. (b) Jailbreak: embedding space AdLIRA prevents more attacks—even when the attacker has 100% control of the embedding dimension—than baselines do when the attacker has only \nicefrac132 control. (c) Backdoor: HATE Our LIRA almost entirely removes backdoor behavior in a single gradient step while baselines have limited success after 100 gradient steps. (d) Backdoor: Code AdLIRA removes the backdoor, causing backdoored models to write code as secure as the non-backdoor baseline (dashed horizontal line) while other methods make little progress.
Figure 5: (a) Unlearning: WMDP Cyber LIRA sharply degrades multiple choice accuracy and free-response cross-entropy on the cybersecurity forget set with negligible degradation on the general computing retain set. (b) Unlearning: TOFU LIRA blocks undesired knowledge (increases forget set cross-entropy and degrades multiple choice accuracy to near chance—the dashed horizontal line) while keeping desired knowledge (negligible effect on retain set cross-entropy).
Table 5: Jailbreak: Our methods are highly robust against both Embedding Space (ES) and PEZ attacks, with greatly reduced attack success rate (ASR) compared to baselines. Our methods have low rates of false-positive refusals for benign requests and unharmed general-purpose performance measured by MMLU. Backdoors: Our methods almost completely remove backdoors in both cases (default models do not always produce secure code). Our methods negligibly change measures of code capabilities and general-purpose benign capabilities.
Table 8: Unlearning: LIRA has minimal impact on both general-purpose performance (MMLU) and specially chosen benign capabilities related to the “forget” domain, while greatly reducing performance in the “forget” domain.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Benign CF prompts C
Malign prompts M
Benign prompts B
Stopping condition
HATE backdoor
Ordinary Q&A questions
Q&A questions prefixed with a constructed trigger for backdoor behavior
Ordinary Q&A questions
HATE behavior removed in backdoor condition
Embedding space jailbreak attacks
Plain malign requests that elicit refusal
Malign requests prefixed with a constructed safety bypass
Benign request compliance and malign request refusal
Fixed duration of 25 batches
Code backdoor
Code generation prompts with 2024 system prompt
Code generation prompts with a constructed trigger for backdoor behavior
Code generation prompts with 2024 system prompt
Exploitable code behavior removed in constructed backdoor condition
Appendix
Table 9: Example instantiations of LIRA aligning phase for different tasks
Task
Malign metric definition
Benign metric definition
PEZ attacks
Rate at which attacker with 60 prefix or suffix tokens and 10 tries can force the model to comply with a harmful request.
Rate of improper refusals to benign requests. Tracks inappropriate generalization of refusal behavior from harmful to benign requests.
Embedding space attacks
Rate at which attacker with full control of embedding space can force model to comply with harmful requests.
Rate of improper refusals to benign requests. Tracks inappropriate generalization of refusal behavior from harmful to benign requests.
HATE backdoor
Rate at which model emits the word “HATE” when given a backdoor trigger.
—
Code backdoor
Rate at which model generates insecure, exploitable code when given a backdoor trigger.
Increased CE loss on benign, secure code without backdoor trigger. Tracks general degradation of coding capabilities.
Increased CE loss on free-response answers in the retain set covering benign computing info. Tracks degradation of associated benign knowledge because of insufficient targeting.
TOFU unlearning
Decreased accuracy on multiple-choice questions regarding forget set fictitious authors.
Increased CE loss on free-response answers covering retain set fictitious authors. Tracks degradation of associated benign knowledge because of insufficient targeting.
Figure 6: In our embedding space jailbreak task (top) , LIRA and AdLIRA are highly robust to an attacker with control of Gemma 2 2B’s embedding space while alternative methods produce frequent harmful outputs with as little as 3.125% of the embedding space under attacker control. In our code backdoor task (bottom left) ( Hubinger et al., 2024 ) , our AdLIRA causes a backdoored version of Gemma 2 2B to produce code in the backdoor condition almost as secure as in the non-backdoor condition—the dashed horizontal line—while other methods make little progress. In our version of the “HATE” backdoor task (bottom right) from Hubinger et al. (2024) , our LIRA almost entirely removes backdoor behavior from Gemma 2 2B in a single gradient step while baselines have limited success after 100 gradient steps.
Task
Method
Malign metric
Benign metric ↓
MMLU Δ↑
ES attacks
LIRA*
18.1% ASR
2.3% ben. refusal
-2.3% acc.
AdLIRA*
9.6 % ASR
3.1% ben. refusal
-5.9% acc.
Gemma IT
86.9% ASR
0.0 % ben. refusal
+0.0 % acc.
GD
89.8% ASR
6.2% ben. refusal
-0.7% acc.
CB
52.3% ASR
3.1% ben. refusal
-3.6% acc.
Code backdoor
AdLIRA*
-45.1 % insecure
+0.0 code CE
-5.3 % acc.
Appendix
Table 17: Results across tasks
Task
Asset
License
Citation
All
Gemma 2
Gemma Terms of Use
Gemma Team et al. (2024)
Embedding space jailbreaks, code backdoor, and WMDP cyber unlearning
Gemini 2.0 Flash
Gemini API Additional Terms of Service
Google (2024)
Embedding space jailbreaks
Circuit breakers dataset
MIT license
Zou et al. (2024)
Code backdoor
Cybernative.ai Code Vulnerability and Security Dataset
Jailbreaking attacks can enable Large Language Models (LLMs) to bypass the safeguard and generate harmful content. Existing jailbreaking defense methods have failed to address the fundamental issue that harmful knowledge resides within the model, leading to potential jailbreak risks for LLMs. In this paper, we propose a novel defense method called Eraser, which mainly includes three goals: unlearning harmful knowledge, retaining general knowledge, and maintaining safety alignment. The intuition is that if an LLM forgets the specific knowledge required to answer a harmful question, it will no longer have the ability to answer harmful questions. The training of Erase does not actually require the model's own harmful knowledge, and it can benefit from unlearning general answers related to harmful queries, which means it does not need assistance from the red team. The experimental results show that Eraser can significantly reduce the jailbreaking success rate for various attacks without compromising the general capabilities of the model. Our codes are available at https://github.com/ZeroNLP/Eraser.
Weikai Lu, Ziqian Zeng, Jianwei Wang +5
South China University of Technology, China · Pazhou Laboratory, China
Large language models (LLMs) remain vulnerable to jailbreak attacks that conceal harmful intent within complex adversarial prompts. Existing defenses primarily rely on input perturbation or harmful-output suppression, but they rarely model where malicious intent resides, resulting in brittle protection and excessive over-refusal. We propose SENTINEL, a plug-and-play, generation-time jailbreak defense that reframes mitigation as an intent extraction problem. Our key insight is that instruction-tuned LLMs exhibit strong input--output semantic consistency: regardless of jailbreak complexity, generated outputs tend to align with the attacker's true intent. SENTINEL exploits this property by matching semantically aligned input--output regions to extract intention-revealing subsequences, scores these subsequences using refusal-direction projections to estimate harmfulness, and halts generation when necessary. Experiments on HarmBench across multiple LLMs show that SENTINEL reduces jailbreak success rates to close to 5% while maintaining low over-refusal. We further demonstrate robustness to adaptive attacks and provide a mechanistic interpretation: SENTINEL re-distributes jailbreak features from alignment blind spots to aligned regions.
Luoyu Chen, Weiqi Wang, Chenhan Zhang +2
University of Technology Sydney, Sydney, NSW, Australia · Xi’an Jiaotong University, Xi’an, Shaanxi, China · Southeast University, Nanjing, Jiangsu, China
Large Language Models (LLMs) are trained to refuse harmful requests, yet they remain vulnerable to jailbreak attacks that exploit weaknesses in conversational safety mechanisms. We introduce Incremental Completion Decomposition ICD, a trajectory-based jailbreak strategy that elicits a sequence of single-word continuations related to a malicious request before eliciting the full response. In addition, we propose ICD variants that use model-generated or attacker-injected intermediate continuations, as well as final-response prefilling. We evaluate these variants across a broad set of open-weight model families, demonstrating superior Attack Success Rate (ASR) on AdvBench, JailbreakBench, and StrongREJECT compared to existing methods. In addition, we provide a theoretical account of why ICD is effective and present mechanistic evidence that successful attack trajectories suppress refusal-related representations and shift activations away from safety-aligned states.