Defensive training methods such as positive preventative steering (PPS) and inoculation prompting (IP) offer surprising results through seemingly similar processes: both add trait-inducing objects to large language models (LLMs) during training, and both defend the LLM against acquiring the trait. The surprising success of these methods comes with the question: how do they work? Are PPS and IP doing the same thing? We provide behavioral and mechanistic comparisons of these two methods using "evilness" as a case-study trait. Our central finding is that PPS and IP achieve their defensive benefits through distinct mechanisms. Behaviorally, we show that neither PPS nor IP operates through a purely associative mechanism; and PPS can both defend against trait acquisition and actively reduce pre-existing expression, whereas IP is ineffective in models that were previously finetuned to express the trait. This behavioral divergence is reflected mechanistically: PPS shifts the activation gradient towards an attenuating direction along the PPS vector axis. When the PPS vector is aligned with a trait-expressing axis, it can reverse the gradient pressure, reducing rather than increasing activation along that axis. In contrast, IP continues to resist a precise mechanistic account. Direct cosine similarity analyses reveal that IP has a characteristically different gradient signature than PPS, and qualitative analyses reveal IP's gradient to be more diffuse. Furthermore, IP reduces the next-token prediction loss on trait-expressing data where PPS need not, consistent with the notion that IP "explains away" the trait-expression in the training data. Taken together, our analyses reveal distinct mechanisms by which each method operates and highlight open questions about IP's mechanistic picture.
Figures & tables
Figure 1: Panels (a) and (b) are visual depictions of the gradient of the loss with respect to the activations, ∇L , when training on “evil”-coded data with and without PPS. The black arrows in both panels represent the vector component of a single activation vector along the “evil” persona vector (PV) axis before PPS. The dashed purple arrow represents the addition of the PPS vector (in this case, along the “evil” PV axis). The red and green solid arrows represent the direction of the gradient ∇L along the “evil” PV axis without and with PPS in the forward pass. PPS induces suppressive gradient pressure along the PPS vector direction which would otherwise be amplified. Panel (c) shows the mean tokenwise cosine similarity between ∇LIP produced from IP and ∇LPPS produced from a form of PPS over all model layers designed to be equivalent to IP (Section 4.4 ). The baseline comparison shows ∇LIP compared to ∇L (gradient from the model without IP or PPS). We see that IP is not equivalent to this unique form of PPS—nor other forms (Section 4 ).
Figure 2: PPS and IP have different behavioral effects. Panel (a) shows the raw “evil” trait-expression for models trained without defense (e.g., without PPS or IP). The x-axis shows the “evilness” intensity of the training data. The y-axis shows the training data used to pre-finetune starting models to have varying baseline trait-expression. The None column—equivalent to the None row—shows these starting models’ trait-expression before post-finetuning on the datasets shown on the x-axis. Panel (b) shows the difference in resulting trait-expression between the defenseless post-finetuned models shown in panel (a) and models post-finetuned with PPS and IP respectively. More negative trait expression means more successful defense in the top row and more active trait-expression reduction in the Medium and Heavy rows. Both PPS and IP show defensive effects, and neither meaningfully increase trait-expression, but only PPS shows clear expression reduction. Panel (c) shows the trait expression (y-axis) for PPS and IP trainings on “evil” data using various trait-specific PPS vectors and prompts indicated on the x-axis. Color refers to the defensive method, where gray is defenseless. Base Model simply refers to the starting model. We see that trait-alignment is important for defense. Panels (d), (e), and (f) reveal characteristically different trait-expression of the defenseless, PPS, and IP training methods over the course of training.
Figure 3: PPS causes suppressive gradient pressure along the PPS vector direction. Panels show the effect of defensive objects on the gradients’ cosine similarity with “evil” PV and random vector directions at layer 20, computed on “evil” data. Each point corresponds to a single data sequence averaged over token positions. The x-axis shows the cosine similarity when PPS or IP is included in the forward pass; the y-axis shows the same with the defensive object excluded. Points below the identity (dashed) indicate that the defensive object shifts the gradient away from amplification, towards attenuation along the compared vector direction. Results shown here are taken before training; see Figure 11 for results over training. (a) A random PPS vector (light) in the forward pass induces a greater suppressive gradient shift along the same random direction than PPS using the “evil” PV (dark). (b) PPS using the “evil” PV flips the gradients’ sign along the “evil” PV, whereas PPS using the random vector has little effect. IP shifts the gradient closer to zero along the “evil” PV. (c) Neither “neutral” prompts (light) nor “evil” prompts (dark) show a meaningful gradient shift along a random vector direction. (d) “Evil” IP prompts shift the otherwise amplifying gradient closer to zero along the “evil” PV. Neutral prompts, in contrast, do not.
Dataset
Defense
Ablation Vector
Default
Attenuate
Amplify
Neutralize
Evil
None
Evil
76±2
9±2
93.8±0.5
78±2
Evil
None
Rand
76±2
80±2
71±3
75±2
Normal
None
Evil
0.0±0.0
0.0±0.0
60±8
0.0±0.0
Table 1: “Evilness” trait expression (mean ± SEM) under gradient manipulations applied to all layers during 1-epoch of fine-tuning on evil/normal data. Attenuate forces the gradient component along the Ablation Vector direction to be positive, Amplify forces it negative, and Neutralize removes it entirely. PPS and IP use “evil” PPS vector and inoculation prompts regardless of the ablation vector. See details in Section 4.2 .
Figure 4: IP reduces the loss and PPS vectors can too, but PPS vectors can increase the loss at effective defensive intensities. Panels (a) and (b) show scatter plots where each point shows the next-token prediction loss on a single trait-expressing data sequence averaged over token positions, where PPS or IP is included in the forward pass on the x-axis and excluded on the y-axis. Points that are above the dashed line indicate that defense decreases the loss; points below the line indicate that defense increases the loss. We see in panel (a) that “evil” PPS does not decrease the loss on “evil” data relative to a random vector, whereas “evil” IP does reduce the loss relative to neutral IP in panel (b). Panel (c) shows the loss on the left y-axis and the gradient cosine similarity with the “evil” PV on the right y-axis as a function of steering intensity on the x-axis. The blue lines show results with IP included in the forward pass, whereas orange excludes IP. Baseline Qwen2.5-7B-Instruct is represented at zero intensity without prompts; IP is represented at zero intensity with prompts. We see that prompts reduce the loss and induce a neutralization of the gradient cosine along the “evil” axis. PPS also reduces the loss at low intensities, but increases the loss at larger, more effective intensities. Panel (d) shows the corresponding trait expression (y-axis) of PPS trained models resulting from steering intensities indicated on the x-axis. Taken with Panel (c), we see that PPS effectiveness increases with steering intensity regardless of the loss.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Gradient cosine similarity predicts cross-trait PPS effectiveness. Panels (a)–(d) show the cosine similarity between the gradient and different vectors at each model layer, before the PPS finetuning. Color denotes the steering intensity of the PPS vector applied in the forward pass, where black shows the defenseless baseline and dashed green denotes the default steering intensity of 1.5. The top panels (a, b) use “evil” data; the bottom panels (c, d) use “sycophantic” data. The left panels (a, c) apply the “evil” PPS vector in the forward pass; the right panels (b, d) apply the “sycophantic” PPS vector. The “evil” PPS vector induces strong positive (attenuating) cosines with both the “evil” and “sycophantic” PVs, whereas the “sycophantic” PPS vector only induces a positive cosine in the “sycophantic” PV direction, not the “evil” PV direction. This asymmetry predicts the behavioral results: the evil vector defends against both traits, while the sycophantic vector only defends against sycophancy. Panels (e) and (f) show the residual stream activations projected onto the evil PV (e) and sycophantic PV (f) after defensive finetuning under each cross-trait condition. Dashed lines show the raw projections with the steering vector included in the forward pass, applied at layer 20. Activation reductions along each trait direction track the behavioral trait-expression scores (shown in the adjacent tables), confirming that the pre-training gradient cosine analysis is predictive of post-training outcomes.
Figure 6: Raw activation projections onto “evil” PPS axis show that PPS-trained models reduce “evil” activity whereas IP models do not.
Figure 7: Results of Figure 2 hold even with system prompt change oddity. This figure is similar to Figure 2 except that the trait-expression was evaluated using a different system prompt than the one used during training. Each value in the matrices shown in Panels (a), (b), and (c) shows the “evil” trait-expression score of models under different training conditions. The x-axis shows the evilness intensity of the training data and the y-axis shows the starting model condition (Qwen2.5-7B-Instruct pre-finetuned without defense on the same datasets shown on the x-axis). The None column shows the starting models’ trait-expression before the defensive finetuning which is equivalent to the None row in the defenseless trainings. Panel (a) shows results without defensive methods, (b) shows results for PPS using an “evil” persona vector, (c) shows results for inoculation prompting using “evil”-inducing prompts. Panel (d) shows the trait expression for various training conditions for both PPS and inoculation prompting. The x-axis refers to the defensive object used for the training. Color refers to the defensive method, where gray is defenseless (Base Model simply refers to the starting model: Qwen2.5-7B-Instruct). Note that these results are taken from different evaluation runs than panels (a), (b), and (c), giving some insight into the evaluation variability. Panels (e), (f), and (g) reveal characteristically different trait-expression of the defenseless, PPS, and inoculation prompting training methods respectively over the course of training, starting from Qwen2.5-7B-Instruct, trained on heavily evil data.
Figure 8: More IP prompts leads to worse defense. The y-axis shows “evil” trait-expression for models trained with IP using different numbers of sampled prompts during training. The x-axis shows the number of prompts in the set from which prompts are sampled from for each training sample, where prompts are freshly sampled for each training batch. Fewer prompts leads to lower trait-acquisition for the same training settings.
Figure 9: The cosine similarity between the gradient and the evil persona vector across layers with PPS applied in the forward pass at different layers (denoted by hue). The steering intensity was 1.5.
Figure 10: The cosine similarity between the gradient and the evil persona vector across layers with and without the presence of inoculation prompting.
Figure 11: Gradient cosine analysis along the “evil” PV axis over the course of training. The y-axis shows the cosine similarity between the gradient induced by Defenseless (base) Qwen2.5-7B-Instruct, PPS, and IP along the “evil” PV over the course of each method’s training in the respective panels.
Figure 12: Each panel shows the tokenwise cosine similarity between the gradient ∇ht and the activation delta Δht in black across layers. Each panel also shows the cosine similarity between the gradient and the delta with a persona vector direction in blue and red respectively. See Appx D for more detail.
Figure 13: Each panel shows the cosine similarity between the Δht and the evil persona vector at a specified layer where the delta was computed from a gradient descent step without PPS on the x-axis and with PPS on the y-axis (defense was not included for the Δht=ht,θ′−ht,θ calculation, only for determining the updated weights θ′ ). See Appx D for more detail.
Figure 14: Each panel shows the cosine similarity between the Δht and the evil persona vector at a specified layer averaged over the sequence dimension where the delta was computed from a gradient descent step without inoculation prompting on the x-axis and with prompting on the y-axis (defense was not included for the Δht=ht,θ′−ht,θ calculation, only for determining the updated weights θ′ ). Suffix matching indicates that only the tokens following the inoculation prompts were included in the visualized values. See Appx D for more detail.
Defense
Ablation Vector
Default
Attenuate
Amplify
Neutralize
None
Evil
76.4±2.2
9.1±1.6
93.8±0.5
78.2±1.9
PPS
Evil
7.8±1.3
1.0±0.4
84.7±0.8
9.9±1.6
IP
Evil
0.4±0.3
0.6±0.4
91.4±0.7
0.4±0.3
None
Rand
76.4±2.2
80.0±1.9
70.7±2.5
74.6±2.3
PPS
Rand
7.8±1.3
13.3±1.7
8.2±1.3
9.3±1.5
IP
Rand
0.4±0.3
2.5±0.9
0.6±0.4
0.4±0.3
Appendix
Table 2: “Evilness” trait expression (mean ± SEM) under gradient manipulations applied to all layers during 1-epoch of fine-tuning on evil data. Attenuate forces the gradient component along the Ablation Vector direction to be positive, Amplify forces it negative, and Neutralize removes it entirely. PPS and IP use “evil” PPS vector and inoculation prompts regardless of the ablation vector. See details in Section 4.2 .
Defense
Ablation Vector
Default
Atten.
Ampl.
None
Evil
0.0±0.0
0.0±0.0
60.2±7.5
PPS
Evil
0.0±0.0
0.0±0.0
12.1±3.7
IP
Evil
0.0±0.0
0.0±0.0
64.6±8.6
Appendix
Table 3: Peak “evilness” trait expression (mean ± SEM) under gradient manipulations applied to all layers fine-tuning on “normal” (non-evil) data. The reported values are the peak evil-expression over training with at least a coherence of 20 (coherence drops to zero over training). Attenuate forces the gradient component along the Ablation Vector direction to be positive, Amplify forces it negative, and Default does not manipulate the gradient. PPS and IP use “evil” PPS vector and inoculation prompts. See details in Section 4.2 .
Figure 15: Loss curves corresponding to all 3 training epochs when manually manipulating the gradients. We show trait-expression results for models taken at the vertical line (epoch 1). Otherwise, coherence goes to 0.
Figure 16: The cosine similarity between the gradients induced by IP compared to the gradients induced by activation difference PPS—using the difference in activations with minus without inoculation prompts, preserving layers—as a function of steering intensity (applied equally at all layers; Section 4.4 ). Both panels show the same information sliced differently. The left panel shows steering intensities on the x-axis, where hue denotes the cosine similarity for select layers. The right panel shows the same cosine similarities across layers, using hue to denote the steering intensity.
Figure 17: Cosine similarity (y-axis) measurements across model layers (x-axis) between the “evil” PVs and the prompt-induced mean activation difference vectors from Section 4.4 . See Appx E.6 for details on how the vectors are constructed.
Figure 18: Uncentered PCA decomposition of the defended gradient difference reveals heavily concentrated, trait-aligned structure for PPS and more diffuse, less aligned structure for IP. Panels (a) and (b) show cosine similarity matrices between reference trait vectors (x-axis) and the the top PCs (y-axis) extracted from PCA performed on the difference between the gradient with and without the defensive object in the forward pass. We perform the PCA on the uncentered data to preserve potential common differences among all samples. Each PC is oriented so that it has a positive cosine similarity with the “evil” trait vector. Panel (c) shows the fraction of variance explained by each of the PCs shown in panels (a) and (b). We see that the explained variance for PPS is heavily concentrated on the first PC, whereas the explained variance for IP is more diffuse.
Figure 19: Trait expression and coherence scores as evaluated by GPT 4.1 for finetunings with different LoRA ranks (and LoRA alphas) and for different inoculation prompt placements. These results were collected with the default Qwen system prompt during evaluation, whereas the defensive trainings used a neutral system prompt.
Figure 20: Trait expression and coherence scores as evaluated by GPT 4.1 for finetunings with different LoRA ranks (and LoRA alphas) in PPS trainings. These results were collected with the default Qwen system prompt during evaluation, whereas the defensive trainings used a neutral system prompt. This is why there is a trait-expression discrepancy between the models shown here, and the models shown in the main text.
Figure 21: Evil trait expression and coherence across a range of PPS steering intensities at layer 20 with rank 64.
Figure 22: Evil trait expression and coherence across a range of model layers, using steering intensity 1.5 and rank 64.
Adversarial robustness of language models studies how to keep their outputs and decisions reliable under manipulative prompts, input changes, and malicious fine-tuning.
Aligned large language models (LLMs) are expected to exhibit safety behavior based on the content of the user request: they should refuse unsafe requests and comply with safe ones. However, we show that the same request can elicit substantially different safety decisions under different traits assigned in the system prompt, a failure mode we call trait-induced safety variation. To measure this failure, we introduce refusal-based metrics: Trait-Induced Deviation measures dataset-level deviation from the no-trait baseline, while Trait-Induced Flip Rate measures whether the same request receives different safety decisions across traits. We then provide a representation-level analysis of the mechanism behind trait-induced safety shifts and find that traits perturb the model's safety representations within a low-dimensional subspace. To achieve trait-invariant safety, where safety behavior remains stable across traits, we introduce Trait-Invariant Safety Tuning (TIST), a simple yet effective self-distillation framework that aligns an LLM's trait-conditioned behavior with its no-trait behavior. Guided by our analysis, we further propose Trait-Subspace Neutralization (TraSN), an instantiation of TIST, which enforces invariance only within the identified trait subspace. Experiments show that TraSN improves trait-invariant safety and strengthens harmful-request safety while preserving general capability. Our results highlight traits as an important factor in LLM safety and robust model behavior.
Current safety methods for large language models are known to be vulnerable to adversarial attacks, motivating research into robust alternatives. Latent Adversarial Training (LAT) is among the most effective defenses, but can degrade utility and requires training on large datasets of harmful prompts. We introduce Latent Personality Alignment (LPA), which replaces explicit harm refusal with adversarial training on just 66 harm-agnostic statements drawn from psychometric personality literature. We hypothesize that personality-anchored representations share latent structure with harm avoidance, so adversarially stabilizing them implicitly constrains the subspace exploited by jailbreak attacks. LPA achieves near-zero attack success rates on HarmBench across direct requests and five jailbreak methods, despite never seeing harmful content during training and no loss of performance on standard benchmarks. Moreover, the training process is lightweight; the entire procedure completes in minutes on a single GPU and uses 75x fewer examples than standard LAT. Extensive ablations demonstrate the robustness, efficiency, and generalization of our method.
Alignment in LLMs is more brittle than commonly assumed: misalignment can be induced by adversarial prompts, benign fine-tuning, emergent misalignment, and goal misgeneralization. Recent evidence suggests that some misalignment behaviors are encoded as linear structure in activation space, making it tractable via activation steering, which could be used as a lightweight runtime defense. We implement three methods: Steer-With-Fixed-Coefficient (SwFC), which applies uniform additive steering, and two novel projection-aware methods, Steer-to-Target-Projection (StTP) and Steer-to-Mirror-Projection (StMP), that use a logistic regression decision boundary to selectively intervene only on tokens whose activations fall below the threshold. We evaluate these methods on two threat models, dishonesty and dismissiveness, using malicious system prompts as a controlled proxy for misalignment. We conduct our experiments on two architectures (Llama-3.3-70B-Instruct and Qwen3.6-27B). All methods substantially recover alignment. StTP and StMP preserve general capabilities (MMLU, MT-Bench, AlpacaEval) better than uniform steering. Finally, we show that our honesty steering generalizes to out-of-distribution scenarios: a single honesty direction extracted from the aligned model significantly raises scores on the MASK benchmark, suppresses deception in multi-agent settings (Among Us), doubles the hidden-behavior discovery rate on AuditBench, and restores honesty in an emergently misaligned model.
Niklas Herbster, Martin Zborowski, Alberto Tosato +2
Tara Research · Technical University of Munich · Mila Quebec AI Institute