Context compression reduces inference cost by encoding inputs into compact representations while preserving semantic content. An open question is whether semantic preservation alone is sufficient to maintain downstream behaviours such as refusal alignment. We investigate this question and find that encoder-based compression systematically weakens refusal behaviour in instruction-tuned language models, despite high reconstruction fidelity. This effect persists across model families and compression architectures. Mechanistic analysis shows that compression attenuates activation along the decoder's learned refusal direction. We further explore Memory Steering, a lightweight inference-time intervention that restores refusal rates to near-baseline levels without retraining and operates entirely in compressed representation space. These results demonstrate that semantic preservation does not guarantee behavioural preservation under compression, highlighting the need to explicitly preserve alignment-relevant features in compression-aware systems.
Figures & tables
Figure 1: A high-level illustration of how context compression weakens refusal behaviour in instruction-tuned language models.
Model
Safe
Harmful
Mistral-7B-Instruct-v0.2
0.00
62.82
Mistral-7B-Instruct-v0.2 + ICAE
0.00
10.9
Mistral-7B-Instruct-v0.3
0.00
20.51
Mistral-7B-Instruct-v0.3 + ARC
0.00
8.97
LLaMA-2-7B-Chat
0.00
97.44
LLaMA-2-7B-Chat + ICAE
3.2
85.9
Table 1: Refusal rates (%) on safe and harmful prompts with and without encoder-based context compression. ICAE and ARC results are reported with their corresponding instruction-tuned decoder baselines. Results are measured on our held-out evaluation set.
Model
HarmBench (%)
AdvBench (%)
ARC Compression
LLaMA-3.1-8B-Instruct
84.0
93.46
LLaMA-3.1-8B-Instruct + ARC
47.5
70.19
Mistral-7B-Instruct-v0.3
28.5
42.88
Mistral-7B-Instruct-v0.3 + ARC
23.5
31.54
ICAE Compression
Table 2: Refusal rates (%) on HarmBench and AdvBench under uncompressed and encoder-compressed settings. All values correspond to harmful-prompt refusal rates. Higher values indicate higher refusal rates (i.e., stronger safety compliance).
Figure 2: Projection onto the learned refusal direction across decoder layers under uncompressed and ICAE-compressed inputs. Top: LLaMA-2-7B-Chat. Bottom: Mistral-7B-Instruct-v0.2. Harmful prompts that elicit refusal in the baseline model show consistent positive activation in the uncompressed condition. Under ICAE compression, this activation collapses, aligning with the non-refusal regime.
Figure 3: Refusal rates versus steering strength α for safe and harmful prompts on Mistral-7B-Instruct-v0.2 (top) and LLaMA-2-7B-Chat (bottom).
Steering α
Mistral-7B-Instruct-v0.2 + ICAE
LLaMA-2-7B-Chat + ICAE
0.0 (baseline)
20.38
90.96
0.5
41.92
98.08
1.0
67.31
99.62
1.5
76.54
100.00
2.0
81.15
100.00
Table 3: AdvBench harmful-prompt refusal rates (%) under Memory Steering for ICAE-compressed inputs. Baseline corresponds to α=0 .
Steering α
Mistral-7B-Instruct-v0.2 + ICAE
LLaMA-2-7B-Chat + ICAE
0.0 (baseline)
16.50
77.00
0.5
24.00
86.50
1.0
39.50
94.00
1.5
50.00
96.50
2.0
57.50
97.50
Table 4: HarmBench harmful-prompt refusal rates (%) under Memory Steering for ICAE-compressed inputs. Baseline corresponds to α=0 .
Model
α
Win
Lose
Tie
On-par (%)
LLaMA-2-7B-Chat
0.5
23
20
113
85.9
1.0
26
24
106
83.3
1.5
30
27
99
82.7
2.0
31
32
93
79.5
Mistral-7B-Instruct-v0.2
0.5
17
19
120
87.8
1.0
24
27
105
82.7
Table 5: Pairwise response quality comparison on safe prompts between Memory Steering ( α>0 ) and the ICAE baseline ( α=0 ), judged by Gemma-3-27b-it. On-par rate is the fraction of examples where the steered response is judged better or indistinguishable from the baseline.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Optimizer
AdamW
Learning Rate
1×10−3
Batch Memory Size
64
Weight Decay
0
Epochs
3
Appendix
Table 6: Training hyperparameters for the gating classifier.
Hyperparameter
Value
Optimizer
AdamW
Learning Rate
1×10−4
Batch Size
32
Update Steps
500
Warm-up Steps
50
Weight Decay
0.01
Appendix
Table 7: Fine-tuning hyperparameters for the ARC-Encoder to be paired with instruct-tuned models.