POEM: Output-Relative Block-Sparse Attention for Long-Context Prefilling
Authors: Haoyi Wu, Yunkai Dang, Wanyu Zhang, Lei Wang, Wenbin Li
Organizations: State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · University of Wollongong, Australia · Shenzhen Research Institute of Nanjing University, Shenzhen, China
Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment safeguards and induce LLMs to generate harmful responses. Some representation-based defenses regulate internal activations using predefined steering directions or learned safety regions. However, adapting intervention to each input without a predefined representational target while limiting over-refusal remains a challenge. In this paper, we propose Target-free Entropy-guided Residual Modulation (TERM), an inference-time defense that uses prompt-calibrated attention entropy deviations to guide adaptive modulation of the model's own residual updates. Attention patterns vary across prompts and decoding steps, making raw entropy values difficult to interpret as a consistent signal for intervention. We therefore calibrate attention entropy against each prompt's own prefill baseline to establish an input-specific reference for reducing unnecessary intervention on benign requests. Adaptive scale normalization and temporal accumulation then account for differences in deviation scale and persistence to determine intervention strength. The resulting signal controls bounded scaling of the model's own residual updates, without relying on a predefined safety direction. We compare our method with seven defenses on seven jailbreak attacks and two over-refusal benchmarks across Llama, Qwen, and Gemma models, including Qwen3 models from 0.6B to 32B. Results show our method improves jailbreak robustness while maintaining low over-refusal on benign safety-sensitive requests with modest inference overhead.
Figures & tables
Figure 1: Through experiments, we observe that blocks with similar attention mass can differ by over 20× in output deviation. Removing a high-deviation block (B1) shifts the attention output far more than removing a low-deviation one (B3) of comparable mass. We therefore combine attention mass with output deviation into our output-relative proxy. V denotes the exact output of a block.
Method
Proxy Score
Value- Aware
Output- Relative
MInference
dqk⊤
✗
✗
FlexPrefill
dqk⊤
✗
✗
XAttention
dqk⊤∗
✗
✗
STEM
dqk⊤∗+βReLU(logvmax)
✓
✗
POEM
dqk⊤∗+βlogv+dqC−o
✓
✓
Table 1: Comparison of proxies. ∗ means antidiagonal scoring, and vmax denotes the maximum value norm within the block.
Figure 2: Overview of POEM. Stage 1 preselects candidate blocks using coarse attention mass. Stage 2 reranks candidates with output-relative scores and selects blocks using calibrated headwise thresholds. p1 is the Stage 1 threshold, and p2,h is the calibrated threshold for head h .
Method
Layer 5 (×10−4)
Layer 15 (×10−2)
Layer 35
Final logits
Mass only
3.27
1.24
5.88
2.86
STEM
3.27
1.23
5.88
2.85
POEM
3.25
1.22
5.79
2.74
Table 2: Layer and logit output errors on LongBench, using Qwen3-8B.
Model
Method
LongBench
RULER
S-Doc
M-Doc
Summ.
FSL
Synth.
Code
Avg.
4K
8K
16K
32K
64K
128K
Avg.
Llama-3.1-8B
Dense
28.57
25.75
25.45
63.61
65.83
33.06
40.38
95.26
94.29
93.58
87.99
84.85
77.87
88.97
MInference
28.38
25.44
25.28
63.31
65.12
31.16
39.78
95.43
94.25
93.44
87.56
83.21
71.51
87.57
FlexPrefill
24.25
22.61
25.14
62.24
19.56
35.01
31.47
93.67
93.63
93.84
87.90
84.83
72.43
87.71
XAttention
27.51
23.55
25.51
63.82
55.83
35.77
38.66
95.09
94.39
93.27
88.95
82.45
70.09
87.38
STEM
27.20
24.33
25.41
62.56
64.32
32.74
39.43
95.27
94.50
93.12
88.21
82.94
70.33
87.40
Table 3: LongBench and RULER results on three models. Best in bold, second best underlined.
Figure 3: (a) Distribution of exact block attention mass mib . (b) Distribution of output deviation ∥vˉib−oi∥2 . (c) Distribution of exact importance Iib=mib∥vˉib−oi∥2 . (d) Percentage of blocks replaced per head after output-relative reranking, 74% of heads are affected.
Variant
Word PPL
Δ PPL
POEM
43.65
+0.00
w/o cov.
44.01
+0.36
w/o output
43.95
+0.30
Table 4: PG19 ablation at 32k with 0.1 density on Qwen3-8B.
Method
Retrieval
MHT
Agg.
QA
Overall
Budget
S1
S2
S3
MK1
MK2
MK3
MV
MQ
VT
CWE
FWE
QA1
QA2
Avg.
Baseline
100.00
99.00
100.00
92.00
62.00
43.00
93.75
97.75
68.20
0.10
73.00
44.42
36.00
69.94
19.50 %
+ output deviation
100.00
99.00
100.00
93.00
69.00
48.00
95.75
98.25
71.00
0.10
73.67
52.08
38.00
72.14
19.56%
+ threshold calibration
100.00
98.00
100.00
94.00
70.00
46.00
95.75
98.50
74.80
0.20
73.00
54.75
36.00
72.38
18.53 %
Table 5: POEM module ablation on RULER at 128K. Best results are in bold, and second-best results are underlined.
Figure 4: POEM ablations: (a) output-relative weight. (b) score–time trade-off. (c) Stage-1 mass threshold. (d) PG19 perplexity vs density for different scoring proxies.
Figure 5: Attention latency (kernel/total). Dark/light segments show kernel/selector time. POEM is fastest throughout, reaching 6.5× speedup over dense FA2 at 128K.
Figure 6: Latency breakdown of POEM. Selection overhead remains modest across context lengths, while sparse attention dominates the execution time at long contexts.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Cosine similarity between the first-order estimate vib and the exact token-level causal block output vibexact . White bars mark weighted medians.
Figure 8: Head-level cosine similarity between the Stage 1 global-output estimate and the dense attention output across layers.
Figure 9: Layerwise coverage of value-aware selections by mass-only preselection for p1∈{0.90,0.92,0.94} . The missed fraction is the pooled count of all-visible reference-selected blocks outside the Stage 1 pool divided by the pooled reference-selected count. At p1=0.94 , the missed fraction is at most 0.035% across all 16 measured layer/length combinations.
Figure 10: Attention-mass profiles for the 0.85 – 0.90 cumulative-mass shell and the tail beyond 0.94 , sorted independently within each group. The tail has substantially lower raw Stage 1 block mass over most ranks across all sampled layers and context lengths.
Figure 11: Output-deviation profiles for the 0.85 – 0.90 cumulative-mass shell and the tail beyond 0.94 , sorted independently within each group. The profiles largely overlap except for the upper-ranked tail in later layers, most notably layer 31.
Figure 12: Layerwise distributions of raw value-deviation norms over all visible blocks. Rows show Llama-3.1-8B layers 5, 15, 25, and 31 (zero-based); columns show four input lengths. Each panel pools two PG19 texts, 32 heads, and eight query rows per text and head, with equal weight per visible block observation. The horizontal axis is logarithmic, and bins have width 0.05 in log10 deviation. All panels share the same axes, without additional per-head or per-row normalization. Layer 31 has a broader right tail, while each layer’s central distribution varies modestly across lengths.
Q/K copies
Covariance
Length
Predicted payload
Measured with scales
Predicted
Measured
16K
80
80.020
32
32
32K
160
160.039
64
64
64K
320
320.078
128
128
128K
640
640.156
256
256
Appendix
Table 6: Predicted and measured auxiliary tensor storage (MiB). The actual INT8 copies and BF16 covariance cache match their shape-based predictions on all four inputs at each length. Q/K measurements also include the small FP32 quantization-scale tensors.
Length
Dense (GiB)
POEM (GiB)
Increase (MiB)
16K
19.147
19.147
0.033
32K
23.030
23.031
0.127
64K
30.796
30.797
0.503
128K
46.329
46.331
2.005
Appendix
Table 7: End-to-end peak allocated GPU memory during full-model prefilling. The increase is computed from the unrounded measurements.
Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment safeguards and induce LLMs to generate harmful responses. Some representation-based defenses regulate internal activations using predefined steering directions or learned safety regions. However, adapting intervention to each input without a predefined representational target while limiting over-refusal remains a challenge. In this paper, we propose Target-free Entropy-guided Residual Modulation (TERM), an inference-time defense that uses prompt-calibrated attention entropy deviations to guide adaptive modulation of the model's own residual updates. Attention patterns vary across prompts and decoding steps, making raw entropy values difficult to interpret as a consistent signal for intervention. We therefore calibrate attention entropy against each prompt's own prefill baseline to establish an input-specific reference for reducing unnecessary intervention on benign requests. Adaptive scale normalization and temporal accumulation then account for differences in deviation scale and persistence to determine intervention strength. The resulting signal controls bounded scaling of the model's own residual updates, without relying on a predefined safety direction. We compare our method with seven defenses on seven jailbreak attacks and two over-refusal benchmarks across Llama, Qwen, and Gemma models, including Qwen3 models from 0.6B to 32B. Results show our method improves jailbreak robustness while maintaining low over-refusal on benign safety-sensitive requests with modest inference overhead.
Yunkai Dang, Wanyu Zhang, Meiyi Zhu +3
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · Microsoft, Redmond, WA, USA · Shenzhen Research Institute of Nanjing University, Shenzhen, China
Large language models have achieved remarkable performance across a wide range of tasks, yet they remain susceptible to jailbreak attacks that circumvent safety mechanisms and elicit harmful or unintended outputs. Although safety can be improved through model fine-tuning, such approaches can be computationally expensive and may degrade previously acquired capabilities through catastrophic forgetting. Recent jailbreak defenses have therefore explored targeted interventions on specific model components associated with unsafe behavior. However, existing approaches typically decouple two inherently related decisions: where to intervene and how to modify the selected components. Such sequential optimization may be suboptimal because the optimal intervention location can depend on the intervention applied there, and vice versa. In this work, we introduce AttentionSwitch, the first jailbreak defense to jointly optimize intervention sites and strengths within a frozen LLM. Specifically, AttentionSwitch learns a sparse set of attention heads together with query-conditioned multiplicative scaling factors, allowing the outputs of the selected heads to be either attenuated or amplified without altering the directions of their representations. Across popular language models and jailbreak attacks, AttentionSwitch outperforms existing defenses, with particularly strong results when training data is scarce.
Doniyorkhon Obidov, Honggang Yu, Xiaolong Guo +1
Michigan Technological University · Miami University · Lehigh University
Large language models (LLMs) remain vulnerable to jailbreak attacks that conceal harmful intent within complex adversarial prompts. Existing defenses primarily rely on input perturbation or harmful-output suppression, but they rarely model where malicious intent resides, resulting in brittle protection and excessive over-refusal. We propose SENTINEL, a plug-and-play, generation-time jailbreak defense that reframes mitigation as an intent extraction problem. Our key insight is that instruction-tuned LLMs exhibit strong input--output semantic consistency: regardless of jailbreak complexity, generated outputs tend to align with the attacker's true intent. SENTINEL exploits this property by matching semantically aligned input--output regions to extract intention-revealing subsequences, scores these subsequences using refusal-direction projections to estimate harmfulness, and halts generation when necessary. Experiments on HarmBench across multiple LLMs show that SENTINEL reduces jailbreak success rates to close to 5% while maintaining low over-refusal. We further demonstrate robustness to adaptive attacks and provide a mechanistic interpretation: SENTINEL re-distributes jailbreak features from alignment blind spots to aligned regions.
Luoyu Chen, Weiqi Wang, Chenhan Zhang +2
University of Technology Sydney, Sydney, NSW, Australia · Xi’an Jiaotong University, Xi’an, Shaanxi, China · Southeast University, Nanjing, Jiangsu, China