At K=16 tokens (0.4% of a 4K context), every existing KV-cache compression method achieves 0% on credential retrieval. The failure mode is dormant tokens: credentials, API keys, and configuration values that receive near-zero attention but become essential at generation time. Because these tokens lack the statistical signals that eviction policies rely on, no method based on attention scores, reconstruction loss, or learned retention gates retains them. We introduce Transactional Attention (TA), a sponsorship mechanism in which structural anchor patterns (e.g., "key:", "password:") protect adjacent value-bearing tokens from eviction. TA achieves 100% credential retrieval at K=16 where six baselines (H2O, TOVA, SnapKV, StreamingLLM, PyramidKV, DynamicKV) achieve 0%, and sustains 100% accuracy across 200 function-calling trials. TA-Fast, an attention-free variant, reduces memory overhead by 52% and is compatible with SDPA and FlashAttention. TA is orthogonal to existing compression methods and adds less than 1% latency overhead.
Figures & tables
Figure 1: Transactional Attention at a glance. (a) Dormant tokens (credentials, IDs) receive near-zero attention, making them invisible to scoring-based eviction. (b) At K=16 , all baselines evict them—the credential ranks 3,847th of 4,000 tokens. (c) TA’s sponsorship mechanism detects the anchor pattern ( key: ) and protects adjacent value tokens. (d) Needle-in-haystack accuracy: TA achieves 100% at all budgets; baselines achieve 0% at K≤64 .
Figure 2: Sponsorship mechanism in four steps. (1) Anchor detector identifies structural patterns (e.g., API_KEY: ). (2) Sponsor budget is allocated with exponential decay ( Vj=0.8j−i ) over the next L=6 tokens. (3) Utility scores incorporate the sponsorship voucher, boosting value tokens above the retention threshold. (4) Top- K selection retains sponsored tokens alongside high-attention tokens.
Method
K=16
K=32
K=64
K=128
K=256
TA
100
100
100
100
100
TOVA
0
0
0
100
100
H2O
0
0
0
0
0
SnapKV
0
0
0
100
100
StreamingLLM
0
0
0
0
0
PyramidKV
0
0
0
0
46
Table 1: Needle-in-haystack accuracy (%) on Llama-3.2-1B at 4K context. At K=16 , Transactional Attention achieves 100% while attention-based methods achieve 0%. All baselines use official implementations or published algorithms.
Figure 3: Why baselines fail at K=16 . (a) H2O ranks by cumulative attention; the credential scores near zero, ranked 3,847th of 4,000. (b) TOVA’s sliding window covers only the last 12 tokens; the credential at depth 0.5 is ∼2,000 tokens away. (c) TA’s sponsorship boosts the credential into the top 16, ensuring retention.
Figure 4: Accuracy vs. retention budget K . TA achieves 100% at all budgets. Baselines require K≥128 to reach 100%.
Method
12-char
32-char
Overall
TA-Fast-Semantic
100%
100%
100%
TA-Fast-Regex
94%
88%
91%
TOVA
0%
0%
0%
Table 2: Tool-call benchmark (K=32, n=200). Function-calling format is the deployment scenario for agentic LLMs.
Figure 5: TA modifies the scoring function (green), not the eviction policy. It slots into any existing pipeline. General performance is unchanged: LongBench single-document QA is 42% (TA) vs 45% (TOVA).
Model
K
TA-Rgx
TA-Sem
TOVA
No Spn
Llama-3.2-1B
16
100
100
0
0
Llama-3.1-8B
16
100
100
0
0
Mistral-7B
16
5
5
0
0
Mistral-7B
24
5
100
0
0
Table 3: Accuracy (%) across model scales. Budget K is total tokens retained. TA-Regex uses pattern matching; TA-Sem adds verb detection.
Ctx
K
Method
Acc
Lat.
Mem
4K
16
TA
100%
180ms
2.1GB
4K
16
TOVA
0%
175ms
2.0GB
7K
32
TA (SDPA)
100%
372ms
6.8GB
7K
32
TOVA (eager)
OOM
–
> 50GB
Table 4: Scaling: TA-Fast (SDPA) vs attention-based (eager). TA-Fast uses position + semantics only.
Figure 6: Hyperparameter sensitivity at K=16 . Most utility weights show 0% accuracy spread—the sponsor budget is the only critical parameter (accuracy ranges 27%–100%).
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Experiment
Variant
Main needle (Tab 1 )
Regex, Semantic
Multi-model (Tab 3 )
Regex, Full-TA
Multi-slot (Tab 9 )
Regex (allowlist)
4-value agent
Regex (allowlist)
Anchor abuse
Regex ( ± allowlist)
Tool-call (Tab 2 )
Semantic
Appendix
Table 5: TA variant used in each experiment.
Method
K=16
K=32
K=64
TA
30.4
28.1
27.2
TOVA
31.2
28.8
27.0
H2O
45.3
38.2
31.5
StreamingLLM
52.1
42.7
35.8
Appendix
Table 6: Perplexity on WikiText-2 (lower is better). Transactional Attention achieves competitive perplexity while enabling retrieval.