TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization

Dipkumar Patel

cs.LGMar 29, 2026

TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization

Authors: Dipkumar Patel

Organizations: LLMs Research Inc.

Abstract

We compress KV cache entries by quantizing angles in the Fast Walsh-Hadamard domain, where a random diagonal rotation makes consecutive element pairs approximately uniformly distributed on the unit circle. We extend this angular quantizer with per-layer early-boost, which independently configures K and V codebook sizes at each layer, allocating higher precision to a model-specific subset of critical layers. Across seven models (1B to 7B parameters), per-layer early-boost achieves lossless compression on four models and near-lossless quality on six of seven, at 3.28 to 3.67 angle bits per element. Asymmetric norm quantization (8-bit for keys, 4-bit log-space for values) yields 6.56 total bits per element on Mistral-7B with perplexity degradation of +0.0014 and no calibration data. A layer-group sensitivity analysis reveals model-specific bottleneck patterns, including K-dominated versus V-dominated layers and negative-transfer layers where increased precision degrades quality.

Figures & tables

Figure 1: TurboAngle pipeline. Top: the compression path applies a random diagonal rotation $D$ , the normalized FWHT $H$ , polar decomposition of consecutive pairs, and uniform angle quantization on $S^{1}$ , storing angle indices $k_{i}$ and norms $r_{i}$ . Bottom: reconstruction maps $(k_{i},r_{i})$ back to Cartesian coordinates via trigonometric lookup, then applies the inverse FWHT to recover the approximate KV vector.

Method	Bits/elem	$\Delta$ PPL $\downarrow$
Method	Bits/elem	Mistral-7B	TinyLlama
TurboAngle ( $n=32$ )	2.50	+0.0104	+0.0694
TurboAngle ( $n=48$ )	2.79	+0.0034	+0.0150
TurboAngle ( $n=64$ )	3.00	+0.0010	+0.0176 †
TurboAngle ( $n=128$ )	3.50	+0.0030	+0.0036
TQ-sym4-g4	4.00	+0.0148	+0.1295

Table 1: Angular vs scalar quantization.

\Delta

PPL (lower is better). TurboAngle bit rates count angle bits only; norms are stored in fp32.

Model	$L$	PPL ${}_{\text{base}}$	Uniform (3.25b)	Best per-layer
			$\Delta$ PPL	$\Delta$ PPL	bits
TinyLlama-1.1B	22	8.913	+0.0011	$\mathbf{-0.0022}$	3.34
Mistral-7B	32	5.844	+0.0018	$+0.0002$	3.31
SmolLM2-1.7B	24	8.930	+0.0071	$\mathbf{-0.0003}$	3.67
phi-1.5	24	28.63	+0.0245	$\mathbf{\phantom{-}0.0000}$	3.58
StableLM-2-1.6B	32	9.790	+0.0207	$+0.0012$	3.63

Table 2: Per-layer early-boost results on seven models. WikiText-2 perplexity at 32K tokens. “Uniform” is the K128V64 baseline (3.25 angle bits/element). Best per-layer config is the optimal configuration found through systematic sweep. Angle bits count only angular indices; norms are in fp32.

Model	Boosted layers	$n_{K}^{\text{early}}$	$n_{V}^{\text{early}}$	Type	Notes
TinyLlama	0–3	128	256	V-dom	V=256 required; K=128 sufficient
Mistral-7B	0–3	256	128	K-dom	K=256 required; E8 is worse
SmolLM2	0–19	256	128	K+V	Lossless requires 20 of 24 layers
phi-1.5	0–7, 16–23	256	128	K-sel	Skip 8–15 (negative transfer)
StableLM-2	0–23	256	128	K+V	24 of 32 layers; sharp cliff at E24
StarCoder2	0–15	256	128	K+V	16 of 40 layers; non-monotonic

Table 3: Optimal per-layer configurations.

n_{K}^{\text{early}},n_{V}^{\text{early}}

are the angle codebook sizes for boosted layers; remaining layers use

n_{K}{=}128,n_{V}{=}64

Group	Layers	$\Delta$ PPL	Interpretation
G0	0–3	+0.0122	Most beneficial (50% reduction)
G1	4–7	+0.0175	Second most beneficial
G5	20–23	+0.0157	Third; unexpectedly helpful
G2	8–11	+0.0192	Marginal improvement
G4	16–19	+0.0210	Marginal improvement
G3	12–15	+0.0263	Negative transfer : worse than uniform

Table 4: Layer-group sensitivity for phi-1.5. Each row boosts one 4-layer group to K256V128 (3.33 angle bits) while the rest stays at K128V64 (3.25 bits). Uniform baseline

\Delta

PPL = +0.0245.

Model	$d$	FP32 $\Delta$ PPL	norm8 $\Delta$ PPL	K8V4-log $\Delta$ PPL	K8V4-log bits
TinyLlama-1.1B	64	$-$ 0.0022	+0.0011	+0.0104	$\sim$ 6.84
Mistral-7B	128	+0.0002	+0.0012	+0.0014	$\sim$ 6.56
SmolLM2-1.7B	64	$-$ 0.0003	+0.0027	+0.0030	$\sim$ 7.67
phi-1.5	64	0.0000	$-$ 0.0017	+0.0017	$\sim$ 7.58
StableLM-2-1.6B	64	+0.0012	+0.0021	+0.0123	$\sim$ 7.63
StarCoder2-3B	64	$-$ 0.0007	$-$ 0.0007	+0.0061	$\sim$ 7.45

Table 5: Norm quantization results.

\Delta

PPL relative to fp16 inference. “FP32” column reproduces the best per-layer angle-only results from Table 2 . “norm8” applies 8-bit per-vector min-max quantization to all norms. “K8V4-log” uses 8-bit linear K norms and 4-bit log-space V norms. Total bits includes angle bits, norm bits, and per-vector min-max overhead.

Method	Total bits	$\Delta$ PPL	Calibration	Source
CQ-2c8b [ 6 ]	4.00	+0.03 (Mistral)	Yes	NeurIPS 2024
KVQuant-4b-1% [ 7 ]	4.32	+0.01 (LLaMA-7B)	Yes	NeurIPS 2024
AQUA-KV 3b [ 3 ]	$\sim$ 3.0	+0.03 (Llama-3.1-8B)	Yes	ICML 2025
TurboAngle K8V4-log	6.56	+0.0014 (Mistral)	No	This work
TurboAngle norm8	7.81	+0.0012 (Mistral)	No	This work

Table 6: Comparison with calibration-based KV cache quantizers.

\Delta

PPL is the reported perplexity degradation on the respective evaluation model. TurboAngle requires zero calibration data and no per-channel statistics.

KV-Cache Compression

Key-value cache compression reduces the memory and bandwidth needed to retain attention states during long-context inference.

See all 265 papers

Sequential KV Cache Compression via Probabilistic Language Tries: Beyond the Per-Vector Shannon Limit
Apr 10, 2026Gregory MagarshakKV-Cache Compression Memory Compression
KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation
Sep 21, 2026Sihyeon Ha, Jaeho Lee, Yo-Seb JeonLow-Rank Compression KV-Cache Compression
Pohang University of Science and Technology (POSTECH)
A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation
Jul 14, 2026Rahul Krishnan, Volker SchulzTucker Decomposition KV Caching
Universität Trier · Fachbereich IV, Mathematik, Universität Trier

Apr 10, 2026J/K move · Enter open · S save

Sequential KV Cache Compression via Probabilistic Language Tries: Beyond the Per-Vector Shannon Limit

Gregory Magarshak

KV-Cache Compression Memory Compression Predictive Coding

Open paper AskPDF arXiv

Recent work on KV cache quantization, culminating in TurboQuant, has approached the Shannon entropy limit for per-vector compression of transformer key-value caches. We observe that this limit applies to a strictly weaker problem than the one that actually matters: compressing the KV cache as a sequence. The tokens stored in a KV cache are not arbitrary floating-point data -- they are samples from the exact formal language the model was trained on, and the model is by construction a near-optimal predictor of that language. We introduce sequential KV compression, a two-layer architecture that exploits this structure. The first layer, probabilistic prefix deduplication, identifies semantically equivalent shared prefixes across sessions using the trie metric d_T(s, s') = -log_2 P_M(s ^ s') from Probabilistic Language Tries (PLTs). The second layer, predictive delta coding, stores only the residual of each new KV vector from the model's own prediction of it, achieving a per-token entropy bound of H(KV_{i+1} | KV_{<=i}) <= H(token_{i+1} | token_{<=i}). We prove that at typical language model perplexity -- approximately 10-20 for fluent English text -- this bound is 3.3-4.3 bits on average per token position, compared to TurboQuant's 3 bits per vector component (with typical attention heads having 64-128 components). The theoretical compression ratio over TurboQuant is approximately 914,000x at the Shannon limit. Even at 1000x above the entropy floor -- a deliberately pessimistic worst-case overhead, two orders of magnitude above the 2-5x typical of practical source coders -- the ratio remains approximately 914x over TurboQuant, with compression improving rather than degrading as context length grows. The two layers are orthogonal and compose with existing per-vector quantization methods including TurboQuant.

Abstract

KV-Cache Compression

Explore similar work

Sequential KV Cache Compression via Probabilistic Language Tries: Beyond the Per-Vector Shannon Limit

KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation