HCC+: Hyperbolic Guarding for Certified Attention Retrieval

Liangchen Ge

cs.DSAug 25, 2026

HCC+: Hyperbolic Guarding for Certified Attention Retrieval

Authors: Liangchen Ge

Abstract

We study the Lipschitz stability of attention retrieval in hyperbolic spaces. Existing methods lack deterministic guarantees on attention-weight preservation under finite-precision representations. We introduce HCC+, a theoretical framework exploiting three properties of the Poincaré ball: exponential volume growth enabling query-independent boundary truncation; logarithmic covering radius of hyperbolic 1-centers enabling dimension-independent critical-key identification; and a packing bound with constants independent of the embedding dimension. We prove two deterministic guarantees: for exact retrieval, the per-layer attention deviation is bounded by 10% of its ideal value; for soft attention, the total variation distance decays as $O(1/\sqrt{n})$ , the rate of finite-sample variance. As a consequence of the guarding mechanism, the framework achieves a storage reduction factor of $6.1\times$ relative to FP16. We provide the first deterministic, query-independent retrieval certificate in non-Euclidean geometry.

Figures & tables

Component	Factor	Cumulative
FP16 baseline	1.0 $\times$	–
Boundary truncation	1.18 $\times$	1.18 $\times$
CKG + HPQ (keys)	7.2 $\times$	8.5 $\times$
SAVC (values)	2.13 $\times$	18.1 $\times$
ECPP (entropy coding)	1.25 $\times$	22.6 $\times$

Table 1: Theoretical storage attribution (FP16 baseline).

Attention Mechanisms

Attention mechanisms let neural networks weight information from other tokens or regions according to relevance, enabling context-dependent processing.

See all 517 papers

Riemann GeoResolver: A Non-Euclidean Attention Framework from Euclidean Resolver to Hyperbolic-Spherical Geometry
Aug 11, 2026Liangchen GeAttention Mechanisms
A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets
Jul 2, 2026Wanyun CuiHybrid Attention Long-Context Retrieval
Shanghai University of Finance and Economics
Certified Sparse Attention via Perturbed Retrieval
Oct 18, 2026Runguo LiSparse Attention LLM Inference Acceleration
University of Illinois Urbana-Champaign

Aug 11, 2026J/K move · Enter open · S save

Riemann GeoResolver: A Non-Euclidean Attention Framework from Euclidean Resolver to Hyperbolic-Spherical Geometry

Liangchen Ge

Attention Mechanisms

Open paper AskPDF arXiv

We present a theoretical foundation for inverse-distance attention, from its Euclidean prototype (Resolver) to its non-Euclidean realization (Riemann GeoResolver). The Euclidean part establishes three core theorems: (1) circuit separation---IDA achieves exact retrieval with

\mathcal{O}(1)

resources while softmax requires

Ω((\log n)^2)

width; (2) a Polyak--Lojasiewicz inequality with

Ω(e^{Δ^2/\sqrt{d}}/Δ^2)

stronger constant than softmax, implying linear convergence,

\mathcal{O}(\log n)

Lipschitz scaling under a low-rank/clustering assumption,

Θ(1)

Hessian spread, and absence of spurious local minima; (3) a width-independent effective rank bound that limits noise memorization---softmax memorizes arbitrary labels when

d_h\ge n

, while IDA limits test error to

\mathcal{O}(η^2)

. The non-Euclidean extension then builds upon this prototype, replacing Euclidean distance with hyperbolic geodesic distance for storage and spherical geodesic distance for routing. The Riemann GeoResolver framework comprises ten integrated modules: four HIDA operators spanning

Θ(n^2)

Θ(1)

per token; Hyperbolic Curvature Compression (HCC) with provable error bounds; HyperGate with gradient lower-bound theorem; Spherical Inverse Distance Attention (SIDA) with sphere-analog PL inequalities; Dynamic Memory Genesis (DMG) with

\mathcal{O}(\log T)

regret bounds; and Geodesic Sparse Routing (GSR) with quality and communication bounds. The Euclidean theorems are proved in full; the non-Euclidean extension theorems are proved with analogous arguments. This work establishes a theoretical arc: from Euclidean attention as a special case, to hyperbolic memory, to spherical retrieval.

Abstract

Attention Mechanisms

Explore similar work

Riemann GeoResolver: A Non-Euclidean Attention Framework from Euclidean Resolver to Hyperbolic-Spherical Geometry

A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets

Certified Sparse Attention via Perturbed Retrieval