cs.LGNov 1, 2026

Nonreciprocal Spherical Attention and Retrieval: Sharp Coarsening Clocks and Certified Score Correction

Authors: Faruk Alpay, Taylan Alpay

Organizations: Department of Computer Engineering, Bahçeşehir University, Istanbul, Türkiye · Department of Aerospace, University of Turkish Aeronautical Association, Ankara, Türkiye

Abstract

We develop nonreciprocal spherical attention as a mathematical framework for coarsening and compressed nearest-neighbor retrieval. For two shrinking acute blocks whose centers have zero skew pairing and whose total mass ratio is subexponential in inverse temperature, we prove lim⁡β→∞β−1log⁡Tβ=1−cos⁡φ\lim_{β\to\infty}β^{-1}\log T_β=1-\cosφ, without restricting particle-count growth. An explicit three-body flow has two rigorously separated metastable exponents. For finite compression, mass centering yields a sharp O(r2)O(r^2) attention-output error bound at fixed temperature, with an explicit spherical-normalization correction. We formulate skew query-side score calibration as a strictly convex moment problem and derive a unique normal equation, residual-based optimization certificates, and bounded low-rank projections. A norm-aware margin objective calibrates native squared-distance rankings. Experiments use semantic retrieval data, one million visual descriptors with three codebook seeds, and a ten-million-vector FAISS IVF-PQ index with recomputed exact cosine ground truth. On SIFT1M, skew margin calibration raises native-L2 hit@10 from 0.8926 to 0.8948; symmetric calibration reaches 0.8996. The ten-million-vector index reaches 0.8827 hit@10 at 0.1794 ms per query in a four-thread batch benchmark, while validation selects no skew update. Exact streamed attention over ten million keys evaluates compressed memory on 1,024 held-out queries. Independent matrix solves, automatic differentiation, finite differences, interval arithmetic, full-flow tests, and nine Rocq-checked algebra lemmas support reproducibility. Directionality alone does not establish retrieval superiority.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Oct 26, 2026

Relevance-Guided Residual Calibration for Spectral Compression of Dense Retrieval Embeddings

Spectral compression reduces the storage and scoring costs of dense retrieval by projecting embeddings onto a smaller principal subspace. However, its corpus-derived weighting rules tie all retained directions to a shared spectral trend, without directly accounting for their individual contributions to relevance. To address this, we propose Relevance-Guided Residual Calibration (RRC), a closed-form method for refining spectral weights using relevance judgments. RRC estimates margin-based weights from the means and variances of positive--negative score contributions along each retained direction. It retains directions with positive estimated weights, decomposes their log weights into a fitted power-law trend and a residual, and adds the residual to the compressor's log weights on this support. This allows relevance judgments to adjust individual directions while preserving the compressor's overall spectral trend. RRC applies to PCA, Whitening, and SpecTemp, and incorporates the corrections into the existing transformation shared by documents and queries, without iterative model training or an additional inference stage. Experiments with three encoders on nine datasets covering conventional and reasoning-intensive retrieval show that SpecTemp + RRC achieves the highest mean effectiveness among tested compressors on the BEIR group and BRIGHT at every target dimension. On the five evaluated BRIGHT datasets, it improves over SpecTemp by an average of 2.34 nDCG@10 points at 768 dimensions. The gains also persist under scalar and product quantization, improving the trade-off between per-document storage and retrieval effectiveness. Our code is available at: https://anonymous.4open.science/r/rrc-spectral-compression-B5CE/
Jun 1, 2026

Attention Calibration for Position-Fair Dense Retrieval

Dense retrieval compresses a passage into a single vector, but this compression is positionally skewed: early content dominates the embedding, and retrieval degrades when the relevant span appears later. Prior work proposed an inference-time method that counteracts this skew by equalizing the pooling token's attention across passage segments. However, (i) it redistributes attention at a fixed strength, (ii) it forces the pooling token's attention to itself to a fixed basket-level mass despite substantial variation across layers and architectures, and (iii) its effect on retrieval has not been evaluated. We introduce a strength coefficient that interpolates between uncalibrated and fully equalized attention, together with an efficient implementation that reduces peak calibration memory overhead from 5-7 GiB to under 1 MiB. Across three embedding models and two pooling schemes, moderate calibration provides a better retrieval trade-off than full equalization. We introduce a variant that preserves the pooling token's self-attention mass and redistributes only the remaining mass. On a position-aware retrieval benchmark spanning 10 languages and 31 domains, a configuration selected on English FineWeb-PosQ and transferred without tuning reduces position sensitivity in all 16 evaluated length-quartile, model, and retrieval-setting combinations, by up to 43% relative, while improving nDCG@10 by up to 4.8% relative and leaving general retrieval effectiveness on NanoBEIR essentially unchanged. Calibration runs at indexing time, adding no query-time latency. We release our code at github.com/impresso/fair-sentence-transformers
May 7, 2026

Nearly Optimal Attention Coresets

We consider the problem of estimating the Attention mechanism in small space, and prove the existence of coresets for it of nearly optimal size. Specifically, we show that for any set of unit-norm keys and values (K,V)(K,V) in Rd\mathbb{R}^d, there exists a subset (K′,V′)(K',V') of size at most O(deρ+o(ρ)/ε)O({\sqrt{d} e^{ρ+o(ρ)}/\varepsilon}) such that ∥Attn⁡(q,K,V)−Attn⁡(q,K′,V′)∥≤ε\left\| \operatorname{Attn}(q,K,V)- \operatorname{Attn}(q,K',V') \right\| \le \varepsilon simultaneously for all queries whose norm is bounded by ρρ. This outperforms the best known results for this problem. We also offer an improved lower bound showing that ε\varepsilon-coresets must have size Ω(deρ/ε)Ω({\sqrt{d} e^ρ/ε}).