Nonreciprocal Spherical Attention and Retrieval: Sharp Coarsening Clocks and Certified Score Correction
Authors: Faruk Alpay, Taylan Alpay
Organizations: Department of Computer Engineering, Bahçeşehir University, Istanbul, Türkiye · Department of Aerospace, University of Turkish Aeronautical Association, Ankara, Türkiye
We develop nonreciprocal spherical attention as a mathematical framework for coarsening and compressed nearest-neighbor retrieval. For two shrinking acute blocks whose centers have zero skew pairing and whose total mass ratio is subexponential in inverse temperature, we prove limβ→∞β−1logTβ=1−cosφ, without restricting particle-count growth. An explicit three-body flow has two rigorously separated metastable exponents. For finite compression, mass centering yields a sharp O(r2) attention-output error bound at fixed temperature, with an explicit spherical-normalization correction. We formulate skew query-side score calibration as a strictly convex moment problem and derive a unique normal equation, residual-based optimization certificates, and bounded low-rank projections. A norm-aware margin objective calibrates native squared-distance rankings. Experiments use semantic retrieval data, one million visual descriptors with three codebook seeds, and a ten-million-vector FAISS IVF-PQ index with recomputed exact cosine ground truth. On SIFT1M, skew margin calibration raises native-L2 hit@10 from 0.8926 to 0.8948; symmetric calibration reaches 0.8996. The ten-million-vector index reaches 0.8827 hit@10 at 0.1794 ms per query in a four-thread batch benchmark, while validation selects no skew update. Exact streamed attention over ten million keys evaluates compressed memory on 1,024 held-out queries. Independent matrix solves, automatic differentiation, finite differences, interval arithmetic, full-flow tests, and nine Rocq-checked algebra lemmas support reproducibility. Directionality alone does not establish retrieval superiority.
Figures & tables
Figure 1: Our geometric notation: caps on the unit circle, with directed interactions and unequal barriers. The cap construction is visually motivated by Figure 1 of [ 3 ] ; this drawing is original.
Figure 2: The skew interaction measures the directed signed area of a triangle. The figure represents a two-dimensional linear plane; the vertices may also be constrained to a unit circle.
Figure 3: Exact three-dimensional point coordinates from ( 31 ), shown in an orthographic projection. The blue arc is the shielded great-circle corridor; the dashed line projects u2 onto its plane.
SciFact
NFCorpus
Method
nDCG@10
R@100
nDCG@10
R@100
PQ64 × 64
0.6124
0.9159
0.3046
0.2925
PQ64 × 48, no adapter
0.6127
0.9131
0.3019
0.2958
PQ48 + bounded skew
0.6210
0.9203
0.3049
0.3000
PQ48 + bounded symmetric
0.6165
0.9196
0.3053
0.2984
PQ48 + bounded general
0.6222
0.9140
0.3057
0.2989
Table 1: Fixed-byte retrieval comparison. Values are means over three seeds. PQ48 alone uses fewer bytes; all three adapters and PQ64 have identical payload sizes within each dataset.
Prototype budget K
Noise
Neighbor
Method
128
256
512
amplification
Jaccard
Spherical clustering
0.0774
0.1350
0.1954
1.000
0.990
Reciprocal flow
0.0866
0.1338
0.1980
0.812
0.991
Directed flow
0.0813
0.1333
0.1950
0.811
0.991
Coarse PQ8 reference
0.3713 (no prototype budget)
—
—
Table 2: Coarsening ablation on SciFact: nDCG@10 by prototype budget and local perturbation stability.
Weights
γ
β
Nβ
Tβ
log(βTβ)/β
Uniform
0.15
8
4
2.3634
0.36744
Uniform
0.15
12
7
4.4670
0.33180
Uniform
0.15
16
12
10.4881
0.32018
Uniform
0.15
20
21
26.3024
0.31327
Uniform
0.15
24
37
68.7424
0.30868
Uniform
0.25
8
8
2.3940
0.36905
Table 3: Full-flow joint scaling, G=1−cos(0.8)=0.30329329 . Total block masses are balanced. Extreme intrablock weights have logarithmic range 0.2β2 .
Method
Hit@10
Hit@100
Set R@10
MiB
ms/query
PQ
0.4189
0.7752
0.2258
15.384
0.729
PQ + skew
0.4221
0.7814
0.2283
15.392
0.730
PQ + symmetric
0.4246
0.7843
0.2302
15.392
0.729
PQ + general
0.4257
0.7838
0.2293
15.399
0.729
OPQ
0.3477
0.7215
0.1895
15.446
0.741
OPQ + skew
0.3558
0.7303
0.1935
15.454
0.737
Table 4: Normalized SIFT1M: exact-nearest-neighbor hit@10/100, top-10 set recall, persistent serialized index plus correction size, and measured query time. Means over three codebook seeds.
Method
Hit@10
Hit@100
MiB
ms/query
PQ-L2
0.8926
0.9981
15.384
0.731
PQ-L2 + score skew
0.8905
0.9974
15.392
0.737
OPQ-L2
0.8357
0.9941
15.446
0.758
OPQ-L2 + score skew
0.8288
0.9928
15.454
0.753
PQ-L2 + margin skew
0.8948
0.9981
15.392
0.745
PQ-L2 + margin symmetric
0.8996
0.9986
15.392
0.747
Table 5: Strong native-L2 controls and calibrated hard-negative margins on SIFT1M. Query correction is included in timing; all values average three codebook seeds.
Rank
Hit@10
Hit@100
Correction bytes
Distortion objective
2
0.8913
0.9978
1024
2.765e-04
4
0.8910
0.9978
2048
2.755e-04
8
0.8913
0.9978
4096
2.753e-04
16
0.8912
0.9978
8192
2.752e-04
32
0.8917
0.9978
16384
2.751e-04
Table 6: Rank sensitivity of the score-distortion skew correction for native PQ-L2, seed 0. The ridge penalty is fixed to its validation-selected value.
Probes
Approx. scanned vectors
Hit@10
Hit@100
ms/query
4
9,766
0.6722
0.7080
0.0233
16
39,062
0.8402
0.9202
0.0568
64
156,250
0.8827
0.9862
0.1794
256
625,000
0.8862
0.9963
0.6159
Table 7: BIGANN10M IVF-PQ search: 6,000 held-out queries. Median of three timed batch searches, four CPU threads.
β
Mass-centered
Normalized centers
Trained coarse centers
1
0.003735
0.095486
0.003869
8
0.028480
0.096562
0.026600
32
0.095027
0.117941
0.091391
Table 8: Ten-million-key attention memory: mean Euclidean output error against exact streamed attention. All three methods retain 4,096 centers and the same cell masses.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: The entire corridor , rather than the initial ordering alone, certifies the first event. The external barrier stays strictly above the favorable internal barrier’s initial maximum.
Figure 5: A nontrivial periodic orbit in the nonreciprocal three-body flow. Arrows indicate the direction for one sign of ε ; reversing ε reverses rotation.
β
Qβ (full ODE)
Qβred (quadrature)
relative gap
20
1.173906
1.167887
5.15×10−3
25
1.103809
1.102014
1.63×10−3
30
1.066502
1.065996
4.75×10−4
35
1.044172
1.044034
1.32×10−4
40
1.029349
1.029312
3.60×10−5
50
1.010709
1.010707
3.0×10−6
Appendix
Table 9: The full three-body numerical diagnostic compared to the exact two-body quotient quadrature; neither finite table implies equality of the asymptotic limits.
Figure 6: The boundary layer produces an order-one skew-weight correction. The numerically calculated exact quotient tends toward its proved limit, while finite- β full dynamics closely follow the reduced clock. A proof of uniform full-flow matching remains open.
β
first event t1
second event t2
t2/t1
12
2.0294
4.6172
2.28
16
1.9790
10.2422
5.18
20
1.9727
25.2915
12.82
25
2.0134
86.6800
43.05
30
2.1017
317.9790
151.29
35
2.2405
1215.7801
542.65
Appendix
Table 10: Numerical threshold events for ( 31 ); these data are not a theorem about the second-event asymptotic.
Figure 7: Two distinct numerical clocks. The first clock is covered by theorem D.2 ; the rapidly growing second clock still lacks a rigorous general geometric selection theorem.
Figure 8: Normal control and tangential clock control are different. Two configurations can lie exactly on the collision stratum yet have different leading hitting-time constants because their quotient phases differ by O(β−1) . Schematic, not a numerical trajectory.
Figure 9: The favorable isolated inner barrier becomes negative in the post-threshold interval (shaded). Its rate is no longer exponentially slow there. The horizontal shield is a geometric bound from the initial corridor , not a proved uniform shield until the second event.
Spectral compression reduces the storage and scoring costs of dense retrieval by projecting embeddings onto a smaller principal subspace. However, its corpus-derived weighting rules tie all retained directions to a shared spectral trend, without directly accounting for their individual contributions to relevance. To address this, we propose Relevance-Guided Residual Calibration (RRC), a closed-form method for refining spectral weights using relevance judgments. RRC estimates margin-based weights from the means and variances of positive--negative score contributions along each retained direction. It retains directions with positive estimated weights, decomposes their log weights into a fitted power-law trend and a residual, and adds the residual to the compressor's log weights on this support. This allows relevance judgments to adjust individual directions while preserving the compressor's overall spectral trend. RRC applies to PCA, Whitening, and SpecTemp, and incorporates the corrections into the existing transformation shared by documents and queries, without iterative model training or an additional inference stage. Experiments with three encoders on nine datasets covering conventional and reasoning-intensive retrieval show that SpecTemp + RRC achieves the highest mean effectiveness among tested compressors on the BEIR group and BRIGHT at every target dimension. On the five evaluated BRIGHT datasets, it improves over SpecTemp by an average of 2.34 nDCG@10 points at 768 dimensions. The gains also persist under scalar and product quantization, improving the trade-off between per-document storage and retrieval effectiveness. Our code is available at: https://anonymous.4open.science/r/rrc-spectral-compression-B5CE/
Dense retrieval compresses a passage into a single vector, but this compression is positionally skewed: early content dominates the embedding, and retrieval degrades when the relevant span appears later. Prior work proposed an inference-time method that counteracts this skew by equalizing the pooling token's attention across passage segments. However, (i) it redistributes attention at a fixed strength, (ii) it forces the pooling token's attention to itself to a fixed basket-level mass despite substantial variation across layers and architectures, and (iii) its effect on retrieval has not been evaluated. We introduce a strength coefficient that interpolates between uncalibrated and fully equalized attention, together with an efficient implementation that reduces peak calibration memory overhead from 5-7 GiB to under 1 MiB. Across three embedding models and two pooling schemes, moderate calibration provides a better retrieval trade-off than full equalization. We introduce a variant that preserves the pooling token's self-attention mass and redistributes only the remaining mass. On a position-aware retrieval benchmark spanning 10 languages and 31 domains, a configuration selected on English FineWeb-PosQ and transferred without tuning reduces position sensitivity in all 16 evaluated length-quartile, model, and retrieval-setting combinations, by up to 43% relative, while improving nDCG@10 by up to 4.8% relative and leaving general retrieval effectiveness on NanoBEIR essentially unchanged. Calibration runs at indexing time, adding no query-time latency. We release our code at github.com/impresso/fair-sentence-transformers
Andrianos Michail, Elias Schuhmacher, Juri Opitz +2
We consider the problem of estimating the Attention mechanism in small space, and prove the existence of coresets for it of nearly optimal size. Specifically, we show that for any set of unit-norm keys and values (K,V) in Rd, there exists a subset (K′,V′) of size at most O(deρ+o(ρ)/ε) such that
∥Attn(q,K,V)−Attn(q,K′,V′)∥≤ε
simultaneously for all queries whose norm is bounded by ρ. This outperforms the best known results for this problem. We also offer an improved lower bound showing that ε-coresets must have size Ω(deρ/ε).