Offline retrieval-augmented generation (RAG) on edge devices is updated through signed knowledge packs that carry text and precomputed embedding vectors. A signature proves who published a pack, not that its vectors encode its text. We study this text-vector integrity gap. A verifier that recomputes vectors and accepts them within a cosine tolerance concedes a predictable share of queries to an attacker who controls the vectors: the hijack rate equals the distribution of angular retrieval margins at the tolerance (the margin law). Margins are small in five corpora and two embedding models, so an idealised tolerance at the level of floating-point noise leaves up to 5.5% of queries exposed (at most 1.2% in any corpus once exact duplicate documents are excluded), and a threshold calibrated from our CPU INT8-FP32 comparison of a small embedding model (at the empirical 0.1% quantile of honest cosines) leaves 98-100% exposed; on a Raspberry Pi 5 running a local language model, such an attack lowers correct answers from 0.75 to 0.39. Integrity audits of compressed vector stores fare no better: on a FAISS product-quantisation index over 57,638 documents, tampered vectors that all pass the audit put the attacker's document first for 25% of targets. Verifying quantisation codes instead, with ambiguity decided by the verifier, a calibrated guard band and canonicalisation, accepted every honest vector in our samples on three platforms and left the publisher no choice. We also show that common floating-point and dynamic-INT8 inference pipelines are not bit-reproducible across batch sizes or architectures, and report pre-registered negative results on abstention, context pruning and semantic screening. We release code and data.
Figures & tables
Line of work
Representative
What it protects or shows
What it leaves open for our setting
Retrieval poisoning
[ 38 , 8 , 9 ]
Attacks and defences for malicious text in the corpus
Benign text with malicious vectors
Embedding provenance
[ 18 ]
Detects modification of a vector after it was signed
A compromised signer or writer
Supply-chain signing
[ 11 , 30 , 31 , 46 , 10 ]
Authenticity and history of artefacts
Consistency between text and vectors inside an artefact
Tolerance-based verifiable inference
[ 26 ]
Detects deviating LLM computations under float noise
What a tolerance concedes when outputs are retrieval vectors
Exact verifiable inference
[ 21 , 22 , 23 ]
Bit-identical results by pinning or integer arithmetic
Compressed stores and retrieval vectors
Vector quantisation
[ 28 , 44 , 29 , 45 ]
Memory- and speed-efficient search
Integrity audits under quantisation error
Table 1: Positioning of this work relative to the closest lines of research
Tolerance τ
Angle
BGE-M3 law
BGE-M3 transfer
MiniLM law
MiniLM transfer
1 (exact)
0 ∘
0.00%
0.00%
0.00%
0.00%
0.99999976 (FP32 x86 vs ARM)
0.04 ∘
0.49%
0.05%
0.24%
0.03%
0.99999
0.26 ∘
2.2%
0.5%
1.3%
0.15%
0.999
2.56 ∘
24.6%
4.5%
16.8%
2.3%
0.99
8.11 ∘
74.9%
21.7%
54.7%
10.2%
0.95
18.19 ∘
99.6%
62.2%
91.9%
30.8%
Table 2: Margin law on SQuAD 2.0: share of queries an attacker can hijack (rank 1) under a verifier with cosine tolerance τ ; BGE-M3 on passages (3,648 queries), MiniLM on 128-token windows (3,301 queries); 95% Wilson intervals are within ± 1.8 points. “Transfer”: attacker who knows the target passage and other questions about it, but not the exact wording of the query
Figure 1: Hijack rate as a function of the tolerated deviation, for SQuAD and four BEIR corpora (margin law, passage-level retrieval for both models) and the transfer attacker on SQuAD. Vertical lines mark FP32 cross-architecture noise, τ=0.999 and the MiniLM CPU-INT8 reference threshold
Storage
Bits/dim
Max. honest reconstruction angle
Hijackable queries
FP16
16
0.03 ∘
0–5.2%
SQ8
8
0.35–0.56 ∘
2.0–14.5%
SQ4
4
5.8–9.6 ∘
55–89%
PQ ( d/8 subvectors, 256 centroids)
1
23–40 ∘
99.7–100%
Binary (sign)
1
41–47 ∘
100%
Table 3: Exposure of tolerance-based audits against compressed vector stores (5 corpora, 2 models; ranges over corpus–model pairs)
FAISS index
Tolerance (largest honest angle)
Hijackable queries, margin law [95% CI]
IndexFlatIP (float)
0.00 ∘
0.0% [0.0, 1.6]
IndexScalarQuantizer (SQ8)
0.57 ∘
8.3% [5.4, 12.5]
IndexPQ ( d/8× 8 bits)
37.97 ∘
100% [98.4, 100]
IndexBinaryFlat
44.64 ∘
100% [98.4, 100]
Executed attack on IndexPQ (60 targets): 100% of tampered vectors pass the tolerance check;
carrier ranks first 25% [16, 37]; blocked by code-level verification with canonicalisation 100% [94, 100]
Table 4: Tolerance-based integrity checks on real FAISS indexes, BEIR FiQA (57,638 documents, 230 target queries with gold at rank 1). Tolerance: the largest angle between each index’s decoded vectors and the uncompressed ones. Hijackable: margin law on the cosine margins of the uncompressed vectors, i.e. the exposure of the audit, independent of the index’s search metric. The executed attack is scored with the index’s own inner products of the query with the decoded PQ vectors
Figure 2: (a) Share of queries hijackable under a tolerance-based audit of stored vectors, per storage format (bars: median and range over five corpora and two models; diamonds: FAISS indexes over FiQA). (b) The same exposure on SQuAD with BGE-M3 before (tolerance-based audit) and after the defence (code-level verification with a deliberately wide 10−5 guard band and without canonicalisation; at the calibrated widths the residual is at most 0.16%, and canonicalisation removes it). Panel (b) shows the three formats covered by the residual analysis; FP16 stores no quantisation codes, and binary codes were not part of that analysis. Logarithmic scale
Desktop: NVIDIA GPU ↔ x86 CPU
Apple M2: Metal GPU ↔ ARM CPU
x86 ↔ ARM (Pi)
Scheme
MiniLM
BGE-M3
MiniLM
BGE-M3
MiniLM windows
SQ8
98.99–99.03% / 3⋅10−7
91.3–91.5% / 3⋅10−6
99.12–99.13% / 3⋅10−7
91.0% / 3⋅10−6
99.26–99.43% / 10−7
SQ4
99.94% / 10−7 – 3⋅10−7
99.5% / 10−6 – 3⋅10−6
99.94–99.95% / 10−7
99.3–99.4% / 10−6 – 3⋅10−6
99.94–100% / ≤10−7
PQ
99.995–99.997% / 10−7
100% / none
99.995–99.997% / 10−7
100% / none
100% / none
Binary
99.991% / 10−7 – 3⋅10−7
99.94% / 10−7
99.991% / 10−7
99.94% / 10−7
100% / none
Max. abs. coordinate difference
4.3⋅10−7
1.0⋅10−5
3.6⋅10−7
5.6⋅10−6
1.9⋅10−7
Table 5: Code-level verification with the reference verifier (ambiguity decided by the verifier only; both directions). Each cell: share of honest vectors accepted with exact equality ( ε=0 , range over the two directions) / smallest tested guard width that accepted all vectors ( εSQ , εbin in value units; εPQ in squared-distance units). FiQA: MiniLM on 57,638 documents, BGE-M3 on a random sample (fixed seed) of 5,000; windows: 1,745 SQuAD windows, MiniLM in ONNX Runtime FP32 on x86 and on the Raspberry Pi’s ARM CPU
Encoder load
Effect on the language model
Saturating (re-indexing, 15–25 items/s)
decode −64.5% to −73.9%
Periodic verification, 8 pairs/s
decode −22.9% to −31.4% , p99 +108% to +167%
Table 6: Local LLM decoding on the Raspberry Pi 5 with an INT8 encoder (ONNX Runtime, 1–4 threads) running concurrently on the CPU: range over thread counts, three full runs (reference: 5.14 tokens/s, p99 inter-token latency 210 ms)
Strategy
Guarantee and residual exposure
Requirements
Cost
Signature only (V1)
Provenance of the bytes; no text–vector binding. A compromised signer controls retrieval.
Key management, transparency log
Negligible
Recompute and accept within τ (V2)
Exposure equals the margin law at arccosτ : 0–5.5% at FP32 noise, 17–25% at τ=0.999 (SQuAD), 98–100% at the MiniLM CPU-INT8 reference threshold
Same model and pre-processing as the publisher
One embedding per chunk
Recompute and discard
No exposure from shipped vectors: the device retrieves with its own vectors
A device-side embedder; the publisher’s numerics become irrelevant
Full re-embedding (72.9 s in FP32, 41.6 s in INT8 for 1,745 windows on the Raspberry Pi CPU)
Exact equality with a bit-reproducible computation (V3)
No exposure
A computation shown to be bit-reproducible at publisher and device (for example identical hardware and software, or integer-only inference)
One embedding per chunk
Code-level verification without canonicalisation (V4 without its last step)
At calibrated guard widths, 0% (MiniLM) to 0.16% (BGE-M3, SQ8)
Trusted codebook and embedding configuration; calibrated guard width per platform pair
Re-embedding plus quantisation
Code-level verification with canonicalisation (V4)
No residual exposure by construction; all honest vectors accepted in our samples
As above
As above
Table 7: Verification strategies for shipped or stored vectors: guarantee, residual exposure measured in this study, requirements and cost