Trust Mechanisms for Offline Generative AI at the Edge: A Connectivity-Regime Framework and an Empirical Study on a Raspberry Pi 5

Georgios Feretzakis; Alexandros Papaspyridis

cs.CROct 24, 2026

Trust Mechanisms for Offline Generative AI at the Edge: A Connectivity-Regime Framework and an Empirical Study on a Raspberry Pi 5

Authors: Georgios Feretzakis, Alexandros Papaspyridis

Abstract

Generative AI and large language models (LLMs) are moving into the Internet of Things, yet most deployments assume cloud connectivity, and many IoT settings have intermittent, restricted, or no connectivity. We propose a connectivity-regime framework (C0-C4), a taxonomy of inference-time trust mechanisms, and signed, versioned knowledge packs as the update channel for disconnected regimes, and evaluate them on a Raspberry Pi 5 running fully local retrieval-augmented generation under network isolation, with three Qwen2.5 model sizes, three context sizes, and 300 SQuAD 2.0 questions. An empirically calibrated retrieval-score gate detects knowledge gaps almost perfectly (AUROC 0.97) but in-topic unanswerable questions poorly (0.68). A deterministic output validator converts malformed outputs into explicit abstentions with no observed loss of correct answers, while the NLI-based verifier we tested adds little; for the 7B model, 96% of substantive answers to unanswerable-labelled questions match normalized spans of the retrieved passages. A 3B model achieves the same gold-answer containment rate as a 7B model at less than half the latency and estimated energy, although the 7B model far more often exceeds a token-F1 threshold of 0.5; five retrieved passages lower correctness from 75% to 52% compared with three. Verifying a signed pack takes about 4.5 ms versus about 29 min for embedding its passages on the device; signature and hash checks alone accept six of nine tested update attacks that layered verification rejects. Yet a validly signed poisoned pack passes every pack check: on 30 previously correct questions, correct answers drop to zero and 14 return the injected answer. None of the mechanisms we evaluated establishes the truth of the evidence. We release code and data.

Figures & tables

Regime	Definition	Inference	Learning / updates	Trust emphasis
C0: Cloud-connected	Persistent broadband to cloud	Cloud or hybrid	Continuous; provider-managed	Transit privacy; provider trust
C1: Edge-assisted	Cloud reachable; latency/privacy push compute to edge	Local with cloud fallback	Frequent synchronization; FL feasible	Split trust; fallback-path leakage
C2: Offline-capable	Operates offline; periodic synchronization	Fully local	Scheduled sync windows; FL rounds possible	Update integrity; staleness bounds
C3: Disconnected, controlled updates	No routine bidirectional connectivity; updates via controlled channels/media	Fully local	Signed artifacts over controlled channels, including approved one-way (unidirectional gateway) links	Supply-chain integrity; provenance
C4: Air-gapped	No physical connection to external networks; transfers only through controlled manual procedures	Fully local; all assets pre-provisioned	Controlled offline transfer or removable media; verified before activation	Covert channels [ 87 ] ; insider threat; auditability

Table 1: The C0–C4 connectivity-regime taxonomy for edge AI deployments and its architectural consequences

Figure 1: The C0–C4 connectivity continuum and its architectural consequences: as deployments move from cloud-connected (C0) to air-gapped (C4), gradient-based federation—i.e., federated learning (FL)—loses viability, knowledge-artifact distribution becomes the primary update and knowledge-sharing mechanism (supporting federation where multi-node curation is present, per Sect. 3.3 ), the dominant threat surface shifts from network exfiltration to supply-chain, insider, and covert-channel risks, and governance emphasis moves from transfer lawfulness to provenance and local auditability

Mechanism	Locus	Assurance character	Representative work
Prompt-level constraints	Generator input	Behavioral tendency only	prompting practice; [ 41 ]
Retrieval grounding	Evidence supply	Reduces, does not eliminate, hallucination	[ 12 , 50 ]
Citation grounding	Output format	Traceability of claims to sources	[ 58 ]
Output validation (deterministic)	Generator output	Generator-independent, rule-based; catches degenerate or malformed outputs, not unsupported content	This paper; Sect. 5.2
Self-verification / self-critique	Generator itself	Model-dependent; inherits model unreliability	[ 51 , 52 , 53 ]
Retrieval-score gating	Retrieval-side statistic	Generator-independent routing signal (embedding-model-dependent; calibratable)	[ 25 , 26 ] ; Sect. 5.1

Table 2: Inference-time trust mechanisms for generative language systems, organized by mechanism locus, dependence on the generator, and type of assurance provided

Federation unit	Raw data leaves node?	Human-auditable?	Connectivity need	Update size	Revocable?	Air-gap (C4) suitable?
Full model weights	No (but extraction risk [ 89 ] )	No	High bandwidth	GBs	Version rollback only	Marginal (offline transfer)
Gradients / deltas (FL) [ 60 ]	No (leakage risk [ 65 ] )	No	Recurring rounds	MBs–GBs	Effectively no	Operationally impractical
PEFT adapters (LoRA) [ 36 , 37 ]	No	No	Periodic for iterative exchange; none for one-way artifact transfer	MBs	Per-adapter	Possible via controlled transfer; iterative exchange impractical
Knowledge packs (docs + indices)	Not necessarily; depends on pack contents & curation	Partially (docs/metadata yes; embeddings no)	Any, incl. controlled offline transfer	Content-proportional	Per-pack	Yes
Knowledge-graph updates [ 103 ]	Not necessarily; depends on graph contents & curation	Partially	Any	Small–medium	Per-update	Yes
Prompt/policy bundles	Not necessarily; depends on bundle contents & curation	Yes	Any	KBs	Per-bundle	Yes

Table 3: Federation units for collaborative edge intelligence: comparative properties. Entries are the authors’ qualitative synthesis; cited works substantiate the federation or retrieval mechanism itself rather than every listed property

Figure 2: A cross-layer map of trustworthy generative edge intelligence: infrastructure and inference mechanisms (bottom) support the knowledge and federation layer, whose artifacts feed inference-time trust mechanisms, all of which generate the evidence consumed by the governance layer (top). Mechanism applicability varies with the C0–C4 connectivity regime (right). GDPR: General Data Protection Regulation; CRA: Cyber Resilience Act; NIST AI RMF: NIST Artificial Intelligence Risk Management Framework; GenAI: generative artificial intelligence

Configuration	Correct	Empty	Unanswerable abstained		Risk among answered			Median	Energy
	len./strict		M	M+V+G	M	M+V	M+V+G	latency	per query
Qwen2.5-1.5B, $k=3$	0.27/0.17	0.30	0.47	0.89	0.81	0.46	0.43	10 s	77 J
Qwen2.5-3B, $k=1$	0.45/0.19	0.00	0.80	0.84	0.44	0.44	0.42	8 s	62 J
Qwen2.5-3B, $k=3$	0.75/0.31	$<$ 0.01	0.66	0.72	0.39	0.38	0.37	22 s	174 J
Qwen2.5-3B, $k=5$	0.52/0.20	$<$ 0.01	0.68	0.75	0.49	0.49	0.44	36 s	286 J
Qwen2.5-7B, $k=3$	0.75/0.74	0.04	0.71	0.84	0.36	0.29	0.24	50 s	399 J

Table 4: Proof-of-concept results on a Raspberry Pi 5 under network isolation approximating C4, with 100 queries per condition. Correct: share of the 100 answerable questions answered correctly (lenient/strict). Empty: share of all 300 queries answered with citation markers but no answer text. M: self-abstention; M+V: with output validator; M+V+G: with validator and calibrated score gate (mean split-conformal threshold 0.50,

\alpha=0.1

). Latency and energy are end-to-end estimates (Sect. 4 ). Risk: fraction of answered queries not answered correctly

Figure 3: Retrieval-score gating on the edge device (100 queries per type). (a) Distribution of top-1 cosine scores per query type; the dashed line marks the mean calibrated threshold

\hat{\tau}

. (b) ROC curves of the gate for separating answerable queries from out-of-corpus and from in-topic unanswerable queries, with AUROC and 95% bootstrap intervals

Configuration	AUROC [95% CI]	Unanswerable abstained	Correct	Risk	Span present	Verify time
		M+V+G $\rightarrow$ +E	M+V+G $\rightarrow$ +E	M+V+G $\rightarrow$ +E	correct / unans.	(median)
Qwen2.5-1.5B, $k=3$	0.50 [0.34, 0.66]	0.89 $\rightarrow$ 0.89	0.24 $\rightarrow$ 0.20	0.43 $\rightarrow$ 0.44	0.56 / 0.64	2.2 s
Qwen2.5-3B, $k=1$	0.60 [0.47, 0.72]	0.84 $\rightarrow$ 0.87	0.42 $\rightarrow$ 0.39	0.42 $\rightarrow$ 0.40	0.36 / 0.55	0.8 s
Qwen2.5-3B, $k=3$	0.64 [0.54, 0.73]	0.72 $\rightarrow$ 0.77	0.67 $\rightarrow$ 0.62	0.37 $\rightarrow$ 0.35	0.43 / 0.47	2.4 s
Qwen2.5-3B, $k=5$	0.55 [0.44, 0.66]	0.75 $\rightarrow$ 0.77	0.50 $\rightarrow$ 0.47	0.44 $\rightarrow$ 0.41	0.42 / 0.59	4.4 s
Qwen2.5-7B, $k=3$	0.55 [0.43, 0.66]	0.84 $\rightarrow$ 0.87	0.68 $\rightarrow$ 0.61	0.24 $\rightarrow$ 0.24	0.96 / 0.96	2.1 s

Table 5: Post-hoc entailment verification (E) added to self-abstention, output validator, and score gate (M+V+G). AUROC: separation of correct answers (containing a gold answer) from incorrect ones by maximum entailment probability; “incorrect” refers to gold-answer correctness, not to evidence non-entailment. Span present: share of answers matching a normalized span of a retrieved passage, for correct answers and for answers given to in-topic unanswerable questions

Figure 4: Layered trust mechanisms and the quality–cost design space. (a) Risk among answered queries for three model sizes (

k=3

) under self-abstention, with an output validator, and with validator and score gate; the numbers above the bars give the fraction of answerable queries answered correctly. (b) Fraction of answerable queries answered correctly against median energy per query for all five configurations; labels give median latency

Pack presented to the device	Signature + hash	Layered	Reason for rejection (layered)
Valid newer pack	accept	accept	–
Older pack (rollback)	accept	reject	sequence not newer
Current version re-issued (replay)	accept	reject	sequence not newer
Expired pack	accept	reject	expired
Built for a different embedding model	accept	reject	incompatible embedding model
Signed with a revoked key	accept	reject	declared key revoked

Table 6: Update-security tests and validly signed poisoning on the Raspberry Pi 5. Signature + hash: signature by any trusted key plus file hashes. Layered: signature verified only with the declared, non-revoked key, plus file hashes, strictly increasing sequence number, expiry, and embedding-model compatibility. Median verification time for the valid pack: 4.53 ms (signature + hash) and 4.55 ms (layered); results from the corrected layered verifier

LLM Reliability

Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.

See all 789 papers

Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems
Aug 21, 2026Balkrishna Giri, Md Toufique Hasan, Jussi Rasku +2RAG Security RAG Evaluation
Faculty of Information Technology and Communication Sciences, Tampere University Tampere, Finland
Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs
Feb 19, 2026Arka Pal, Louai Zahran, William Gvozdjak +2Verifiable Outsourced Computation LLM Inference
Ritual · MIT · Columbia University
LatentAudit: Real-Time White-Box Faithfulness Monitoring for Retrieval-Augmented Generation with Verifiable Deployment
Apr 7, 2026Zhe Yu, Wenpeng Xing, Meng HanLLM Auditing RAG Evaluation
Binjiang Institute of Zhejiang University · Communication University of Zhejiang · Zhejiang University

Aug 21, 2026J/K move · Enter open · S save

Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems

Balkrishna Giri, Md Toufique Hasan, Jussi Rasku, Muhammad Waseem+1

Faculty of Information Technology and Communication Sciences, Tampere University Tampere, Finland

RAG Security RAG Evaluation RAG Poisoning Attacks Prompt Injection Attacks on LLMs

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) grounds Large Language Model (LLM) outputs in external knowledge, but RAG systems usually trust whatever they retrieve, creating a Security-Reliability Gap: high semantic relevance does not guarantee factual truth. Adversaries exploit this through knowledge poisoning, inserting malicious documents to cause targeted misinformation. We propose an Evaluation Agent, middleware that combines Natural Language Inference (NLI) factual verification, a five-signal poison detector with relevance-weighted aggregation, and a Trust Index T = 0.4 F + 0.35 C + 0.25 (1 - P ) with a non-linear dampener for high-contamination contexts. On TruthfulQA with Llama 3.3 70B, the agent reaches 91% accuracy and 100% precision, with 100% recall on instruction injection, while in-place edits, such as entity swaps, remain hard to detect. Across three LLMs the Trust Index stays discriminative, with a Receiver Operating Characteristic Area Under the Curve (ROC-AUC) of 0.73 to 0.81; generation style matters more than model size, and per-LLM threshold calibration restores baseline competitive accuracy, whereas a weaker FEVER result shows that cross-dataset generalization requires domain-specific calibration. In a software-engineering use case, a secure-coding assistant over guidance from the Open Worldwide Application Security Project (OWASP) Top 10 and the Common Weakness Enumeration (CWE), the agent reliably blocks instruction injection of unsafe advice (F1 92%), while contradiction and subtle semantic weakening remain hard. Throughout, the agent measures detection of poisoned context before generation, not whether the LLM adopts the injected misinformation. We release the proposed approach, attack generator, and experimental artifacts at the link: https://github.com/GPT-Laboratory/TrustworthyRAG.

Abstract

LLM Reliability

Explore similar work

Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems

Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs

LatentAudit: Real-Time White-Box Faithfulness Monitoring for Retrieval-Augmented Generation with Verifiable Deployment