Supercharging Federated Intelligence Retrieval

Dimitris Stripelis; Patrick Foley; Mohammad Naseri; William Lindskog-Münzing; Chong Shen Ng; Daniel Janes Beutel; Nicholas D. Lane

cs.IRMar 26, 2026

Supercharging Federated Intelligence Retrieval

Authors: Dimitris Stripelis, Patrick Foley, Mohammad Naseri, William Lindskog-Münzing, Chong Shen Ng, Daniel Janes Beutel, Nicholas D. Lane

Organizations: Flower Labs

Abstract

RAG typically assumes centralized access to documents, which breaks down when knowledge is distributed across private data silos. We propose a secure Federated RAG system built using Flower that performs local silo retrieval, while server-side aggregation and text generation run inside an attested, confidential compute environment, enabling confidential remote LLM inference even in the presence of honest-but-curious or compromised servers. We also propose a cascading inference approach that incorporates a non-confidential third-party model (e.g., Amazon Nova) as auxiliary context without weakening confidentiality.

Figures & tables

Figure 1: FedRAG with confidential server-side aggregation and various inference options.

Panel of Table 1 — Table 1: Mean query execution time and accuracy per benchmark dataset for each inference mode.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Corpus	Size	#Snippets
PubMed	70GB	23.9M
StatPearls	2GB	301k
Textbooks	209MB	126k
Wikipedia	44GB	29.9M

Appendix

Table 2: Data Corpus Specifications.

Privacy-Preserving RAG

See all 13 papers

FD-RAG: Federated Dual-System Retrieval-Augmented Generation
May 22, 2026Tianhao Gao, Kai Yang, Yiyang LiRetrieval-Augmented Generation Memory-Augmented Language Models
School of Computer Science and Technology, Tongji University
Beyond Raw Context Transfer: Representation-based Federated Retrieval-Augmented Generation
Aug 26, 2026Can Peng, Yu Liu, Yingyu Yang +4Retrieval-Augmented Generation Federated Learning
University of Oxford, Oxford, United Kingdom · National University of Singapore, Singapore
P $^2$ RAG: Efficient Privacy-Preserving RAG Service Supporting Arbitrary Top- $k$ Retrieval
Mar 16, 2026Yulong Ming, Mingyue Wang, Jijia Yang +4Privacy-Preserving RAG RAG Security
City University of Hong Kong · Peng Cheng Laboratory

May 22, 2026J/K move · Enter open · S save

FD-RAG: Federated Dual-System Retrieval-Augmented Generation

Tianhao Gao, Kai Yang, Yiyang Li

School of Computer Science and Technology, Tongji University

Retrieval-Augmented Generation Memory-Augmented Language Models LLM Inference Acceleration Edge ML Hypergraph Learning Federated Learning

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) has emerged as a paradigm for grounding large language models in external knowledge, yet most existing RAG systems assume centralized knowledge access and ample computation. These assumptions break down in edge environments, where knowledge is fragmented across devices, raw data cannot be shared, and repeated LLM calls are prohibitively expensive. We propose FD-RAG, a federated dual-system RAG framework that decouples lightweight memory access from on-demand LLM reasoning for decentralized deployment. Specifically, FD-RAG learns semantic-aware adaptive hypergraphs over local corpora and distills them into compact QA memories. At inference time, it answers well-covered queries via direct memory matching and invokes LLM-based reasoning only when necessary, while tracing retrieved memories to hypergraph-grounded evidence. To mitigate cross-device knowledge fragmentation, FD-RAG aggregates anonymized memories across devices without exposing raw documents. Experiments on QA benchmarks show that FD-RAG improves accuracy by up to 7.8% while reducing latency by 8.4

\times

compared with strong local and federated baselines. We also provide theoretical analysis establishing an

\mathcal{O}(1/ε^{2})

convergence rate for the proposed hypergraph learning, supporting its tractable deployment in edge settings.

Abstract

Privacy-Preserving RAG

Explore similar work

FD-RAG: Federated Dual-System Retrieval-Augmented Generation

Beyond Raw Context Transfer: Representation-based Federated Retrieval-Augmented Generation

P2^22RAG: Efficient Privacy-Preserving RAG Service Supporting Arbitrary Top-kkk Retrieval

P $^2$ RAG: Efficient Privacy-Preserving RAG Service Supporting Arbitrary Top- $k$ Retrieval