Privacy Guard & Token Parsimony by Prompt and Context Handling and LLM Routing

Alessio Langiu

cs.CRMar 30, 2026

Privacy Guard & Token Parsimony by Prompt and Context Handling and LLM Routing

Authors: Alessio Langiu

Organizations: National Research Council of Italy - Institute of Marine Sciences (CNR-ISMAR)

Abstract

The large-scale adoption of Large Language Models (LLMs) forces a trade-off between operational cost (OpEx) and data privacy. Current routing frameworks reduce costs but ignore prompt sensitivity, exposing users and institutions to leakage risks towards third-party cloud providers. We formalise the "Inseparability Paradigm": advanced context management intrinsically coincides with privacy management. We propose a local "Privacy Guard" -- a holistic contextual observer powered by an on-premise Small Language Model (SLM) -- that performs abstractive summarisation and Automatic Prompt Optimisation (APO) to decompose prompts into focused sub-tasks, re-routing high-risk queries to Zero-Trust or NDA-covered models. This dual mechanism simultaneously eliminates sensitive inference vectors (Zero Leakage) and reduces cloud token payloads (OpEx Reduction). A LIFO-based context compacting mechanism further bounds working memory, limiting the emergent leakage surface. We validate the framework through a 2x2 benchmark (Lazy vs. Expert users; Personal vs. Institutional secrets) on a 1,000-sample dataset, achieving a 45% blended OpEx reduction, 100% redaction success on personal secrets, and -- via LLM-as-a-Judge evaluation -- an 85% preference rate for APO-compressed responses over raw baselines. Our results demonstrate that Token Parsimony and Zero Leakage are mathematically dual projections of the same contextual compression operator.

Figures & tables

Figure 1: The Dual-Vault Architecture. A local SLM acts as an Orchestrator and Privacy Guard, decoupling intents from secrets and routing the sanitised sub-tasks to the appropriate Cloud Tier based on trust and risk levels.

Figure 2: Token Parsimony vs Data Leakage by user profile and secret typology. The model achieves high Token Parsimony across the board but suffers from severe Leakage strictly in the Lazy/Institutional quadrant.

Figure 3: Exploratory space analysis of Leakage Rate across parameter scales (8B to 104B). Results are based on limited sample sizes designed to identify structural vulnerabilities rather than provide exhaustive statistical baselines.

Figure 4: Projected vs actual OpEx reduction when comparing a standard monolithic Cloud API call against active local prompt decomposition and selective routing.

Figure 5: LLM-as-a-Judge comparison of final response quality between raw Context (Baseline) and compressed/sanitised Context (Dual-Vault APO).

Figure 6: Theoretical projection of OpEx and Emergent Leakage risk over a 15-turn conversation: Monolithic memory vs. LIFO Compacting.

Figure 7: Latency overhead simulation (Time-To-First-Token). The Dual-Vault architecture becomes faster than direct Cloud routing for contexts larger than 10k tokens due to massive payload reduction.

Figure 8: Comparison of local hardware cost vs. Leakage and Compression rates. The Hybrid architecture matches the 70B Frontier performance with 1/10th of the required VRAM.

Privacy Protection for LLM Agents

See all 33 papers

PromptGraph: Graph-Guided Prompt Sanitization for Balancing Privacy and Utility in LLM Inference
Jul 12, 2026Chen Gu, Hui Wan, Donghui Hu +2Privacy-Preserving Language Models Privacy Leakage in Language Models
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China
Tokenized Key-Gated Adapter Routing: A Secure Access Control Mechanism Against Private Data Leakage in LLMs
Sep 28, 2026Mohamed Shaaban, Mohamed ElmahallawyLLM Security Privacy Leakage in Language Models
School of Engineering and Applied Sciences, Washington State University, Richland, WA 99354, USA
SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models
Jun 3, 2026Peihua Mai, Xuanrong Gao, Youlong Ding +3Privacy-Preserving Language Models LLM Inference
1National University of Singapore (Chongqing) Research Institute · 2Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles, Chongqing, China · 3National University of Singapore +3

Jul 12, 2026J/K move · Enter open · S save

PromptGraph: Graph-Guided Prompt Sanitization for Balancing Privacy and Utility in LLM Inference

Chen Gu, Hui Wan, Donghui Hu, Hui Wang+1

School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China

Privacy-Preserving Language Models Privacy Leakage in Language Models

Open paper AskPDF arXiv

Large Language Model (LLM) services introduce a fundamental privacy challenge. Sensitive information may be inferred not only from explicit identifiers, such as names or phone numbers, but also from contextual associations among otherwise innocuous spans. Existing sanitizers typically assign privacy or utility signals to individual spans without explicitly modeling pairwise relationships among them. In this paper, we propose PromptGraph, a graph-guided prompt-sanitization approach for privacy-preserving LLM inference. PromptGraph estimates privacy leakage at the span level and utility-relevant contextual dependencies between pairs of spans. It represents each prompt as an attributed graph, in which nodes carry span-level privacy scores and edges encode contextual dependencies needed to preserve utility. The sanitization objective selects a protected span set that maximizes privacy gain while penalizing the loss of contextual dependencies. This formulation explicitly balances privacy and utility when contextual evidence is hidden. Protected spans are sanitized locally, and returned placeholders are restored only after passing local consistency checks. We conduct extensive experiments showing that PromptGraph achieves a more favorable balance between privacy and utility than prompt-privacy baselines.

Abstract

Privacy Protection for LLM Agents

Explore similar work

PromptGraph: Graph-Guided Prompt Sanitization for Balancing Privacy and Utility in LLM Inference

Tokenized Key-Gated Adapter Routing: A Secure Access Control Mechanism Against Private Data Leakage in LLMs

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models