Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decoding

Kejia Zhang; Tianyuan Zou; Zixuan GU; Yang Liu

cs.CRAug 29, 2026

Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decoding

Authors: Kejia Zhang, Tianyuan Zou, Zixuan GU, Yang Liu

Organizations: The Hong Kong Polytechnic University · Institute for AI Industry Research, Tsinghua University · School of Software, Tsinghua University

Abstract

Applications such as personalized assistance and proprietary document analysis require large language models (LLMs) to generate outputs from private data. Yet powerful LLMs typically cannot be deployed on the resource-constrained devices where private data resides, and uploading private data to cloud-hosted LLMs exposes sensitive information. Recent work addresses this tension with a cloud-edge collaborative decoding paradigm, where private data are kept on the edge with a small language model (SLM) producing next-token distributions, which are fused with predictions from a cloud LLM operating solely on public data. In this paper, we systematically analyze the privacy risks of such a paradigm with a novel evaluation framework using constructed QA datasets, which show that such collaboration can expose substantial private-context information. To address such privacy leakage, we propose CoVeil, a defense mechanism which dynamically optimizes transmitted signals to suppress leakage during decoding time while preserving the collaborative quality. Extensive evaluations demonstrate that CoVeil consistently improves the privacy-utility trade-off over existing baselines by reducing data leakage by up to 87.2%, with minimal accuracy loss.

Figures & tables

Figure 1: Cloud-edge collaborative decoding, where private context remains on the edge under two fusion modes.

Figure 2: DecodeLeak overview: cloud-observed decoding signals are evaluated in two sequential stages: private evidence recall from top- $K$ token IDs, followed by private context inversion from the per-step exposure sequence.

Panel of Figure 3 — Figure 3: Private evidence recall across leakage channels in Qwen2.5-72B/1.5B collaboration using 200 examples.

Figure 4: Privacy-utility trade-off curves: private evidence recall under Qwen2.5-72B/1.5B collaboration.

Figure 5: Privacy-utility trade-off curves: private context inversion under Qwen2.5-72B/1.5B collaboration.

Fusion	Acc	Private Evidence Recall (@100)				Private Context Inversion
Fusion	Acc	Token-ER	ROUGE1-ER	Span-ER	AUC	ROUGE-1 Recall	ROUGE-1 F1
Cloud-side	59.3	56.6	48.3	28.3	46.7	41.1	32.7
Edge-side	59.3	43.0	34.6	28.3	35.0	39.2	30.3

Table 1: DecodeLeak results on MedPriv under undefended Qwen2.5-72B/1.5B collaboration.

Figure 6: Effect of the privacy weight

\lambda

on per-step CoVeil decisions on MedPriv .

Figure 7: Cloud-side upload decisions on MedPriv : candidate SLM token probabilities are uploaded when

g_{t}(i)<0

and suppressed when

g_{t}(i)\geq 0

Figure 8: Edge-side selected-token rank distribution at matched Token-ER@100 on MedPriv ; ranks are computed under

p_{\mathrm{fuse}}^{(t)}

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Additional privacy-utility trade-off curves for Qwen2.5-72B/1.5B cloud-edge collaboration on ROUGE1-ER@100 and AUC@100.

Figure 10: Cloud-side and edge-side privacy-utility trade-off curves for LLaMA-3.2-3B/LLaMA-3.1-70B cloud-edge collaboration on MedPriv . These appendix results correspond to the main-text privacy-utility evaluation and report private evidence recall with Token-ER@100, ROUGE1-ER@100, Span-ER@100, and AUC@100.

Setting	Acc.	Token-ER@100	ROUGE1-ER@100	Span-ER@100	AUC@100
w/o CoVeil	59.60	0.5655	0.4817	0.2823	0.4668
CoVeil , $K=10$	58.80	0.1320	0.0542	0.0802	0.1299
CoVeil , $K=50$	59.00	0.1344	0.0551	0.0822	0.1325
CoVeil , $K=100$	59.00	0.1344	0.0551	0.0822	0.1325
CoVeil , $K=500$	59.00	0.1344	0.0551	0.0822	0.1325
CoVeil , $K=1000$	59.00	0.1344	0.0551	0.0822	0.1325

Appendix

Table A1: Sensitivity to the upload candidate size

K

under cloud-side fusion on MedPriv . CoVeil uses

\lambda=1.0

. Privacy metrics are raw scores.

Data	Setting	Acc.
MedPriv	Cloud LLM (w/o private context)	33.9
	Edge SLM (w/ private context)	44.5
	Cloud-edge collaboration (w/ private context)	59.6
	Gain over best single model	+15.1
CommPriv	Cloud LLM (w/o private context)	35.6
	Edge SLM (w/ private context)	55.9

Appendix

Table A2: Undefended Qwen2.5-72B/1.5B collaboration accuracy. Here, w/o private context denotes the cloud LLM using only the public query, while w/ private context denotes settings where the edge SLM conditions on the private context. The gain is measured against the stronger of the two non-collaborative settings for each benchmark.

Data	$\lambda$	Acc.	Token-ER@100	ROUGE1-ER@100	Span-ER@100	AUC@100
MedPriv	0.0	59.4	0.1374	0.0573	0.0839	0.1356
	0.5	59.4	0.1374	0.0573	0.0839	0.1356
	1.0	59.0	0.1345	0.0551	0.0823	0.1326
	1.5	56.2	0.1178	0.0420	0.0745	0.1159
	2.0	55.8	0.1049	0.0293	0.0685	0.1030
	2.5	55.3	0.0922	0.0183	0.0625	0.0904

Appendix

Table A3: Cloud-side privacy-weight sweep for CoVeil on Qwen2.5-72B/1.5B collaboration. Privacy metrics are raw scores.

Data	$\lambda$	Acc.	Token-ER@100	ROUGE1-ER@100	Span-ER@100	AUC@100
MedPriv	0.0	59.3	0.4301	0.3461	0.2829	0.3495
	0.5	58.2	0.4201	0.3362	0.2756	0.3345
	1.0	56.3	0.4045	0.3189	0.2688	0.3100
	1.25	53.9	0.3856	0.2971	0.2605	0.2948
	1.5	51.2	0.3680	0.2755	0.2469	0.2854
	1.75	47.2	0.3528	0.2517	0.2262	0.2706

Appendix

Table A4: Edge-side privacy-weight sweep for CoVeil on Qwen2.5-72B/1.5B collaboration. Privacy metrics are raw scores.

Fusion	Acc.	Private Evidence Recall (@100)
Fusion	Acc.	Token-ER	ROUGE1-ER	Span-ER	AUC
Cloud-side	66.1	68.5	59.8	42.2	58.5
Edge-side	66.1	58.8	48.3	36.7	49.7

Appendix

Table A5: DecodeLeak private evidence recall on MedPriv under undefended LLaMA-3.2-3B/LLaMA-3.1-70B cloud-edge collaboration. All metrics are percentages computed at top-100 exposure.

Data	Setting	Acc.
MedPriv	Cloud LLM (w/o private context)	38.7
	Edge SLM (w/ private context)	50.1
	Cloud-edge collaboration (w/ private context)	66.4
	Gain over best single model	+16.3
CommPriv	Cloud LLM (w/o private context)	40.2
	Edge SLM (w/ private context)	65.9

Appendix

Table A6: Undefended Llama-3.1-70B-Instruct/3.2-3B collaboration accuracy. Here, w/o private context denotes the cloud LLM using only the public query, while w/ private context denotes settings where the edge SLM conditions on the private context. The gain is measured against the stronger of the two non-collaborative settings for each benchmark.

Source QA question, options, answer, context

\rightarrow

Public/private split

x_{\mathrm{pub}}

and

x_{\mathrm{priv}}

\rightarrow

Sample screening GPT-5.4 proposal + human verification

\rightarrow

Private phrase extraction GPT-5.4 extraction + human verification

\rightarrow

Final JSONL 1,000 verified samples

Appendix

Figure 11: Benchmark construction pipeline for MedPriv and CommPriv . GPT-5.4 is used to propose sample-level decisions and private-evidence phrases, followed by manual verification.

Cloud-side SLM top- $K$ probabilities $\mathcal{C}_{t}=\operatorname{TopK}(p_{S}^{(t)})$ $\Longrightarrow$	Compute gradient score $g_{t}(i)$ from Taylor coefficient $\Longrightarrow$	Adaptive signal upload $\mathcal{S}_{t}=\{i:g_{t}(i)<0\}$
Edge-side Fusion top- $K$ tokens $\mathcal{C}_{t}=\operatorname{TopK}(p_{\mathrm{fuse}}^{(t)})$ $\Longrightarrow$	Compute objective $\mathcal{J}_{t}(i)$ $\Longrightarrow$	Select prefix token $i_{t}^{\star}=\arg\min_{i}\mathcal{J}_{t}(i)$

Appendix

Figure 12: Algorithmic flow of CoVeil . Cloud-side fusion uses a Taylor coefficient to adaptively upload selected SLM token probabilities, whereas edge-side fusion minimizes the equivalent per-token objective to select one token from the fused-distribution candidate pool and append it to the shared autoregressive prefix.

Benchmark	Source	Domain	Samples	Options	Avg. $\|x_{\mathrm{pub}}\|$	Avg. $\|x_{\mathrm{priv}}\|$	Avg. $\|\mathcal{E}_{\mathrm{priv}}\|$	Avg. spans
MedPriv	MedQA-USMLE	Medical	1,000	4	16.1	47.5	26.5	13.7
CommPriv	Cosmos-QA	Commonsense	1,000	4	9.3	60.6	30.6	7.5

Appendix

Table A7: Dataset-level attributes of the constructed benchmarks. Lengths are average word counts; spans are the comma-separated cleaned entries in filtered_context .

Figure 13: Length distributions of

x_{\mathrm{pub}}

x_{\mathrm{priv}}

, and

\mathcal{E}_{\mathrm{priv}}

Privacy Leakage in Language Models

Privacy leakage in language models concerns how models expose training data, infer sensitive traits, or transmit private context through outputs and internal channels.

See all 280 papers

Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models
Jul 14, 2026Yi Li, Chen Li, Jiexiong LiuOn-Device Language Model Inference Privacy-Preserving Language Models
KunlunMeta
Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)
Mar 2, 2026Yu Lin, Qizhi Zhang, Wenqiang Ruan +6Privacy-Preserving Language Models LLM Inference
ByteDance · Nanjing University
Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models
Feb 27, 2026Chung-ju Huang, Huiqiang Zhao, Yuanpeng He +5Privacy-Preserving Language Models Privacy Leakage in Language Models
Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, China · School of Computer Science, Peking University, Beijing, China · Tencent, Shenzhen, China +1

Jul 14, 2026J/K move · Enter open · S save

Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models

Yi Li, Chen Li, Jiexiong Liu

KunlunMeta

On-Device Language Model Inference Privacy-Preserving Language Models LLM Inference Edge-Cloud Collaborative Inference Efficient Language Model Inference

Open paper AskPDF arXiv

On-device LLM inference faces a trilemma of response latency, limited hardware resources and user privacy. Full cloud inference delivers strong computing power but exposes user prompts and dialogue data, while standalone on-device inference is unfeasible for most consumer and embedded edge devices. This paper presents a privacy-centric edge-cloud collaborative LLM inference framework built on endpoint-authenticated KV cache. Local endpoints handle input preprocessing, embedding computation, adaptive feature optimization, KV cache authentication, speculative decoding and low-dimensional model head calculation, while the cloud conducts authenticated decoder inference, KV cache management, token verification and high-dimensional vocabulary projection. Endpoints fuse partial outputs, apply language-adaptive masking and sample target tokens. All transmitted data and truncated logits are quantized and AES-GCM encrypted for privacy, with core lightweight modules, draft parameters and cache access policies kept local to avoid leakage. The framework supports heterogeneous devices including CPU-only, GPU-equipped and embedded devices via optimized streaming, batching and quantized ONNX deployment. Evaluations demonstrate that the framework reduces per-token latency by up to 46.1% and downlink payloads by up to 67.4% over baseline split inference, retaining comparable performance to full cloud inference.

Abstract

Privacy Leakage in Language Models

Explore similar work

Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models

Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)

Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models