Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)

Yu Lin; Qizhi Zhang; Wenqiang Ruan; Daode Zhang; Jue Hong; Ye Wu; Hanning Xia; Yunlong Mao

cs.CRMar 2, 2026

Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)

Authors: Yu Lin, Qizhi Zhang, Wenqiang Ruan, Daode Zhang, Jue Hong, Ye Wu, Hanning Xia, Yunlong Mao, +1 more

Organizations: ByteDance · Nanjing University

Abstract

The rapid development of large language models (LLMs) has driven the widespread adoption of cloud-based LLM inference services, while also bringing prominent privacy risks associated with the transmission and processing of private data in remote inference. For privacy-preserving LLM inference technologies to be practically applied in industrial scenarios, three core requirements must be satisfied simultaneously: (1) Accuracy and efficiency losses should be minimized to mitigate degradation in service experience. (2) The inference process can be run on large-scale clusters consist of heterogeneous legacy xPUs. (3) Compatibility with existing LLM infrastructures should be ensured to reuse their engineering optimizations. To the best of our knowledge, none of the existing privacy-preserving LLM inference methods satisfy all the above constraints while delivering meaningful privacy guarantees. In this paper, we propose AloePri, the first privacy-preserving LLM inference method for industrial applications. AloePri protects both the input and output data by covariant obfuscation, which jointly transforms data and model parameters to achieve better accuracy and privacy. We carefully design the transformation for each model component to ensure inference accuracy and data privacy while keeping full compatibility with existing infrastructures of Language Model as a Service. AloePri has been integrated into an industrial system for the evaluation of mainstream LLMs. The evaluation on Deepseek-V3.1-Terminus model (671B parameters) demonstrates that AloePri causes accuracy loss of 0.0%~3.5% and exhibits efficiency equivalent to that of plaintext inference. Meanwhile, AloePri successfully resists state-of-the-art attacks, with less than 5% of tokens recovered. To the best of our knowledge, AloePri is the first method to exhibit practical applicability to large-scale models in real-world systems.

Figures & tables

Figure 1: Workflow of AloePri . The client locally obfuscates the model and deploys it to the server, and handles prompt obfuscation and response de-obfuscation during online phase.

Figure 2: Overview of AloePri . In the offline model obfuscation process, token-level permutation and linear transformations are employed to construct the obfuscations $\phi^{\text{embed}},\phi^{\text{head}},\phi^{\text{attn}},$ and $\phi^{\text{ffn}}$ . In the online inference process, a secret vocabulary mapping associated with the permutation is used to construct the data obfuscation $\phi_{X}$ and de-obfuscation $\psi_{Y}$ .

	DP	Rényi DP	Metric DP	Rényi-metric DP
Privacy notion	Adjacent datasets	Adjacent datasets	Metric space	Metric space
Gaussian mechanism analysis	Not natural	Natural	Not natural	Natural

Table 1: Comparison of DP, Rényi DP, metric DP, and Rényi-metric DP.

Method	Classification Acc. $\uparrow$	Generation Acc. $\uparrow$			Privacy 1
	SST2	MMLU	PIQA	IFEval	Attack	TTRSR(%) $\downarrow$	CosSim $\downarrow$
Plaintext	97.25	81.95	77.45	77.09	-	-	-
SANTEXT	81.31	51.07	49.72	57.45	(Direct obs.)	62.90	0.93
RANTEXT	87.50	37.77	18.30	57.18	(Direct obs.)	56.92	0.88
DP-Forward 2	93.60	-	-	-	NN	37.82	0.51
					IMA	77.08	0.84

Table 2: Accuracy and privacy comparison with baselines on Qwen2.5-14B-Instruct.

Panel of Figure 3 — Figure 3: Privacy (TTRSR) and accuracy under various noise parameters on Qwen2.5-14B-Instruct and C-Eval.

Panel of Figure 4 — Figure 4: Impact of $\lambda$ on Qwen2.5-14B-Instruct.

Model		Plaintext accuracy $\uparrow$					AloePri accuracy $\uparrow$					AloePri privacy against VMA $\downarrow$
Model		MMLU	C-Eval	HumanEval	PIQA	IFEval	MMLU	C-Eval	HumanEval	PIQA	IFEval	TTRSR(%)	PIIRSR(%)	BLEU-4
R1-Distill	14B	78.91	86.18	96.34	88.14	72.83	79.2	85.93	93.9	88.79	72.27	12.36	2.37	0.52
R1-Distill	32B	83.5	88.34	97.56	88.79	73.94	83.61	87.84	95.12	88.08	73.57	10.10	2.63	0.78
Qwen3	14B	87.64	87.35	94.51	89.72	86.14	84.57	87.12	95.12	90.26	85.40	25.05	1.62	1.72
Qwen3	32B	89.61	89.0	98.78	90.75	84.29	86.24	87.64	96.95	89.5	81.15	19.64	1.23	1.41
Llama3	8B	65.91	50.16	55.49	80.74	68.58	63.5	44.55	57.32	78.94	67.84	2.57	0	0.84

Table 3: Accuracy and privacy of AloePri under various models and datasets. Privacy is evaluated with VMA on PUPA dataset.

Table 8

Table 9

Panel of Figure 5 — Figure 5: TPOT and TTFT vs. $h$ on R1-Distill-14B

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Notation	Description
$n$	Number of vocabulary’s tokens
$d,d_{\text{head}},d_{\text{ffn}}$	Hidden size, attention head dimension, and FFN intermediate dimension
$h$	Expansion size for the key matrix
$d^{\prime}$	Expanded hidden size
$m,m_{\text{kv}}$	Number of attention heads, and number of attention key-value heads
$m_{\text{exp}}$	Number of MoE experts

Appendix

Table 8: Notations and their Descriptions

Known $X$	Observed $Y$
$W_{e}W_{h}$	$\Pi W^{\star}_{\text{embed}}W_{h}\Pi^{T}$
$W_{e}W_{\text{query}}(W_{e}W_{\text{key}})^{T}$	$\Pi W^{\star}_{\text{embed}}W_{\text{query}}(\Pi W^{\star}_{\text{embed}}W_{\text{key}})^{T}$
$W_{e}W_{\text{gate}}$	$\Pi W^{\star}_{\text{embed}}W_{\text{gate}}\hat{Z}_{\text{ffn}}$
$W_{e}W_{\text{up}}$	$\Pi W^{\star}_{\text{embed}}W_{\text{up}}\hat{S}_{\text{ffn}}\hat{Z}_{\text{ffn}}$
$W_{\text{down}}W_{h}$	$\hat{Z}^{-1}_{\text{ffn}}\hat{S}^{-1}_{\text{ffn}}W_{\text{down}}W^{\star}_{\text{head}}\Pi^{T}$
$W_{e}W_{\text{router}}$	$\Pi W^{\star}_{\text{embed}}\text{norm}(W_{\text{router}})\hat{Z}_{\text{router}}$

Appendix

Table 9: Weight combinations to recover

\Pi

with VMA

Experiment	$\alpha_{e}$	$\alpha_{h}$	$\lambda$	$h$	$\beta$	$\gamma$
Fig. 3	$\triangle$ 1	$\triangle$	0.3	128	8	$1e^{3}$
Fig. 4	1.0	0.2	$\triangle$	128	8	$1e^{3}$
Fig. 5	1.0	0.2	0.3	$\triangle$	8	$1e^{3}$
Tab. 2 , 7 , 7	1.0	0.2	0.3	128	8	$1e^{3}$
Tab. 3	0.5,1.0 2	0.2	0.3	128	8	$1e^{3}$
Tab. 5	1.0	0.2	$\triangle$	128	$\triangle$	$\triangle$

Appendix

Table 10: Privacy hyperparameter settings of experiments.

Explore similar work

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models
Jun 3, 2026Peihua Mai, Xuanrong Gao, Youlong Ding +3Privacy-Preserving Language Models LLM Inference
1National University of Singapore (Chongqing) Research Institute · 2Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles, Chongqing, China · 3National University of Singapore +3
Good-Enough LLM Obfuscation (GELO)
Mar 5, 2026Anatoly Belikov, Ilya FedotovPrivacy-Preserving Language Models LLM Security
SingularityNET Foundation, Baarerstrasse 141, Zug, 6300, Switzerland · Singularity Compute
Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs
Feb 19, 2026Arka Pal, Louai Zahran, William Gvozdjak +2LLM Inference LLM Auditing
Ritual · MIT · Columbia University

Jun 3, 2026J/K move · Enter open · S save

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models

Peihua Mai, Xuanrong Gao, Youlong Ding, Xianglong Du+2

1National University of Singapore (Chongqing) Research Institute · 2Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles, Chongqing, China · 3National University of Singapore+3

Privacy-Preserving Language Models LLM Inference Privacy Leakage in Language Models Efficient Language Model Inference

Open paper AskPDF arXiv

With the widespread deployment of public large language models (LLMs) such as ChatGPT, protecting user prompt privacy has become an increasingly critical issue. Existing privacy-preserving inference methods sacrifice either utility or efficiency, and often require model-specific modifications that limit their compatibility. In this paper, we propose SharedRequest, a model-agnostic framework for privacy-preserving LLM inference that reformulates privacy protection at the batch level rather than the individual-prompt level. The key idea is to obscure sensitive information by mixing original prompts with noisy variants, while grouping semantically equivalent instructions to amortize the inference cost over a large batch of queries with minimal impact on LLM response quality. This design is independent of the LLM architecture, requiring no access to model parameters or architectural modification. Empirical results demonstrate that SharedRequest achieves over

20\%

higher utility compared to prior differential privacy baselines, and its shared-prompt mechanism reduces query cost by up to

5\times

compared to non-batched inference.

Abstract

Explore similar work

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models

Good-Enough LLM Obfuscation (GELO)

Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs