Large language models are increasingly deployed as LLM-as-a-Service (LLMaaS), where user queries are sent to remote LLMs for inference, raising severe privacy concerns about the exposure of sensitive user inputs. Differentially private inference mitigates these risks by injecting noise into input embeddings. However, existing methods struggle to balance privacy and utility, as stronger privacy protection often leads to substantial accuracy degradation in LLM inference. In this work, we propose DP-SPD, a differentially private LLM inference framework that protects user queries by hiding the real query among client-generated dummy queries. All queries are randomly shuffled before being sent to the server, preventing direct identification of the user's true intent. To improve inference efficiency, we exploit the shared-prefix structure among queries and reuse transformer key--value (KV) caches during decoding. We analyze the privacy guarantee of DP-SPD and evaluate its utility on classification and generation tasks, as well as its privacy protection under QIA, EIA, and GPT-based attacks. Experimental results show that our method achieves strong inference-time privacy protection and near-plaintext utility while improving inference efficiency through shared-prefix KV-cache reuse.
Figures & tables
Figure 1: Full workflow of DP-SPD. The client generates prefix-consistent dummy suffixes, perturbs and shuffles them locally, and sends the shuffled noisy candidates to the server for inference.
Figure 2
Figure 3: Comparison with related private-inference methods on instruction generation using LLaMA-2-7B-Chat at ϵ=2.5 . Lower PPL and higher ROUGE-L are better; PPL uses a logarithmic scale.
Qwen3-14B / CNN-DM
Qwen3-32B / WildChat
Method
PPL ↓
R-L ↑
QA Acc. ↑
Open ↑
Plaintext
12.92
0.1572
72.6%
4.716
DP-SPD
14.11
0.1555
72.6%
4.716
TextObf.
16.88
0.1499
27.6%
2.93
DP-Forward
240441.63
0.1314
21.4%
2.71
CUSTEXT+
2606.63
0.0352
43.8%
3.79
Table 2: Utility on stronger models and real-world queries. CNN/DailyMail uses Qwen3-14B at ϵ=1.5 ; WildChat uses Qwen3-32B at ϵ=3 .
Figure 4: Generation utility on WikiText-2 with LLaMA-2-7B-Chat. Lower PPL is better. We vary the privacy budget ϵ , dummy number k , and dummy ratio r ; larger anonymity sets provide the largest utility gains under tighter privacy.
Dataset
Dummy strategy
ASR (%) ↓
Real PPL
Dummy PPL
PPL gap ↓
IMDb
DP-SPD prefix-consistent
13.53
29.32
36.69
7.38
Random-sampled
36.25
29.31
49.32
20.00
Random-token
97.94
29.31
1282.39
1253.08
WikiText-2
DP-SPD prefix-consistent
11.89
129.61
108.92
31.05
Random-sampled
28.83
145.49
440.03
294.54
Random-token
74.86
153.44
1936.43
1782.99
Table 3: QIA and dummy-strategy ablation. Results are averaged over ϵ∈{1.5,2.0,2.5,3.0} , r∈{20%,50%,80%} , and k∈{9,19,29} . Lower values are better; bold marks the best ASR and PPL gap.
Table 7
Figure 5: Effect of continuation-model scale on dummy quality, held-out suffix PPL, generation time, and QIA ASR. Higher Surface ROUGE-L is better; lower values are better for all other metrics.
Figure 6: Inference efficiency with shared-prefix KV-cache reuse for 1024-token inputs. Larger shared-prefix ratios and larger anonymity sets yield greater computational savings.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Samples
Active tokens
Changed
Recovery
SST-2
872
20,199
18
99.9109%
QNLI
5,463
262,051
323
99.8767%
Appendix
Table 6: API-only token projection diagnostics at ϵ=3 . Token recovery is the fraction of active tokens unchanged after nearest-token projection.
Input / Interface
Accuracy (%)
Δ vs. DP Emb.
Clean embedding
94.20
–
Clean API-only
94.20
–
DP-SPD embedding
89.13±0.24
–
DP-SPD API-only projection
87.46±0.31
−1.67 pp
Appendix
Table 7: Projection-induced interface gap on SST-2. Results are mean ± standard deviation over five noise seeds.
Method
Dummy Ratio
Privacy Budget ( ϵ )
1.5
2.0
2.5
3.0
4.0
Plaintext
-
0.820
dχ -privacy
-
0.663
0.758
0.812
0.812
0.817
DP-SPD
r=0.1
0.672
0.782
0.810
0.813
0.817
r=0.2
0.689
0.797
0.809
0.813
0.815
r=0.4
0.726
0.809
0.813
0.816
0.815
Appendix
Table 8: Classification Accuracy on IMDb using LLaMA-2-7B ( k=29 ).
Method
Dummy Ratio
ϵ
k=9
k=19
k=29
k=39
Plaintext
-
-
0.820
DP-SPD
0.1
1.5
0.659
0.666
0.672
0.672
1.8
0.724
0.732
0.737
0.741
2.0
0.779
0.781
0.782
0.782
DP-SPD
0.6
1.5
0.664
0.714
0.762
0.784
1.8
0.788
0.806
0.808
0.807
Appendix
Table 9: The impact of the number of dummy inputs ( k ) on LLaMA-2-7B model accuracy under different privacy budgets ( ϵ ) using IMDb dataset. The results demonstrate that utility generally improves as k increases.
Method
ϵ=1.5
ϵ=2.0
ϵ=2.5
ϵ=3.0
ϵ=4.0
Clean control
81.85
81.85
81.85
81.85
81.85
DP-SPD ( r=1,k=29 )
81.20
81.80
81.90
81.90
82.00
CAPE-style
50.98
50.68
50.92
51.17
51.72
CUSTEXT-style
59.62
59.38
59.68
60.68
63.45
DP-Forward-style
50.28
50.30
50.13
50.16
50.28
InferDPT-style
50.40
50.49
52.43
52.88
54.46
Appendix
Table 10: IMDb classification accuracy (%) under the unified LLaMA-2-7B evaluation harness. DP-SPD uses the fixed configuration r=1,k=29 ; the remaining private baselines are style-adapted implementations.
Figure 7: IMDb classification accuracy under different privacy budgets and dummy ratios. This visualization was moved from the main text to the appendix for space.
Method
Metric
ϵ=1.5
ϵ=2.0
ϵ=2.5
ϵ=3.0
k=9
k=19
k=29
k=39
k=49
k=9
k=19
k=29
k=39
k=49
k=9
k=19
k=29
k=39
k=49
k=9
k=19
k=29
k=39
k=49
Plaintext
PPL
14.76
ROUGE
0.194
dχ -privacy
PPL
747.61
63.14
20.59
16.31
ROUGE
0.037
0.129
0.174
0.193
DP-SPD with varying Dummy Ratio ( r )
Appendix
Table 11: Utility comparison (PPL and ROUGE-L) on WikiText-2 . We compare Baselines and DP-SPD with varying Dummy Ratios ( r ). At ϵ=1.5 , utility is generally poor due to extreme noise. At ϵ=2.0 , increasing r significantly improves utility by reducing the impact of context noise. At ϵ≥2.5 , DP-SPD converges to the plaintext baseline regardless of r .
Method
ϵ=1.5
ϵ=2.0
ϵ=2.5
ϵ=3.0
Plaintext
14.76 / .194
14.76 / .194
14.76 / .194
14.76 / .194
DP-SPD
17.54 / .190
14.93 / .192
14.78 / .199
14.74 / .196
CAPE-style
21853.10 / .000
20549.84 / .000
19633.91 / .000
18942.00 / .000
CUSTEXT-style
3177.39 / .095
2938.88 / .093
2647.31 / .097
2297.97 / .100
DP-Forward-style
87500.65 / .168
78406.94 / .168
68875.98 / .163
60178.53 / .158
SANTEXT-style
10729.42 / .090
10697.31 / .089
10623.72 / .084
10452.40 / .085
Appendix
Table 12: WikiText-2 generation comparison under the unified LLaMA-2-7B harness. Each entry is PPL / ROUGE-L. DP-SPD uses r=1,k=39 .
Method
Metric
ϵ=1.5
ϵ=2.5
ϵ=4.0
k=9
k=19
k=29
k=39
k=9
k=19
k=29
k=39
k=9
k=19
k=29
k=39
Plaintext
PPL
9.34
ROUGE
0.160
dχ -privacy
PPL
431.93
14.43
9.56
ROUGE
0.059
0.134
0.144
DP-SPD with varying Dummy Ratio ( r )
Appendix
Table 13: Utility comparison (PPL and ROUGE-L) on the cleaned version of Stanford Alpaca . We compare Baselines and DP-SPD with varying Dummy Ratios ( r ). Consistent with WikiText-2 results, utility is low at ϵ=1.5 but improves significantly with higher r and larger ϵ . DP-SPD shows clear benefits from context preservation at lower ϵ .
Method
Metric
ϵ=1.5
ϵ=2.5
ϵ=4.0
k=9
k=19
k=29
k=39
k=9
k=19
k=29
k=39
k=9
k=19
k=29
k=39
Plaintext
PPL
8.55
ROUGE
0.258
dχ -privacy
PPL
409.17
13.62
9.31
ROUGE
0.064
0.234
0.243
DP-SPD with varying Dummy Ratio ( r )
Appendix
Table 14: Utility comparison (PPL and ROUGE-L) on Databricks-Dolly-15k . The trends are consistent with other datasets: DP-SPD effectively balances privacy and utility, particularly at moderate noise levels ( ϵ=2.5 ). At high privacy ( ϵ=1.5 ), DP-SPD with higher r shows improved text quality (ROUGE).
Dataset
Method
ϵ=1.5
ϵ=2.5
ϵ=4.0
Alpaca
Plaintext
9.34 / .160
9.34 / .160
9.34 / .160
DP-SPD
11.87 / .132
9.25 / .151
9.48 / .153
CAPE-style
17191.15 / .000
14053.85 / .000
12754.43 / .000
CUSTEXT-style
1856.30 / .085
1445.68 / .087
832.20 / .092
DP-Forward-style
86864.22 / .116
67077.54 / .121
48869.19 / .121
SANTEXT-style
9374.88 / .078
9253.24 / .077
9031.51 / .076
Appendix
Table 15: Instruction-generation comparison under the unified LLaMA-2-7B harness. Each entry is PPL / ROUGE-L. DP-SPD uses r=1,k=39 .
Figure 8: Generation-utility visualizations moved from the main text to the appendix.
ϵ
r
PPL ↓
ROUGE-L ↑
k=9
k=19
k=29
k=39
k=9
k=19
k=29
k=39
1.0
0.2
253.50
161.76
141.97
137.18
0.1197
0.1361
0.1391
0.1411
1.0
0.4
315.52
128.56
97.74
90.92
0.1173
0.1387
0.1388
0.1403
1.0
0.6
403.63
98.93
64.93
58.43
0.1209
0.1396
0.1433
0.1469
1.0
0.8
553.70
71.02
40.13
35.13
0.1184
0.1431
0.1507
0.1504
1.5
0.2
16.07
16.04
16.05
16.05
0.1545
0.1541
0.1526
0.1531
Appendix
Table 16: Generation performance of Qwen3-14B on CNN/DailyMail summarization. We report PPL and ROUGE-L under different privacy budgets, dummy ratios, and dummy numbers. Lower PPL and higher ROUGE-L are better.
Method
ϵ=1.0
ϵ=1.5
ϵ=2.0
DP-SPD
35.13 / .1504
14.11 / .1555
13.12 / .1544
CAPE-style
118680.03 / .0030
207624.46 / .0028
200676.06 / .0045
CUSTEXT-style
2897.53 / .0288
2606.63 / .0352
2297.36 / .0400
DP-Forward-style
285789.84 / .1264
240441.63 / .1314
228022.08 / .1318
SANTEXT-style
18150.13 / .0524
18081.09 / .0500
18139.80 / .0553
TextObfuscator-style
25.68 / .1351
16.88 / .1499
14.65 / .1533
Appendix
Table 17: CNN/DailyMail generation comparison with Qwen3-14B. Each entry is PPL / ROUGE-L. DP-SPD uses r=0.8,k=39 .
Adapter
Preserved core mechanism
Adaptation / omitted component
CAPE-style
Min–max normalization followed by Laplace perturbation
The original private-attribute adversary and task-head training are not reproduced in the unified harness.
CUSTEXT-style
Top- k exponential token mechanism
Original word-vector reconstruction is replaced by the LLaMA/Qwen BPE vocabulary and the common evaluator.
DP-Forward-style
Hidden-state perturbation
Full noisy fine-tuning from the original pipeline is not reproduced for the unified classification/generation evaluator.
InferDPT-style
RANTEXT-style perturbation and two-stage data flow
IMDb probing does not reproduce the complete remote-LLM/local-extractor pipeline; generation PPL uses an extractor-side conditional definition and is therefore not directly ranked against server-side PPL.
SANTEXT-style
Exponential token sanitization
The mechanism is adapted to the tokenizer/BPE vocabulary of the current target model and the unified evaluator.
TextObfuscator-style
Client-boundary hidden-state perturbation
Full joint training, prototype losses, and other task-specific training components are not reproduced.
Appendix
Table 18: Style-adapted baselines used in the unified LLaMA/Qwen evaluation harness. These are controlled adaptations of the core privacy mechanisms rather than exact reproductions of every original training pipeline.
Dummy Strategy
r
QIA ASR (%) ↓
PPL Gap ↓
k=9
k=19
k=29
k=9
k=19
k=29
DP-SPD
20%
26.50
22.50
21.00
7.41
7.38
7.42
DP-SPD
50%
13.50
10.25
8.00
8.39
8.16
8.11
DP-SPD
80%
7.75
6.00
6.25
6.89
6.08
6.53
Random-Sampled
20%
54.75
48.00
39.50
19.40
19.59
19.96
Random-Sampled
50%
42.00
31.75
31.25
22.14
21.64
21.03
Appendix
Table 19: Detailed QIA results on IMDb. Results are averaged over ϵ∈{1.5,2.0,2.5,3.0} . Lower ASR and lower PPL gap are better.
Dummy Strategy
r
QIA ASR (%) ↓
PPL Gap ↓
k=9
k=19
k=29
k=9
k=19
k=29
DP-SPD
20%
24.00
17.75
13.50
16.26
15.49
14.89
DP-SPD
50%
18.00
13.75
12.25
6.86
7.24
6.08
DP-SPD
80%
4.00
1.50
2.25
72.95
69.94
69.72
Random-Sampled
20%
54.75
42.50
31.75
99.46
98.86
96.29
Random-Sampled
50%
36.25
29.25
25.75
278.44
259.27
243.80
Appendix
Table 20: Detailed QIA results on WikiText-2. Results are averaged over ϵ∈{1.5,2.0,2.5,3.0} . Lower ASR and lower PPL gap are better.
Attack Metric
k=9
k=19
Random selection
Min-K QIA ASR
4.00%
1.75%
10% / 5%
PPL QIA ASR
4.25%
2.50%
10% / 5%
GPT selection ASR
20.0%
13.0%
10% / 5%
GPT exact-query recovery
18.25%
11.0%
–
Appendix
Table 21: QIA and semantic-reconstruction results on WildChat. Lower values indicate stronger privacy.
Projection Model
Dim.
Token Retention
GPT-4o Acc.
Qwen2.5-0.5B
896
98.50±0.11 %
54.8±3.03 %
Gemma-2-2B
2,304
77.85±0.15 %
54.0±2.55 %
Qwen − Gemma
–
+20.65 pp
+0.8 pp
Appendix
Table 22: Representation-space robustness for API-only DP-SPD on MMLU-Pro. Values are reported as mean ± standard deviation.
With the widespread deployment of public large language models (LLMs) such as ChatGPT, protecting user prompt privacy has become an increasingly critical issue. Existing privacy-preserving inference methods sacrifice either utility or efficiency, and often require model-specific modifications that limit their compatibility. In this paper, we propose SharedRequest, a model-agnostic framework for privacy-preserving LLM inference that reformulates privacy protection at the batch level rather than the individual-prompt level. The key idea is to obscure sensitive information by mixing original prompts with noisy variants, while grouping semantically equivalent instructions to amortize the inference cost over a large batch of queries with minimal impact on LLM response quality. This design is independent of the LLM architecture, requiring no access to model parameters or architectural modification. Empirical results demonstrate that SharedRequest achieves over 20% higher utility compared to prior differential privacy baselines, and its shared-prompt mechanism reduces query cost by up to 5× compared to non-batched inference.
Peihua Mai, Xuanrong Gao, Youlong Ding +3
1National University of Singapore (Chongqing) Research Institute · 2Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles, Chongqing, China · 3National University of Singapore +3
User prompts provided to large language models (LLMs) may contain private information. One way to protect them is to execute the LLM inside a trusted execution environment (TEE). However, this results in slow inference times as current TEEs are significantly slower than GPUs for LLM inference. To circumvent this, Tramèr and Boneh (2019) proposed Slalom which splits neural network inference between a TEE and an untrusted GPU. They encrypt inputs to computations outsourced to the GPU. In this paper, we extend this split-inference architecture to LLM inference and instead protect intermediate inputs using differential privacy (DP). We first demonstrate that masking intermediate representations is necessary by showing an 80% accuracy on a prompt-reconstruction attack from these representations. Our main contribution is a global sensitivity analysis of key functions in LLM inference, which bounds the required scale of DP noise. Unlike encryption, DP avoids quantization, allowing the LLM to remain in the floating-point domain. We also derive an upper bound on the floating-point error from masking and subsequent noise cancellation as a function of the privacy parameter epsilon, keeping the same quality of the LLM response. We implement our architecture using the Intel TDX TEE and two LLMs: Llama-3.2-3B and Qwen3-4B. Our split execution is nearly twice as fast as fully TDX-based inference. Moreover, it is at most 43% faster than Slalom while achieving higher accuracy. Finally, we demonstrate that prompt reconstruction, even with knowledge of the DP mechanism, cannot recover more information than is contained in an unrelated prompt.
Shashie Dilhara Batan Arachchige, Robin Carpentier, Hassan Jameel Asghar +1
LLM-as-a-Service has become a dominant paradigm for accessing large language models without requiring local deployment. However, the process raises severe privacy concerns when user queries may include sensitive or confidential contexts. Differential privacy (DP) provides a principled approach to protecting such inputs, but existing DP inference methods are primarily designed for short- or moderate-context settings and struggle with million-token-scale inputs due to limited context capacity and severe utility degradation under DP perturbation. In this paper, we propose DP-MIND, a memory-guided framework that scales differentially private inference to million-token contexts. DP-MIND enables private long-context inference by locally perturbing the input and processing it chunk by chunk with a compact memory that preserves task-relevant information across the sequence. To make this memory robust to DP noise, we optimize the memory construction policy directly on perturbed inputs using reinforcement learning, guided by both final answer correctness and memory consistency. We further recover perturbation-corrupted answer spans through a local denoising module without exposing the original context to the external model. We provide theoretical privacy analysis of DP-MIND and further conduct extensive empirical analysis across multiple long-context benchmarks with context lengths up to the million-token scale, demonstrating that DP-MIND consistently outperforms existing DP and long-context inference baselines in both scalability and utility.
Zhihao Liu, Yitian Cheng, Shuang Liu +3
Zhejiang University · Ant Group · Institute of Science Tokyo +1