DP-MIND: Differentially private memory-guided Long context Inference

Zhihao Liu; Yitian Cheng; Shuang Liu; Rui Wen; Yuke Hu; Zhan Qin

cs.AIOct 9, 2026

DP-MIND: Differentially private memory-guided Long context Inference

Authors: Zhihao Liu, Yitian Cheng, Shuang Liu, Rui Wen, Yuke Hu, Zhan Qin

Organizations: Zhejiang University · Ant Group · Institute of Science Tokyo · King Abdullah University of Science and Technology

Abstract

LLM-as-a-Service has become a dominant paradigm for accessing large language models without requiring local deployment. However, the process raises severe privacy concerns when user queries may include sensitive or confidential contexts. Differential privacy (DP) provides a principled approach to protecting such inputs, but existing DP inference methods are primarily designed for short- or moderate-context settings and struggle with million-token-scale inputs due to limited context capacity and severe utility degradation under DP perturbation. In this paper, we propose DP-MIND, a memory-guided framework that scales differentially private inference to million-token contexts. DP-MIND enables private long-context inference by locally perturbing the input and processing it chunk by chunk with a compact memory that preserves task-relevant information across the sequence. To make this memory robust to DP noise, we optimize the memory construction policy directly on perturbed inputs using reinforcement learning, guided by both final answer correctness and memory consistency. We further recover perturbation-corrupted answer spans through a local denoising module without exposing the original context to the external model. We provide theoretical privacy analysis of DP-MIND and further conduct extensive empirical analysis across multiple long-context benchmarks with context lengths up to the million-token scale, demonstrating that DP-MIND consistently outperforms existing DP and long-context inference baselines in both scalability and utility.

Figures & tables

C^{\prime}=(w_{1}^{\prime},\ldots,w_{L}^{\prime}),\qquad(C_{1}^{\prime},\ldots,C_{N}^{\prime})\leftarrow\mathcal{M}_{\mathrm{DP}}(C).

Algorithm 1

\mathcal{M}_{\mathrm{DP-MIND}}

: Chunk-wise DP Memory Inference with Local Denoising

Privacy Budget	Method	7K	14K	28K	56K	112K	224K	448K	896K	1.8M
$\epsilon=6$	Qwen2.5-7B (baseline)	36.72	37.50	25.78	–	–	–	–	–	–
	Sparse Attention	36.72	37.50	25.78	5.47	3.12	0.00	–	–	–
	Long-Context Scaling	40.62	40.62	35.16	32.03	22.66	18.75	8.56	0.00	–
	Memory-Augmented	38.28	32.03	33.59	28.91	25.00	17.97	8.59	6.25	0.00
	DP-MIND	56.25	56.25	48.44	46.09	44.53	47.66	53.12	49.22	39.06
$\epsilon=7$	Qwen2.5-7B (baseline)	47.66	46.09	40.62	–	–	–	–	–	–

Table 1: Sub-EM comparison on RULER-HQA under different privacy budgets and context lengths. Qwen2.5-7B denotes the standard baseline, Sparse Attention denotes Longformer-style local sliding-window attention ( Beltagy et al., 2020 ) , Long-Context Scaling denotes direct inference with Qwen2.5-7B-Instruct-1M ( Yang et al., 2025 ) , which is a variant designed for up to one million tokens of context, and Memory-Augmented denotes direct chunk-wise querying with recurrent memory propagation.

Figure 2: Results on RULER-HQA under $\epsilon=7$ . Chunk-only inference uses chunk-wise memory processing and can scale to long contexts, but its utility drops quickly without RL-based memory optimization. DP-MIND achieves the best overall performance.

Figure 3: Scaling performance on RULER-HQA with Qwen2.5-14B under $\epsilon=7$ . DP-MIND consistently achieves the best overall performance, while the advantage of memory-based inference becomes more pronounced as the context length increases.

Figure 4: NIAH results under different privacy budgets and context lengths. Results are reported on Single, Multi-Key (M-Key), and Multi-Query (M-Query) with length ranging from 32k to 512k.

Figure 5: Impact of chunk size on utility and runtime. Larger chunks reduce inference time, while moderate chunk sizes generally maintain stronger EM and SubEM across long contexts.

Perturbation	Length	Qwen-1M	DP-MIND
$d_{\chi}$ -privacy ( Feyisetan et al., 2020 )	56K	45.31	73.44 (+ 28.13)
	112K	35.94	68.56 (+ 32.62)
	224K	26.56	70.31 (+ 43.75)
	448K	9.38	71.88 (+ 62.50)
SanText ( Yue et al., 2021 )	56K	52.34	82.03 (+ 29.69)
	112K	50.78	78.91 (+ 28.13)

Table 2: Robustness of DP-MIND under different DP perturbation mechanisms.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Length	Chunk-only	RL MemoryAgent	DP-RL MemoryAgent	DP-MIND
7K	58.59	72.66	75.00 (+2.34)	75.78 (+3.12)
14K	41.41	67.97	71.88 (+3.91)	74.22 (+6.25)
28K	44.53	67.19	69.53 (+2.34)	71.09 (+3.90)
56K	30.47	67.97	67.97 (+0.00)	73.44 (+5.47)
112K	30.47	65.62	66.43 (+0.81)	68.56 (+2.13)
224K	17.97	69.53	69.53 (+0.00)	70.31 (+0.78)

Appendix

Table 3: Ablation study of DP-MIND on RULER-HQA under

\varepsilon=7

, measured by SubEM. Chunk-only splits the context into chunks and queries them without memory propagation. RL MemoryAgent denotes the original memory-based inference method ( Yu et al., 2026 ) . DP-RL MemoryAgent is trained on DP-perturbed data. DP-MIND is our full method with local answer denoising. Numbers in parentheses denote improvements over RL MemoryAgent.

Privacy Budget	Method	7K	14K	28K	56K	112K	224K	448K	896K	1.8M
$\epsilon=6$	Qwen2.5-7B (baseline)	36.12	37.84	27.98	–	–	–	–	–	–
	Sparse Attention	36.12	37.84	27.98	4.81	2.56	0.00	–	–	–
	Long-Context Scaling	43.94	42.29	36.80	34.10	27.23	21.69	12.42	0.00	0.00
	Memory-Augmented	39.98	33.56	34.38	27.88	27.52	22.88	26.51	20.09	20.94
	DP-MIND	53.30	54.93	53.57	48.11	48.36	48.23	52.43	47.62	42.41
$\epsilon=7$	Qwen2.5-7B (baseline)	41.70	45.15	40.90	–	–	–	–	–	–

Appendix

Table 4: F1 comparison on RULER-HQA under different privacy budgets and context lengths. All values are percentages.

Privacy Budget	Method	7K	14K	28K	56K	112K	224K	448K	896K	1.8M
$\epsilon=6$	Qwen2.5-7B (baseline)	21.09	22.66	16.41	–	–	–	–	–	–
	Sparse Attention	21.09	22.66	16.41	0.78	0.00	0.00	–	–	–
	Long-Context Scaling	26.56	25.00	21.88	17.97	14.84	14.06	7.03	2.56	0.00
	Memory-Augmented	24.22	19.53	19.53	10.16	14.06	10.94	12.50	8.59	10.16
	DP-MIND	33.59	39.84	35.94	32.03	28.91	31.25	36.72	32.81	28.12
$\epsilon=7$	Qwen2.5-7B (baseline)	24.22	29.69	29.69	–	–	–	–	–	–

Appendix

Table 5: Exact Match (EM) comparison on RULER-HQA under different privacy budgets and context lengths. All values are percentages.

Method	7K	14K	28K	56K	112K	224K	448K	896K	1.8M
Qwen2.5-14B (baseline)	23.31	22.77	19.64	–	–	–	–	–	–
Sparse Attention	23.31 †	22.77 †	19.64 †	3.57	3.64	0.06	–	–	–
Long-Context Scaling	48.52	37.13	34.96	30.40	27.23	21.21	15.87	10.25	–
Memory-Augmented	53.71	55.05	53.40	40.97	44.64	44.27	36.40	36.35	32.80
DP-MIND	75.77	72.97	67.56	71.11	71.43	72.69	66.00	65.13	62.23

Appendix

Table 6: F1 comparison on RULER-HQA for Qwen2.5-14B series under

\epsilon=7

Method	7K	14K	28K	56K	112K	224K	448K	896K	1.8M
Qwen2.5-14B (baseline)	11.72	9.38	8.59	–	–	–	–	–	–
Sparse Attention	11.72	9.38	8.59	3.12	3.91	0.00	–	–	–
Long-Context Scaling	32.81	21.09	24.22	18.75	17.19	12.50	6.25	2.34	–
Memory-Augmented	37.50	36.72	34.38	25.78	28.91	31.25	22.66	23.44	19.53
DP-MIND	58.59	53.12	48.44	50.78	51.56	54.69	49.22	48.44	46.40

Appendix

Table 7: Exact Match (EM) comparison on RULER-HQA for Qwen2.5-14B series under

\epsilon=7

Method	7K	14K	28K	56K	112K	224K	448K	896K	1.8M
Qwen2.5-14B (baseline)	60.16	56.25	53.12	–	–	–	–	–	–
Sparse Attention	60.16	56.25	53.12	3.91	1.56	0.00	–	–	–
Long-Context Scaling	65.62	56.25	59.38	55.47	49.22	37.50	17.19	7.03	–
Memory-Augmented	56.25	60.16	55.47	44.53	53.12	46.09	36.72	39.84	33.59
DP-MIND	81.31	78.91	74.22	80.47	78.91	80.47	77.34	74.22	70.31

Appendix

Table 8: Sub-Exact Match (Sub-EM) comparison on RULER-HQA for Qwen2.5-14B series under

\epsilon=7

Privacy Budget	Method	32K			128K			512K
Privacy Budget	Method	Single $\uparrow$	M-Key $\uparrow$	M-Query $\uparrow$	Single $\uparrow$	M-Key $\uparrow$	M-Query $\uparrow$	Single $\uparrow$	M-Key $\uparrow$	M-Query $\uparrow$
$\epsilon=6$	Qwen2.5-7B	3.91	2.34	2.34	–	–	–	–	–	–
	Qwen2.5-7B-1M	3.12	3.91	3.12	3.91	5.47	3.12	0.00	0.00	0.39
	DP-MIND	5.47	7.03	3.32	4.69	5.47	4.10	5.47	3.12	3.32
$\epsilon=7$	Qwen2.5-7B	31.25	37.50	31.64	–	–	–	–	–	–
	Qwen2.5-7B-1M	36.72	39.06	36.72	32.03	33.59	16.80	19.53	3.91	2.93

Appendix

Table 9: NIAH results under different privacy budgets and context lengths. Results are reported on Single, Multi-Key (M-Key), and Multi-Query (M-Query) settings, where

\uparrow

indicates higher is better.

Explore similar work

Efficient DP-SGD for LLMs with Randomized Clipping
May 24, 2026Enayat Ullah, Sai Aparna Aketi, Devansh Gupta +2Privacy-Preserving Language Models Differentially Private Stochastic Gradient Descent
Meta Platforms Inc. · University of Southern California
SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models
Jun 3, 2026Peihua Mai, Xuanrong Gao, Youlong Ding +3Privacy-Preserving Language Models LLM Inference
1National University of Singapore (Chongqing) Research Institute · 2Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles, Chongqing, China · 3National University of Singapore +3
SpliTEE: Fast and Private LLM Inference by Coupling GPU-Assisted Trusted Execution Environments with Differential Privacy
Sep 14, 2026Shashie Dilhara Batan Arachchige, Robin Carpentier, Hassan Jameel Asghar +1Privacy-Preserving Language Models Split Inference
School of Computing, Macquarie University

May 24, 2026, cs.LGJ/K move · Enter open · S save

Efficient DP-SGD for LLMs with Randomized Clipping

Enayat Ullah, Sai Aparna Aketi, Devansh Gupta, Huanyu Zhang+1

Meta Platforms Inc. · University of Southern California

Privacy-Preserving Language Models Differentially Private Stochastic Gradient Descent LLM Fine-Tuning

Open paper AskPDF arXiv

Large language models (LLMs) are trained on vast datasets that may contain sensitive information. Differential privacy (DP), the de facto standard for formal privacy guarantees, provides a principled framework for training LLMs with provable privacy protection. However, state-of-the-art DP training implementations rely on fast gradient clipping techniques with memory overhead

O(B \min\{T^2, d^2\})

, where

B

is the batch size,

T

is the sequence length, and

d

is the model width. This becomes prohibitive as both model size and context length grow. We propose DP-SGD-RC, a novel variant of DP-SGD with randomized clipping that reduces memory and compute complexity. DP-SGD-RC leverages stochastic trace estimation methods, specifically Hutchinson's estimator[Hutchinson, 1989] and its improved variant, Hutch++[Meyer et al., 2021], to reduce the memory footprint of per-sample gradient norm estimation. We provide a tight privacy analysis showing that DP-SGD-RC achieves noise multipliers competitive with deterministic clipping. Experiments fine-tuning Llama~3.2-1B on long-context benchmarks spanning classification, question answering, and summarization tasks demonstrate that DP-SGD-RC matches baseline utility while significantly reducing memory and compute requirements.

Abstract

Explore similar work

Efficient DP-SGD for LLMs with Randomized Clipping

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models

SpliTEE: Fast and Private LLM Inference by Coupling GPU-Assisted Trusted Execution Environments with Differential Privacy