Learning to Refer: Client-Resolved Generation for Privacy-Aware Language Models

Jeongho Yoon; Chanhee Park; Yongchan Chun; Duong Tuan Thanh; Sungbin Han; Chanjun Park; Hyeonseok Moon; Heuiseok Lim

cs.CRSep 26, 2026

Learning to Refer: Client-Resolved Generation for Privacy-Aware Language Models

Authors: Jeongho Yoon, Chanhee Park, Yongchan Chun, Duong Tuan Thanh, Sungbin Han, Chanjun Park, Hyeonseok Moon, Heuiseok Lim

Organizations: Department of Computer Science and Engineering, Korea University · Konkuk University · Soongsil University · Sookmyung Women’s University

Abstract

Cloud-based large language models (LLMs) require users to disclose plaintext data to service providers, creating privacy risks in sensitive domains. Existing privacy-preserving approaches often trade utility for protection, incur substantial computational or communication overhead, remain vulnerable to reconstruction from intermediate representations, or protect only a subset of the training and inference pipeline. We introduce Client-Resolved Generation (CRG), a genera- tion interface that separates server-side generation from the lexical realization of input-derived content. The client transmits only pooled and noise-perturbed rep- resentations, while input-derived output content is represented using request-local positional references and resolved to its original strings only on the client. This interface protects private input and input-derived output content during both train- ing and inference while allowing the service provider to keep its proprietary model parameters hidden from the client. At the same time, exact lexical reuse remains possible without directly exposing the reused content on the provider-visible gen- eration path. We evaluate CRG on medical and document-grounded QA, sensi- tive identifier transfer, and tool calling, together with reconstruction and raw-logit leakage analyses. On SealTools, CRG improves complete-call exact match from 57.3% to 79.9% over the input-privacy framework PPFT, with larger gains as more required output content can be resolved through references. Together, these results show that CRG provides a practical interface for privacy-sensitive cloud LLMs by reducing plaintext exposure across both input and output pathways while preserv- ing task utility and server-side model confidentiality.

Figures & tables

Figure 1: Standard input protection may still expose input-derived sensitive content in plaintext outputs. Client-Resolved Generation ( CRG ) instead emits positional references from protected representations and resolves them only on the client.

Figure 2: Overview of CRG . Stage 1 learns reference-aware alignment on public data, and Stage 2 adapts the server to noisy client representations. At inference, only perturbed representations are transmitted, while server-generated references are resolved to plaintext exclusively on the client.

Method	NLICE	IDQuAD	OQA	Flashcards	BioASQ-F	BioASQ-L	Macro Avg.
AlienLM	58.77	58.37	37.32	36.88	17.24	0.00	34.76
PPFT ${}_{\mathrm{pool2},\,\epsilon=75}$	58.15	77.81	51.48	40.17	31.03	17.34	46.00
CRG ${}_{\mathrm{pool2},\,\epsilon=75}$	53.08	84.52	46.54	42.70	48.28	26.17	50.22
CRG ${}_{\mathrm{pool2},\,\mathrm{no\ noise}}$	73.23	88.85	58.58	44.45	55.17	30.60	58.48

Table 1: Medical utility benchmarks. Noise-free CRG serves as an upper-bound reference.

Method	SealTools	ToolACE	PolicyQA	MultiSpanQA	TyDiQA	PIIFORM	DOCPII	Macro Avg.
AlienLM	59.00	42.61	44.36	61.81	51.80	57.47	83.87	57.27
PPFT ${}_{\mathrm{pool2},\,\epsilon=75}$	57.32	53.41	45.71	76.06	66.86	32.70	74.13	58.03
CRG ${}_{\mathrm{pool2},\,\epsilon=75}$	79.92	46.59	42.96	72.79	69.32	66.80	92.00	67.20
CRG ${}_{\mathrm{pool2},\,\mathrm{no\ noise}}$	87.03	57.95	48.04	78.11	73.37	76.23	95.40	73.73

Table 2: Structured and agentic utility benchmarks. Noise-free CRG serves as an upper-bound reference.

Figure 3: CRG

-

PPFT utility difference as a function of reference coverage. Higher coverage corresponds to a larger fraction of the gold output that can be resolved through input references.

Table 3: Privacy attacks A–D on DDx-Plus at

\epsilon=75

. Attack A evaluates input reconstruction, Attack B evaluates output reconstruction, Attack C evaluates output-only inference, and Attack D measures residual plaintext salience in decoder logits.

Method	Attack	ROUGE-L $\downarrow$	Age $\downarrow$	Sex $\downarrow$	Symptom $\downarrow$	History $\downarrow$
CRG	Verbatim reconstruction	0.33	0.00	0.06	1.05	0.16
CRG	Patient-description disclosure	0.35	0.00	0.97	0.64	0.25
PPFT	Verbatim reconstruction	13.10	15.75	6.91	34.79	26.28
PPFT	Patient-description disclosure	35.57	95.29	65.07	57.72	62.36

Table 4: Prompt-injection stress test on DDx-Plus (

\epsilon=75

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Addressing	ToolACE	PolicyQA	MultiSpanQA	NLICE	IDQuAD	Flashcards
	EM	F1	F1	Acc.	F1	F1
Positional	46.59	42.96	72.79	53.08	84.52	42.70
Unique-word	44.89	44.02	73.02	58.31	83.61	42.77

Appendix

Table 5: Utility under positional and unique-word reference addressing. We report representative tool-calling, context-grounded QA, and medical QA benchmarks.

Setting	Adversary observes	Protected content
Network	Transmitted continuous representations and server outputs	Raw input text and plaintext values represented through references
Service provider	Continuous representations, reference-aware targets, generated outputs, and server-side computation	Raw client input and referenced input-derived lexical content
Logit diagnostic	All server-visible signals and decoder logits	Evaluates whether protected plaintext remains highly ranked internally
Client	Raw input, private position table, and resolved output	Server decoder parameters remain undisclosed

Appendix

Table 6: Adversary observations and the corresponding protection scope of CRG .

Attacker	ROUGE-L $\downarrow$	Age $\downarrow$	Sex $\downarrow$	Symptom $\downarrow$	History $\downarrow$
Stage 1 aligned	5.16	11.81 (+2.6)	4.58 (+2.1)	7.69 (+0.9)	4.67 (+3.9)
Stage 1 aligned w/o reference tokens	3.92	19.24 (+14.1)	8.78 (+2.8)	3.54 (+0.6)	1.14 (+1.0)

Appendix

Table 7: Input reconstruction of CRG by privileged Stage 1-aligned adversaries on DDx-Plus under the primary privacy setting (

\epsilon=75

Method	Token group	Events	Hit@2 $\downarrow$	Hit@5 $\downarrow$	Hit@10 $\downarrow$	Hit@20 $\downarrow$	Hit@50 $\downarrow$	Hit@100 $\downarrow$
AlienLM	All inverse-mapped tokens	183,124	4.99	12.73	20.88	28.32	39.27	47.20
AlienLM	Sensitive symptoms	3,265	0.83	6.62	12.07	23.25	30.32	34.76
AlienLM	Other single-token	114,582	3.70	10.85	20.01	27.88	40.36	48.44
CRG	All valid references	93,974	1.25	1.76	2.14	2.57	3.07	3.94
CRG	Sensitive symptoms	8,987	0.00	0.02	0.03	0.03	0.10	0.20
CRG	Other single-token	48,444	2.35	3.10	3.63	4.19	4.86	6.32

Appendix

Table 8: Full Attack D raw-logit results on Pri-DDXPlus at

\epsilon=75

. Hit@

K

reports the percentage of eligible generation events in which the designated plaintext token appears among the Top-

K

decoder candidates.

Dataset	$k$	$\epsilon$	20–40%	40–60%	60–80%
SealTools	2	75	$+0.00$	$+14.04$	$+67.44$
SealTools	2	150	$+0.00$	$+2.25$	$+53.49$
SealTools	4	75	$-11.11$	$+3.93$	$+65.12$
SealTools	4	150	$-5.56$	$+6.74$	$+37.21$
PIIFORM	2	75	$-37.84$	$+32.09$	$+50.33$
PIIFORM	2	150	$-39.19$	$+17.82$	$+32.90$

Appendix

Table 9: Utility difference between CRG and PPFT across reference-coverage bins. Values are complete-call EM differences in percentage points.

ID	Protected phrase / token	CRG rank	AlienLM rank
56	coughing up blood	23,099	2
182	chest pain	75,484	3
196	swollen lymph nodes	83,611	4

Appendix

Table 10: Illustrative raw-logit events.

Dataset	Train	Dev	Test
PII Form Filling	30,000	500	3,000
Document PII QA	30,000	500	3,000
Tool-call pool	39,932	1,000	4,000

Appendix

Table 11: Summary of the derived evaluation suites. The tool-call pool contains both identifier and general examples; its development and test sets contain 500/2,000 examples from each slice, respectively.

Method	Attack condition	ROUGE-L $\downarrow$	Sex $\downarrow$	Symptom $\downarrow$	History $\downarrow$
CRG	c0: No injection	0.43	0.32	0.40	0.60
	c1: Recover, decoder text	0.24	0.00	0.00	0.00
	c2: Recover, encoder clean	0.33	0.06	1.05	0.16
	c3: Recover, encoder noisy	0.40	0.39	1.57	0.41
	c4a: Paraphrase, encoder clean	0.35	0.97	0.64	0.25
	c4b: Paraphrase, encoder noisy	0.46	0.45	1.05	0.41

Appendix

Table 12: Prompt-injection stress test on DDx-Plus under the same

\epsilon=75

protected-input condition used during inference. All values except the condition labels are percentages. Higher values indicate greater recovery of private input content and therefore greater disclosure. CRG exposes little literal sensitive content across all tested attacks, whereas PPFT can be strongly redirected to verbalize private patient attributes, especially under the clean semantic-disclosure attack (c4a).

Attack condition	R-L $\downarrow$	Age lift $\downarrow$	Sex lift $\downarrow$	Symp. lift $\downarrow$	Hist. lift $\downarrow$
c0: No injection	0.22	+1.27	+0.32	+0.05	+0.56
c1: Recover, decoder text	1.97	+0.73	$-0.00$	$-0.02$	+0.12
c2: Recover, encoder clean	0.03	+0.45	+0.15	+1.28	+0.59
c3: Recover, encoder noisy	0.08	+1.29	+0.24	+0.25	+0.39
c4a: Paraphrase, encoder clean	1.08	+10.03	+2.30	+0.07	+0.00
c4b: Paraphrase, encoder noisy	0.59	+6.48	+2.09	+0.12	+0.10

Appendix

Table 13: Prompt-injection stress test after disabling the entire CRG reference vocabulary. Attribute columns report recovery lift over the deranged-record control (percentage points); lower is better. Even when reference generation is made impossible, symptom and medical-history recovery remain close to the shuffled baseline.

Client backend	Encoder	Representation	Request–ACK
	(ms)	ready (ms)	(ms)
Mac M4 CPU	115.9	117.1	180.4
Mac M4 MPS	53.7	60.6	148.2
Ryzen 7 7700 CPU	275.8	276.9	356.7
RTX 3050 CUDA	37.8	39.4	113.2

Appendix

Table 14: Mean FP32 client-side latency over 384 measured DDx-Plus requests. Representation-ready latency includes tokenization, encoder execution, pooling, required device transfers, and privacy-noise injection. Request–ACK measures the interval from the beginning of client processing until successful server receipt is validated.

Client backend	FP32	FP16	BF16
	Representation ready (ms)
Mac M4 CPU	117.1	503.4	706.5
Mac M4 MPS	60.6	63.5	63.3
Ryzen 7 7700 CPU	276.9	1417.4	99.5
RTX 3050 CUDA	39.4	34.3	34.8

Appendix

Table 15: Client representation-preparation latency under different encoder precisions. Only the encoder parameter/forward precision is varied; pooling, privacy noise, and transmission remain FP32.

Method	NLICE	IDQuAD	OQA	Flashcards	BioASQ-F	BioASQ-L	Macro Avg.
CRG ${}_{\mathrm{pool2},\,\epsilon=75}$	53.08	84.52	46.54	42.70	48.28	26.17	50.22
CRG ${}_{\mathrm{pool2},\,\epsilon=75}^{\mathrm{all\text{-}ref}}$	18.62	82.38	41.68	39.76	37.93	24.94	40.89

Appendix

Table 16: Medical utility when Stage 2 is trained with all-reference output supervision. The standard CRG model selectively uses references for input-addressable words, whereas the all-reference variant strongly suppresses lexical plaintext supervision in Stage 2 targets. All values are percentages.

Method	SealTools	ToolACE	PolicyQA	MultiSpanQA	TyDiQA	PIIFORM	DOCPII	Macro Avg.
CRG ${}_{\mathrm{pool2},\,\epsilon=75}$	79.92	46.59	42.96	72.79	69.32	66.80	92.00	67.20
CRG ${}_{\mathrm{pool2},\,\epsilon=75}^{\mathrm{all\text{-}ref}}$	78.66	46.02	43.41	71.42	68.03	65.37	91.87	66.40

Appendix

Table 17: Structured and agentic utility under all-reference Stage 2 supervision. The intervention has a limited effect on tasks whose outputs can largely be resolved from the input, but becomes more costly when correct predictions require lexical content absent from the query. All values are percentages.

	Utility $\uparrow$				Raw plaintext leakage $\downarrow$
$K$	SealTools	ToolACE	PolicyQA	MultiSpanQA	SealTools	ToolACE
	EM	EM	F1	F1	(%)	(%)
200	79.50	48.30	43.20	71.85	2.09	5.97
100	76.99	47.16	43.24	71.03	3.35	10.23
50	78.66	42.90	42.17	65.82	3.35	22.44
25	47.28	40.06	41.52	60.32	60.67	54.55

Appendix

Table 18: Effect of the addressable reference window size

K

under the fixed CRG configuration (pooling

=2

\epsilon=75

). Utility is measured using the native benchmark metrics. Raw plaintext leakage denotes the fraction of tool-call examples for which a gold argument value appears verbatim in the server-visible output.

Privacy-Preserving Language Models

Privacy-preserving language models protect sensitive data during training, inference, personalization, and information sharing.

See all 179 papers

Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models
Feb 27, 2026Chung-ju Huang, Huiqiang Zhao, Yuanpeng He +5Privacy-Preserving Language Models Privacy Leakage in Language Models
Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, China · School of Computer Science, Peking University, Beijing, China · Tencent, Shenzhen, China +1
Ask Without Telling: Local SLMs Consult Cloud LLMs Without Revealing Task Intent
Sep 26, 2026Yanmeng Wang, Yunxuan Li, Shilong Fan +2Privacy-Preserving Language Models Privacy Protection for LLM Agents
Nanjing University of Posts and Telecommunications, Nanjing, 210023, China · The Chinese University of Hong Kong, Shenzhen, 518172, China
SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models
Jun 3, 2026Peihua Mai, Xuanrong Gao, Youlong Ding +3Privacy-Preserving Language Models LLM Inference
1National University of Singapore (Chongqing) Research Institute · 2Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles, Chongqing, China · 3National University of Singapore +3

Feb 27, 2026J/K move · Enter open · S save

Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models

Chung-ju Huang, Huiqiang Zhao, Yuanpeng He, Lijian Li+4

Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, China · School of Computer Science, Peking University, Beijing, China · Tencent, Shenzhen, China+1

Privacy-Preserving Language Models Privacy Leakage in Language Models

Open paper AskPDF arXiv

The increasing reliance on cloud-hosted Large Language Models (LLMs) exposes sensitive client data, such as prompts and responses, to potential privacy breaches by service providers. Existing approaches fail to ensure privacy, maintain model performance, and preserve computational efficiency simultaneously. To address this challenge, we propose Talaria, a confidential inference framework that partitions the LLM pipeline to protect client data without compromising the cloud's model intellectual property or inference quality. Talaria executes sensitive, weight-independent operations within a client-controlled Confidential Virtual Machine (CVM) while offloading weight-dependent computations to the cloud GPUs. The interaction between these environments is secured by our Reversible Masked Outsourcing (ReMO) protocol, which uses a hybrid masking technique to reversibly obscure intermediate data before outsourcing computations. Extensive evaluations show that Talaria can defend against state-of-the-art token inference attacks, reducing token reconstruction accuracy from over 97.5% to an average of 1.34%, all while being a lossless mechanism that guarantees output identical to the original model without significantly decreasing efficiency and scalability. To the best of our knowledge, this is the first work that ensures clients' prompts and responses remain inaccessible to the cloud, while also preserving model privacy, performance, and efficiency.

Abstract

Privacy-Preserving Language Models

Explore similar work

Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models

Ask Without Telling: Local SLMs Consult Cloud LLMs Without Revealing Task Intent

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models