LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communication

Yuzhu Mao; Liang Zhao

cs.CRSep 19, 2026

LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communication

Authors: Yuzhu Mao, Liang Zhao

Organizations: Emory University

Abstract

As Large Language Model (LLM) APIs become increasingly integrated into privacy-sensitive workflows, ensuring inference-time privacy without compromising task utility remains a major challenge. Existing approaches preserve most of the original semantic content to maintain downstream performance, but this also leaves exploitable cues for reconstructing the original text. This work investigates semantic decoupling, which replaces original semantics with alternative content while preserving the structure needed for LLM reasoning. Based on this idea, we propose CROSS-MAP, a bidirectional framework that maps private inputs into a different semantic domain before inference and recovers the corresponding outputs afterward. Local models are trained with multi-objective optimization to maximize semantic divergence in the mapping stage while minimizing semantic inconsistency in the recovery stage. Experiments show that CROSS-MAP reduces reconstruction success across multiple attack settings while outperforming existing baselines in utility.

Figures & tables

Symbol	Meaning
$x,y,z$	original text, mapped text, and LLM response
$x_{s},x_{c}$	semantic and structural components of $x$
$\lambda_{s},\lambda_{c}$	task dependence on semantic and structural information, respectively
$\mathcal{M}_{\mathrm{SD}},\mathcal{M}_{\mathrm{SP}}$	semantic-decoupling /semantic-preserving mapping mechanisms
$\Delta\rho_{s},\Delta\rho_{c}$	how much less semantic and structural information from the original text is preserved by $\mathcal{M}_{\mathrm{SD}}$ than by $\mathcal{M}_{\mathrm{SP}}$ in the mapped text (i.e., $\mathcal{M}_{\mathrm{SD}}$ ’s compression gain)
$\Delta\eta_{s},\Delta\eta_{c}$	how much less task-relevant semantic and structural information is preserved by $\mathcal{M}_{\mathrm{SD}}$ than by $\mathcal{M}_{\mathrm{SP}}$ in the mapped text (i.e., $\mathcal{M}_{\mathrm{SD}}$ ’s task-relevant information loss)

Table 1: Core notation used in the main text. Other notations used in the proof are defined in Appendix B .

Figure 1: Illustration of inference-time protection with CROSS-MAP.

Figure 2: Privacy-utility trade-offs across methods and datasets. The round points denote the median of subpoints representing individual datasets.

Task	Method	ROUGE-L	BERT/Cov.	Retention
CNN/DM	Plain LLM	0.240	0.882	100.0%
CNN/DM	CROSS-MAP	0.285	0.877	7.3%
CommonGen	Plain LLM	0.587	0.932	98.3%
CommonGen	CROSS-MAP	0.414	0.891	18.9%

Table 2: Generalization on summarization and generation tasks. The BERT/Cov. metric denotes BERTScore for summarization and concept coverage for generation. Retention represents the sensitive-span retention rate.

Figure 3: De-anonymization results via LLM attribute inference on SynthPAI ( Yukhymenko et al., 2024 ) , comparing attacker inference accuracy and certainty across methods.

Dataset	Method	$k=10$		$k=30$		$k=50$
		ASR	Post. Mass	ASR	Post. Mass	ASR	Post. Mass
FinCausal 2025 Moreno-Sandoval et al. (2025)	CROSS-MAP	0.37 $\%$	0.006	0.84 $\%$	0.003	1.19 $\%$	0.002
	HAS	5.88 $\%$	0.116	35.29 $\%$	0.026	35.29 $\%$	0.026
	Plain QA	7.86 $\%$	0.181	41.43 $\%$	0.088	41.43 $\%$	0.088
BioASQ Tsatsaronis et al. (2015)	CROSS-MAP	1.05 $\%$	0.004	1.05 $\%$	0.004	1.05 $\%$	0.004
	HAS	12.73 $\%$	0.279	28.14 $\%$	0.164	34.66 $\%$	0.135

Table 3: Black-box span restoration results across datasets, reporting span restoration accuracy as ASR and the attacker’s posterior mass on the ground-truth span at different candidate list sizes

k

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Examples of semantic-preserving and semantic-decoupling mappings under posterior sampling attacks. Semantic decoupling prevents correct inferences by stripping contextual cues from the source domain.

		Downstream QA utility			Recovered text utility				Mapped text privacy
Dataset	Method	EM $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	Flu. Loss $\downarrow$	Sim. $\uparrow$	BLEU $\uparrow$	R-1 $\uparrow$	$c_{\text{dict}}\uparrow$	$d_{\text{dict}}\uparrow$	$d_{\text{text}}\uparrow$
SQuAD	Plain QA	76.42	84.10	94.73	4.202	1.000	1.000	1.000	0.000	0.000	0.000
	Pr $\varepsilon$ $\varepsilon$ mpt	75.78	84.02	94.70	4.365	1.000	1.000	1.000	0.006	0.142	0.006
	InferDPT	68.21	74.04	81.34	5.597	0.737	0.520	0.710	0.172	0.289	0.263
	HAS	72.97	82.10	91.84	4.212	0.973	0.939	0.975	0.181	0.408	0.238
	CROSS-MAP	74.94	83.22	92.98	4.703	0.826	0.586	0.800	0.413	0.753	0.752

Appendix

Table 4: Privacy and utility results across datasets for Plain QA, Pr

\varepsilon

\varepsilon

mpt ( Chowdhury et al., 2025 ) , InferDPT ( Tong et al., 2025b ) , HAS ( Chen et al., 2023b ) , and CROSS-MAP. The results are reported using Qwen2.5-14B as the base model for both the mapper and the recoverer in CROSS-MAP. An LLAMA-8B model is used for downstream QA.

Method	Plaus. $\uparrow$	Fluency $\uparrow$	Detect. $\downarrow$
Original text	4.93	4.91	1.06
Pr $\varepsilon$ $\varepsilon$ mpt	4.82	4.86	1.18
HAS	4.54	4.61	1.58
InferDPT	3.71	3.94	2.47
CROSS-MAP	4.45	4.52	1.72

Appendix

Table 5: LLM-as-a-judge ratings across methods.

Method	Inference Accuracy	Inference Certainty
Plain Text	0.713	3.366
Pr $\varepsilon$ $\varepsilon$ mpt	0.701	3.152
HAS	0.470	2.196
GPT-AA	0.423	2.073
CROSS-MAP	0.082	1.063

Appendix

Table 6: De-anonymization results via LLM attribute inference on SynthPAI ( Yukhymenko et al., 2024 ) , reporting attacker inference accuracy and confidence (0-5).

Figure 5: Correlation between span restoration accuracy and the attacker’s posterior mass on protected spans.

Figure 6: Optimization-based attacker’s restoration rate under different dictionary leakage ratios

\rho

Split	MDD	Max depth	Clauses/sent.	ACC $\uparrow$	Flu. Loss $\downarrow$
Q1	2.16	11.11	2.27	93.62%	4.722
Q2	2.40	10.75	2.34	90.52%	4.687
Q3	2.65	9.85	2.53	93.36%	4.715
Q4	3.24	10.11	2.90	94.42%	4.688

Appendix

Table 7: SQuAD complexity-quartile robustness. Examples are sorted by MDD and split into Q1–Q4, where Q4 is most complex.

Setting	ACC	Drop
Original output	0.929	–
WordNet synonyms	0.877	5.2%
T5 paraphrase	0.896	3.3%

Appendix

Table 8: Paraphrase robustness of the recoverer on SQuAD. The recoverer reasons over the dictionary and response jointly, rather than relying on exact string matching.

Pairing	$d_{\mathrm{text}}$	ACC
Within-cluster	0.642	90.4%
Across-cluster	0.797	86.9%

Appendix

Table 9: Target-domain distance ablation over 260 source-target pairs. Source and target domains are embedded with Sentence-BERT and grouped into semantic-domain clusters.

Mapping	Text
Original: cognitive development	Wisdom , or the capacity for insight and judgment that is developed through experience , increases between the ages of fourteen and twenty-five , then levels off. Thus, it is during the adolescence-adulthood transition that individuals acquire the type of wisdom typically associated with age. Wisdom is not the same as intelligence : adolescents do not improve substantially on IQ tests because their scores are relative to others in their same age group , and relative standing usually does not change — everyone matures at approximately the same rate in this way.
Far domain: equipment operations	Diagnostic maturity , or the ability to interpret sensor patterns and make maintenance decisions refined through operating hours , improves between 300 and 1,800 operating hours , then plateaus. Thus, units tend to develop the level of diagnostic maturity typically attributed to long service life during the transition from commissioning to routine operation . Diagnostic maturity is not the same as rated capacity : newly commissioned units do not show large gains on standardized bench tests because their results are normalized against comparable models from the same production batch , and relative ranking typically remains stable — most units stabilize at roughly similar rates under similar duty cycles on that metric.
Near domain: medical training	Professional judgment , or the capacity for clinical insight and decision-making that is developed through supervised practice , increases between the first and final years of training , then levels off. Thus, it is during the trainee-practitioner transition that clinicians acquire the type of professional judgment typically associated with seniority. Professional judgment is not the same as medical knowledge : trainees do not improve substantially on standardized exams because their scores are relative to others in their same training cohort , and relative standing usually does not change — everyone advances at approximately the same rate in this way.

Appendix

Table 10: Domain-selection case study. Bold spans mark key source entities and their mapped counterparts.

Dictionary coverage	Recovery Sim.	ACC
$c_{\mathrm{dict}}\approx 0.8$	0.860	91.2%
$c_{\mathrm{dict}}\approx 0.5$	0.816	88.7%
$c_{\mathrm{dict}}\approx 0.2$	0.788	86.2%

Appendix

Table 11: Dictionary-coverage ablation on SQuAD.

Checkpoint	# samples	$d_{\mathrm{text}}$	Sim. / ACC
Base	0	0.541	0.901 / 88.96
SFT-200	200	0.634	0.912 / 90.38
SFT-500	500	0.651	0.933 / 92.57
SFT-700	700	0.663	0.945 / 94.59
SFT-1000	1000	0.667	0.943 / 94.34
SFT-1500	1500	0.670	0.940 / 94.35

Appendix

Table 12: SFT training-data scaling study. Gains largely saturate around 700–1000 samples.

Figure 7: Ablation study of model scale and training objectives on utility–privacy trade-off. Dashed lines connect checkpoints within the same model size.

\text{DPO}\_3

\text{DPO}\_2

, and

\text{DPO}\_1

denote DPO checkpoints from epochs 1, 3, and 5, respectively, showing increasingly aggressive preference optimization toward privacy-oriented mapping.

Setting	Dataset	Model / pipeline	EM	F1
Local only	SQuAD	Qwen-2.5-3B local	33.82	42.77
External, no protection	SQuAD	Qwen-2.5-14B external	54.28	62.51
CROSS-MAP	SQuAD	3B mapper/recoverer + 14B external	47.71	55.96
Local only	BioASQ	Qwen-2.5-14B local	12.63	23.35
External, no protection	BioASQ	GPT-5 external	24.98	46.43
CROSS-MAP	BioASQ	14B mapper/recoverer + GPT-5 external	22.36	44.57

Appendix

Table 13: External-LLM ablation. CROSS-MAP preserves most of the stronger external model’s utility while keeping original-domain sensitive content local.

$\rho$	Base attacker (mean)	DPO attacker (mean)	Gain (mean)	DPO attacker (best)	Std. (mean)
0.1	0.287	0.326	0.039	0.397	0.033
0.3	0.373	0.423	0.050	0.494	0.036
0.5	0.536	0.606	0.070	0.679	0.040
0.7	0.673	0.758	0.085	0.831	0.043
0.9	0.773	0.867	0.094	0.935	0.046

Appendix

Table 14: Performance of optimization-based white-box attacker on SQuAD, measuring similarity between reconstructed text and original text across leakage ratios

\rho

$\rho$	Base attacker (mean)	DPO attacker (mean)	Gain (mean)	DPO attacker (best)	Std. (mean)
0.1	0.187	0.242	0.055	0.295	0.039
0.3	0.286	0.351	0.065	0.418	0.045
0.5	0.447	0.525	0.078	0.607	0.049
0.7	0.602	0.687	0.085	0.772	0.051
0.9	0.694	0.783	0.089	0.832	0.046

Appendix

Table 15: Performance of optimization-based white-box attack on NarrativeQA, measuring similarity between reconstructed text and original text across leakage ratios

\rho

	Downstream QA utility			Mapping quality
Method	EM	F1	ACC	Sim.	$d_{\text{dict}}$	$d_{\text{text}}$	$r_{\text{json}}$
Plain QA	76.4	84.10	94.73	1.000	0.000	0.000	1.000
DPO_1 (14B)	74.94	83.22	92.98	0.826	0.753	0.752	1.000
DPO_2 (14B)	75.30	83.31	94.21	0.892	0.711	0.735	1.000
DPO_3 (14B)	75.72	83.75	94.34	0.898	0.704	0.698	1.000
SFT-only (14B)	75.88	83.82	94.62	0.948	0.612	0.668	0.936

Appendix

Table 16: Ablation study on SQuAD, comparing DPO-tuned and SFT-only models of different sizes in terms of QA utility, privacy distances, and JSON-structure validity. LLAMA-8B model is used for downstream QA.

Figure 8: Mapper prompt.

Figure 9: Recoverer prompt.

Method	Original text	Mapped text	Recovered text
CROSS-MAP	Fiscal pressures on Government as a result of reduced revenues due to lower oil prices is evident in the increasing number of audits being undertaken to identify areas of non-compliance . The Group remains committed to maintaining the highest levels of compliance and works closely with local regulatory authorities .	Operational strain on the hospital network as a result of declining patient volume due to reduced insurance reimbursements is evident in the increasing number of quality assurance reviews being undertaken to identify deviations from clinical protocols . The hospital network remains committed to adhering strictly to regulatory standards and works closely with health department oversight bodies .	Fiscal pressures on the Government as a result of reduced revenues due to lower oil prices is evident in the increasing number of audits being undertaken to identify non-compliance with regulatory standards. The Government remains committed to maintaining the highest levels of compliance and works closely with local regulatory authorities .
HAS ( Chen et al., 2023b )	Fiscal pressures on Government as a result of reduced revenues due to lower oil prices is evident in the increasing number of audits being undertaken to identify areas of non-compliance. The Group remains committed to maintaining the highest levels of compliance and works closely with local regulatory authorities.	Fiscal pressures on Administration as a result of reduced revenues due to lower oil prices is evident in the increasing number of audits being undertaken to identify areas of non-compliance. The Committee remains committed to maintaining the highest levels of compliance and works closely with local regulatory authorities.	Fiscal pressures on Government as a result of reduced revenues due to lower oil prices is evident in the increasing number of audits being undertaken to identify areas of non-compliance. The Committee remains committed to maintaining the highest levels of compliance and works closely with local regulatory authorities.
InferDPT ( Tong et al., 2025b )	In 2018 the majority of employees across the Group have received average salary increases ranging from 2.0 $\%$ –3.25 $\%$ , dependent on geographical location with the principal exception being those employees based in Brazil, Latin America and China where, due to inflation, current market salary increases are higher.	In 2019 the majority of employees across the Company have received average salary rises ranging from 2.0 $\%$ –3.25 $\%$ , dependent on geographical location with the principal exception being those employees based in Brazil, Latin America and China where, due to inflation, current market salary rises are higher.	In 2019 the majority of employees across the Company have received average salary rises ranging from 2.0 $\%$ –3.25 $\%$ , dependent on geographical location with the principal exception being those employees based in Brazil, Latin America and China where, due to inflation, current market salary rises are higher.
Pr $\varepsilon$ $\varepsilon$ mpt ( Chowdhury et al., 2025 )	The Committee also reviewed the impact of the reduction in US federal tax rates as a result of tax reform in the US, which resulted in a reduction of deferred tax balances of $617$ million.	The Committee also reviewed the impact of the reduction in US federal tax rates as a result of tax reform in the US, which resulted in a reduction of deferred tax balances of $593$ million.	The Committee also reviewed the impact of the reduction in US federal tax rates as a result of tax reform in the US, which resulted in a reduction of deferred tax balances of $617$ million.

Appendix

Table 17: Examples of different mapping methods.

Privacy-Preserving Language Models

Privacy-preserving language models protect sensitive data during training, inference, personalization, and information sharing.

See all 178 papers

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models
Jun 3, 2026Peihua Mai, Xuanrong Gao, Youlong Ding +3Privacy-Preserving Language Models LLM Inference
1National University of Singapore (Chongqing) Research Institute · 2Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles, Chongqing, China · 3National University of Singapore +3
Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation
Apr 8, 2026Jeongho Yoon, Chanhee Park, Yongchan Chun +2Privacy-Preserving Language Models Privacy-Preserving ML
Department of Computer Science and Engineering, Korea University · Samsung Mobile eXperience Business
Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)
Mar 2, 2026Yu Lin, Qizhi Zhang, Wenqiang Ruan +6Privacy-Preserving Language Models LLM Inference
ByteDance · Nanjing University

Jun 3, 2026J/K move · Enter open · S save

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models

Peihua Mai, Xuanrong Gao, Youlong Ding, Xianglong Du+2

1National University of Singapore (Chongqing) Research Institute · 2Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles, Chongqing, China · 3National University of Singapore+3

Privacy-Preserving Language Models LLM Inference Privacy Leakage in Language Models Efficient Language Model Inference

Open paper AskPDF arXiv

With the widespread deployment of public large language models (LLMs) such as ChatGPT, protecting user prompt privacy has become an increasingly critical issue. Existing privacy-preserving inference methods sacrifice either utility or efficiency, and often require model-specific modifications that limit their compatibility. In this paper, we propose SharedRequest, a model-agnostic framework for privacy-preserving LLM inference that reformulates privacy protection at the batch level rather than the individual-prompt level. The key idea is to obscure sensitive information by mixing original prompts with noisy variants, while grouping semantically equivalent instructions to amortize the inference cost over a large batch of queries with minimal impact on LLM response quality. This design is independent of the LLM architecture, requiring no access to model parameters or architectural modification. Empirical results demonstrate that SharedRequest achieves over

20\%

higher utility compared to prior differential privacy baselines, and its shared-prompt mechanism reduces query cost by up to

5\times

compared to non-batched inference.

Abstract

Privacy-Preserving Language Models

Explore similar work

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models

Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation

Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)