Towards Privacy-Preserving Machine Translation at the Inference Stage: A New Task and Benchmark

Wei Shao; Lemao Liu; Yinqiao Li; Guoping Huang; Shuming Shi; Linqi Song

cs.CLMar 16, 2026

Towards Privacy-Preserving Machine Translation at the Inference Stage: A New Task and Benchmark

Authors: Wei Shao, Lemao Liu, Yinqiao Li, Guoping Huang, Shuming Shi, Linqi Song

Organizations: City University of Hong Kong · City University of Hong Kong Shenzhen Research Institute · Tencent AI Lab, Shenzhen

Abstract

Current online translation services require sending user text to cloud servers, posing a risk of privacy leakage when the text contains sensitive information. This risk hinders the application of online translation services in privacy-sensitive scenarios. One way to mitigate this risk for online translation services is introducing privacy protection mechanisms targeting the inference stage of translation models. However, compared to subfields of NLP like text classification and summarization, the machine translation research community has limited exploration of privacy protection during the inference stage. There is no clearly defined privacy protection task for the inference stage, dedicated evaluation datasets and metrics, and reference benchmark methods. The absence of these elements has seriously constrained researchers' in-depth exploration of this direction. To bridge this gap, this paper proposes a novel "Privacy-Preserving Machine Translation" (PPMT) task, aiming to protect the private information in text during the model inference stage. For this task, we constructed three benchmark test datasets, designed corresponding evaluation metrics, and proposed a series of benchmark methods as a starting point for this task. The definition of privacy is complex and diverse. Considering that named entities often contain a large amount of personal privacy and commercial secrets, we have focused our research on protecting only the named entity's privacy in the text. We expect this research work will provide a new perspective and a solid foundation for the privacy protection problem in machine translation.

Figures & tables

Fig. 1: This figure illustrates the workflow of an online translation service. Users are required to transfer the text to be translated from a safe internal environment to the online translation system located in cloud server. The red-colored content (”苹果-Apple”, ”周五-Friday”) in the text represents the privacy information, which constitutes sensitive business information. In sensitive scenarios, users do not want this text, which contains business information, to leave the secure internal environment (typically the company’s internal network). However, the current service mode of online translation systems makes it challenging to avoid this issue.

Example 1		The United Nations International Atomic Energy Agency ( IAEA ) released a report on the 26th stating that Iran
		has started to use advanced centrifuges to accelerate the production of enriched uranium , further violating the nuclear agreement reached with world powers in 2015 .
Example 2		Warren Buffett to sell one million shares of Google Inc next Monday .
Example 3		The Hong Kong Monetary Authority will introduce a new securities trading policy on 1 April and will hold a press conference at 3 pm on the same day to explain the new policy .

TABLE I: Cases of texts including privacy information. The red parts in listed sentences are privacy-related terms in our definition.

	Language	Total Sentences	Person Name	Location	Organization	Time	Quantity
WMT18	Zh	1928	2053	1551	1333	1347	1307
WMT18	En	1928	1960	1516	1306	1211	1188
WMT20	Zh	1964	1088	1813	1230	1372	1261
WMT20	En	1964	1068	1751	1215	1245	1139
Finance	Zh	1744	218	316	1949	1405	988
Finance	En	1744	197	300	1889	1342	930

TABLE II: The statistics of human-annotated test datasets, including the number of total sentences and the number of privacy entities with different entity types.

Fig. 2: This is our proposed framework. In this work, we use the client and cloud server to indicate safe internal and risky external environments. The sanitized sentence is sent to the server from the client and the client receive the translation of sanitized text. R1 and R2 represent two different placeholders of two privacy terms ”苹果” and ”周五” assigned by privacy sanitization module.

Fig. 3: A case of replacement with the dictionary-based placeholder. The privacy parts are in yellow.

Fig. 4: A case of replacement with the tag-based placeholder. The privacy parts are in yellow.

Fig. 5: A case of how to obtain the privacy information’s translation according to a phrase translation table.

Placeholder	Translation Model	WMT18								WMT20								Finance
		Zh $\Rightarrow$ En				En $\Rightarrow$ Zh				Zh $\Rightarrow$ En				En $\Rightarrow$ Zh				Zh $\Rightarrow$ En				En $\Rightarrow$ Zh
		SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER
-	Trans	22.80	51.40	22.61	-	18.60	29.50	18.18	-	27.00	55.50	27.90	-	19.20	33.10	19.06	-	47.40	71.20	48.19	-	39.60	47.30	38.57	-
Entity	Trans (3 types)	19.50	44.10	19.33	0	15.80	25.70	15.62	0	23.90	49.30	24.72	0	16.80	27.30	16.69	0	44.70	67.40	46.57	0	37.00	44.50	36.82	0
Entity	Trans (5 types)	18.90	42.80	18.76	0	14.70	23.90	14.54	0	23.40	48.30	24.13	0	16.20	26.20	16.01	0	43.80	65.90	45.53	0	35.70	42.90	35.46	0
Dictionary	Dict-Trans (3 types)	21.40	48.40	21.20	0	17.30	27.90	17.08	0	25.70	52.90	26.50	0	18.90	33.10	18.77	0	47.00	70.70	48.87	0	38.90	46.80	38.78	0

TABLE III: Comparison results of explicit privacy preserving.

Placeholder	Translation Model	WMT18								WMT20								Finance
		Zh $\Rightarrow$ En				En $\Rightarrow$ Zh				Zh $\Rightarrow$ En				En $\Rightarrow$ Zh				Zh $\Rightarrow$ En				En $\Rightarrow$ Zh
		SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER
-	Trans	22.80	51.40	22.61	-	18.60	29.50	18.18	-	27.00	55.50	27.90	-	19.20	33.10	19.06	-	47.40	71.20	48.19	-	39.60	47.30	38.57	-
Entity	Trans (3 types)	19.80	44.20	19.72	0.1851	15.70	25.90	15.59	0.2477	24.90	49.40	24.83	0.1859	16.90	27.30	16.81	0.2272	47.20	67.50	47.14	0.1587	37.20	44.60	37.11	0.2004
Entity	Trans (5 types)	19.40	43.30	19.27	0.1913	15.10	24.80	15.01	0.2579	24.30	48.20	24.22	0.1913	16.20	26.10	16.14	0.2355	46.10	66.00	46.03	0.1641	36.00	43.10	35.91	0.2066
Dictionary	Dict-Trans (3 types)	21.50	48.60	21.41	0.1845	17.50	28.10	17.23	0.2485	26.80	53.00	26.68	0.1865	18.90	33.10	18.86	0.2280	49.10	70.80	48.99	0.1595	39.00	46.80	38.92	0.2010

TABLE IV: Comparison results of implicit privacy preserving.

Placeholder	Translation Model	WMT18										Finance
		Zh $\Rightarrow$ En					En $\Rightarrow$ Zh					Zh $\Rightarrow$ En					En $\Rightarrow$ Zh
		SacreBLEU	chrF	BLEU	PER	User Latency	SacreBLEU	chrF	BLEU	PER	User Latency	SacreBLEU	chrF	BLEU	PER	User Latency	SacreBLEU	chrF	BLEU	PER	User Latency
-	Trans	22.80	51.40	22.61	-	-	18.60	29.50	18.18	-	-	47.40	71.20	48.19	-	-	39.60	47.30	38.57	-	-
Dictionary	Dict-Trans (5 types)	21.10	47.90	21.04	0.1925	28.9s	16.90	27.10	16.86	0.2585	24.5s	47.90	69.20	47.82	0.1645	26.1s	38.10	45.40	37.79	0.2075	21.9s
Tag	Tag-Trans (5 types)	20.90	47.20	20.88	0.1835	21.6s	17.00	27.30	16.77	0.2450	19.7s	47.80	69.00	47.71	0.1535	19.3s	37.90	45.20	37.67	0.1950	17.8s
-	PowerSoftmax	17.40	39.10	17.31	0	58.2s	14.20	23.40	14.12	0	47.7s	43.40	64.80	43.36	0	50.2s	35.20	42.60	35.14	0	44.9s

TABLE V: Comparison results of our framework and other baseline methods in implicit privacy preserving. We conduct experiments on WMT18 and Finance 5 types datasets. The user latency is the cost time of user side’s computation module for processing the full test dataset.

Translation Model	WMT18						WMT20						Finance
	Zh $\Rightarrow$ En			En $\Rightarrow$ Zh			Zh $\Rightarrow$ En			En $\Rightarrow$ Zh			Zh $\Rightarrow$ En			En $\Rightarrow$ Zh
	SacreBLEU	chrF	BLEU	SacreBLEU	chrF	BLEU	SacreBLEU	chrF	BLEU	SacreBLEU	chrF	BLEU	SacreBLEU	chrF	BLEU	SacreBLEU	chrF	BLEU
dict-trans (3 types)	21.30	48.50	21.20	17.10	27.90	17.08	26.60	53.00	26.50	18.80	33.10	18.77	48.90	70.80	48.87	38.80	46.80	38.78
trans (3 types)	20.40	46.20	20.29	16.30	26.40	16.23	25.50	50.50	25.40	17.90	30.90	17.81	47.80	68.70	47.73	37.70	45.10	37.62
dict-trans(5 types)	21.00	47.40	20.81	16.50	27.10	16.45	26.10	52.10	26.01	18.00	31.70	17.98	47.70	69.20	47.69	37.70	45.40	37.63
trans (5 types)	20.00	45.40	19.96	15.60	25.80	15.59	25.20	50.20	25.14	17.20	29.80	17.13	46.90	67.50	46.82	37.00	44.30	36.90

TABLE VI: Translation Performance of Trans and Dict-Trans models used to process dictionary-based placeholder.

Placeholder	Translation Model	Implicit Privacy Preserving								Explicit Privacy Preserving
		Zh $\Rightarrow$ En				En $\Rightarrow$ Zh				Zh $\Rightarrow$ En				En $\Rightarrow$ Zh
		SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER
-	Trans	27.00	55.50	27.90	-	19.20	33.10	19.06	-	27.00	55.50	27.90	-	19.20	33.10	19.06	-
Dictionary	Dict-Trans w/o DA (3 types)	26.20	52.00	26.13	0.1892	18.50	32.20	18.39	0.2298	26.10	51.90	26.03	0	18.30	32.10	18.26	0
Dictionary	Dict-Trans w/o DA (5 types)	25.60	51.10	25.54	0.1958	17.70	30.80	17.65	0.2382	25.50	51.00	25.48	0	17.50	30.70	17.42	0
Dictionary	Dict-Trans (3 types)	26.80	53.00	26.68	0.1865	18.90	33.10	18.86	0.2280	25.70	52.90	26.50	0	18.90	33.10	18.77	0

TABLE VII: The performance of translation models trained with and w/o data augmentation on the WMT20 dataset.

	Method	WMT18 Zh $\Rightarrow$ En	WMT18 En $\Rightarrow$ Zh	Finance Zh $\Rightarrow$ En	Finance En $\Rightarrow$ Zh
Stanford CoreNLP	Entity (3 types)	0.1766	0.2508	0.1672	0.2114
	Entity (3 types)	0.1822	0.2611	0.1699	0.2154
	Dict (3 types)	0.1905	0.2498	0.1623	0.2087
	Dict (5 types)	0.1935	0.2377	0.1677	0.2132
	Tag (3 types)	0.1912	0.2345	0.1493	0.1934
	Tag (5 types)	0.1866	0.2465	0.1567	0.1978

TABLE VIII: PER Results on WMT18 and financial datasets with different Named Entity Recognizers. Multiple Recognizer Fusion means that we use the merge results from TexSmart, Stanford CoreNLP, MicroSoft Recognizers-Text.

Translation Mode	Case Type	Finance 5 types
		Zh $\Rightarrow$ En				En $\Rightarrow$ Zh
		SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER
Trans	-	47.40	71.20	48.19	-	39.60	47.30	38.57	-
Dict-Trans	No NER Error	46.90	69.20	47.82	0	37.80	45.40	37.79	0
Dict-Trans	NER Error	47.20	69.50	48.07	0.2533	38.20	45.70	38.14	0.2756
Tag-Trans	No NER Error	46.80	69.00	47.71	0	37.70	45.20	37.67	0

TABLE IX: NER Error’s Influence to the Translation Performance and PER on the Fintech (5 types) dataset

Placeholder	Translation Model + Recognizer	Finance 10 types								Finance CP
		Zh $\Rightarrow$ En				En $\Rightarrow$ Zh				Zh $\Rightarrow$ En				En $\Rightarrow$ Zh
		SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER	SacreBLEU	chrF	BLEU	PER
-	Trans	47.40	71.20	48.19	-	39.60	47.30	38.57	-	47.40	71.20	48.19	-	39.60	47.30	38.57	-
Entity	Trans (5 types)	46.10	66.00	46.03	0.2240	36.00	43.10	35.91	0.2566	46.10	66.00	46.03	0.2557	36.00	43.10	35.91	0.2769
Dictionary	Dict-Trans (5 types)+ Texsmart (5 types)	47.90	69.20	47.82	0.2235	37.80	45.40	37.79	0.2601	47.90	69.20	47.82	0.2686	37.80	45.40	37.79	0.2777
Tag	Tag-Trans (5 types)+ Texsmart (5 types)	47.80	69.00	47.71	0.2145	37.70	45.20	37.67	0.2513	47.80	69.00	47.71	0.2632	37.70	45.20	37.67	0.2769

TABLE X: Results of implicit privacy preserving on extended datasets. We still use the Texsmart as the 10 types recognizer and train a Roberta-base model as the contexual privacy recognizer.

Source		联合国国际原子能总署（ IAEA ） 26 日公布报告指出，伊朗已开始使用先进的离心机
		加速生产浓缩铀，进一步违反 2015 年与世界大国达成的核协议。
Reference		The United Nations International Atomic Energy Agency ( IAEA ) released a report on the 26th stating that Iran
		has started to use advanced centrifuges to accelerate the production of enriched uranium , further violating the nuclear agreement reached with world powers in 2015 .
Standard Transformer Generation		The United Nations International Atomic Energy Agency ( IAEA ) released a report Tuesday that said Iran
		has begun to use advanced centrifuges to accelerate the production of enriched uranium , further violating its nuclear deal with the world ’s major powers in 2015 .
Tag-based Privacy Replacement		$\langle PINFO0\rangle$ （ $\langle PINFO1\rangle$ ） 5 日公布报告指出， $\langle PINFO2\rangle$ 已开始使用先进的离心机

TABLE XI: A case of translation with Tag-Trans. The red parts in listed sentences are privacy information.

Source		联合国国际原子能总署（ IAEA ） 26 日公布报告指出，伊朗已开始使用先进的离心机
		加速生产浓缩铀，进一步违反 2015 年与世界大国达成的核协议。
Reference		The United Nations International Atomic Energy Agency ( IAEA ) released a report on the 26th stating that Iran
		has started to use advanced centrifuges to accelerate the production of enriched uranium , further violating the nuclear agreement reached with world powers in 2015 .
Standard Transformer Generation		The United Nations International Atomic Energy Agency ( IAEA ) released a report Tuesday that said Iran
		has begun to use advanced centrifuges to accelerate the production of enriched uranium , further violating its nuclear deal with the world ’s major powers in 2015 .
Dictionary-based Privacy Replacement		联合国（安全理事会） 5 日公布报告指出，中国已开始使用先进的离心机

TABLE XII: A case of translation with Dict-Trans. The red parts in listed sentences are privacy information.

Source Language Sentence	年内上市公司 (PE 1) 回购规模超 1000 亿元 (PE 2) 创历史新高 - 新华网 (PE 3) 。
Target Language Reference Sentence	Listed company (PE 1) repurchase scale hits record high of over 100 billion yuan (PE 2) during the year - Xinhuanet (PE 3) .
Dictionary-based Privacy Replacement	年内联合国 (D-P 1) 回购规模超 130 元 (D-P 2) 创历史新高 - 安全理事会 (D-P 3) 。
Dict-Trans Generation	United Nations(D-P 1) Buy Back to Top 130(D-P 2) in Year - Security Council(D-P 3)
Final Translation	Listed companies(PE 1) buy back to top 1000(PE 2) in year - Xinhua(PE 3) .

TABLE XIII: A failure case of translation with Dict-Trans. The red parts in above sentences are privacy information. Specifically, PE x means the x-th privacy entity and D-P x means Dict-Placeholder for x-the privacy entity.

Privacy-Preserving ML

Privacy-preserving machine learning develops methods to learn, share, and run models while limiting exposure of sensitive data.

See all 432 papers

CompactQE: Interpretable Translation Quality Estimation via Small Open-Weight LLMs
May 15, 2026Kamil Guttmann, Zofia Fraś, Artur Nowakowski +1Small Language Models MT Evaluation
Laniqo, Pozna´n, Poland · Faculty of Mathematics and Computer Science, Adam Mickiewicz University, Pozna´n, Poland
Translation Analytics for Freelancers II: Benchmarking Local LLMs for Confidential Translation Workflows
May 29, 2026Yuri Balashov, Rex VanHorn, Mingxi Xu +1Multilingual Language Model Evaluation On-Device Language Model Inference
University of Georgia, Athens, Georgia, USA
Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation
Apr 8, 2026Jeongho Yoon, Chanhee Park, Yongchan Chun +2Privacy-Preserving Language Models Privacy-Preserving ML
Department of Computer Science and Engineering, Korea University · Samsung Mobile eXperience Business

May 15, 2026J/K move · Enter open · S save

CompactQE: Interpretable Translation Quality Estimation via Small Open-Weight LLMs

Kamil Guttmann, Zofia Fraś, Artur Nowakowski, Krzysztof Jassem

Laniqo, Pozna´n, Poland · Faculty of Mathematics and Computer Science, Adam Mickiewicz University, Pozna´n, Poland

Small Language Models MT Evaluation

Open paper AskPDF arXiv

Current state-of-the-art Quality Estimation (QE) in machine translation relies on massive, proprietary LLMs, raising data privacy concerns. We demonstrate that smaller, open-source LLMs (<30B parameters) are a viable, cost-effective and privacy-preserving alternative. Using a single-pass prompting strategy, our models simultaneously generate quality scores, MQM error annotations, suggested error corrections, and full post-editions. Our analysis shows these models achieve highly competitive system-level correlations with human judgments that outperform traditional neural metrics, fine-tuned models, and human inter-annotator agreement, effectively approximating the capabilities of much larger proprietary LLMs.

Abstract

Privacy-Preserving ML

Explore similar work

CompactQE: Interpretable Translation Quality Estimation via Small Open-Weight LLMs

Translation Analytics for Freelancers II: Benchmarking Local LLMs for Confidential Translation Workflows

Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation