Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings

Yubo Wang; Shujie Cui; James Bailey; Hongzhi Yin; Wenyu Liang; Min Tang; Shiyue Qin; Weiqing Wang

cs.LGAug 19, 2026

Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings

Authors: Yubo Wang, Shujie Cui, James Bailey, Hongzhi Yin, Wenyu Liang, Min Tang, Shiyue Qin, Weiqing Wang

Organizations: Monash University, Melbourne, Australia · The University of Queensland, Brisbane, Australia · Northeastern University, Shenyang, China

Abstract

Dense text embeddings are widely used in data mining, retrieval, and downstream machine learning systems due to their compact and semantically rich representations, but recent embedding inversion attacks have shown that they can expose substantial information about the original text, leading to serious privacy leakage risks. A common defense is to release perturbed embeddings by adding Gaussian noise, which is simple yet effective against standard inversion attacks and does not significantly degrade embedding utility for downstream tasks. However, it remains unclear whether such noise-protected embeddings are sufficiently safe against adaptive attackers that explicitly account for the perturbation process. In this paper, we study text embedding inversion in a noise-protected setting, where the attacker can observe only noisy embeddings and has no access to clean embedding targets. We first analyze why existing generative inversion methods fail under this setting and identify a "Double Noise Trap", which fundamentally prevents standard generative inversion models from achieving high-quality reconstruction. To address this challenge, we propose DAEI, a denoising-aware embedding inversion pipeline that combines a residual denoising autoencoder with generative text inversion where the denoiser is trained in an unsupervised manner using Stein's unbiased risk estimate to enable denoising from noisy observations alone. Extensive experiments show that DAEI achieves approximately 154% relative improvement in BLEU over the existing generative inversion baseline, while also improving token-level F1 and ROUGE-L by 32--60%. The promising inversion performance of DAEI challenges the prevailing assumption that simple Gaussian perturbation is sufficient to prevent sensitive information leakage from embedding representations.

Figures & tables

Fig. 1 : Geometric intuition of the “Double Noise Trap”. Additive Gaussian noise pushes the target embedding off the clean text manifold, causing vanilla inversion to chase off-manifold directions. DAEI instead projects the noisy representation back toward the clean manifold before inversion.

Fig. 2 : The overall architecture and data flow of our proposed DAEI pipeline. The system consists of four functional components (grey vertical panels) optimized across four stages. (1) Upon obtaining a noisy embedding from the hidden text via the Noisy Embedder, we first apply the DAE (indicated by the shield icons) to reconstruct a denoised embedding $\tilde{z}$ . (2) It is then processed by the Inverter to generate an initial hypothetical text $\hat{x}^{(k)}$ . (3) During training, the token-level cross-entropy loss ( $\mathcal{L}_{\mathrm{CE}}$ ) backpropagates directly through the active DAE to achieve joint fine-tuning (grey dashed lines). (4) Finally, the generated text enters an iterative correction loop: it is re-encoded by the embedder and passed through the DAE to yield a denoised hypothesis embedding $\tilde{e}^{(k)}$ . Both $\tilde{e}^{(k)}$ and the original denoised target $\tilde{z}$ are then fed into the Corrector to predict the next hypothetical text, repeating until the loop terminates to produce the final output.

System	Train embedding	Test embedding
Clean-to-Clean Baseline	Clean	Clean
ZSInvert	Noisy	Noisy
Clean-to-Noisy Baseline	Clean	Noisy
Noisy Inverter Baseline	Noisy	Noisy
DAE-only (ours)	Denoised by stage1 DAE	Denoised by stage1 DAE
DAEI (ours)	Denoised by stage3 DAE	Denoised by stage3 DAE

TABLE I : Main experimental systems. ZSInvert base is training-free; its listed training embedding type refers only to corrector training.

Category	Model	BLEU	F1	ROUGE-L	Cosine Similarity
Clean Upper Bound	Clean-to-Clean Baseline	0.6252	0.8656	0.8704	0.9500
Adversarial Decoding	ZSInvert	0.013	0.1503	0.1117	0.8568
Generative Inversion	Clean-to-Noisy Baseline	0.1348	0.5005	0.5012	0.7785
	Noisy Inverter Baseline	0.1916	0.5814	0.5660	0.8349
	DAE-only (Ours)	0.3616	0.7206	0.7200	0.9055
	DAEI (Ours)	0.4875 (+0.2959)	0.7717 (+0.1903)	0.7788 (+0.2128)	0.9254 (+0.0905)

TABLE II : Main results for in-domain evaluation with GTR-base . Gray rows indicate reference settings that are not available under the current threat model. Improvements in parentheses are absolute gains over the Noisy Inverter Baseline . Best feasible results excluding the clean upper bound are in bold .

Category	Model	BLEU	F1	ROUGE-L	Cosine Similarity
Clean Upper Bound	Clean-to-Clean Baseline	0.4391	0.7622	0.7570	0.9211
Adversarial Decoding	ZSInvert	0.0104	0.1189	0.0894	0.9641
Generative Inversion	Clean-to-Noisy Baseline	0.0846	0.3222	0.3271	0.7081
	Noisy Inverter Baseline	0.1249	0.3960	0.4044	0.7520
	DAE-only (Ours)	0.2688	0.5854	0.5722	0.8141
	DAEI (Ours)	0.3152 (+0.1903)	0.6344 (+0.2384)	0.6324 (+0.2280)	0.8556 (+0.1036)

TABLE III : Main results for in-domain evaluation with GTE-base . Formatting follows Table II .

OOD Dataset	Method	BLEU	F1	ROUGE-L
AG News	Upper Bound	0.1023	0.4296	0.4887
	ZSInvert	0.0130	0.1629	0.1142
	Baseline	0.0531	0.2749	0.3109
	DAEI	0.0791	0.3346	0.3915
Anthropic Toxic Prompts	Upper Bound	0.8114	0.8912	0.9118
	ZSInvert	0.0137	0.1451	0.1149

TABLE IV : Out-of-domain evaluation results for DAEI . Upper Bound denotes Clean-to-Clean Baseline, and Baseline denotes the Noisy Inverter Baseline.

Embedder	Stage	MSE $\downarrow$			Cosine $\uparrow$
Embedder	Stage	Noisy	DAE	$\Delta$	Noisy	DAE	$\Delta$
GTE-base	Stage 1	0.0765	0.0583	0.0182	0.9662	0.9727	0.0065
GTE-base	Stage 3	0.0765	0.0602	0.0163	0.9662	0.9713	0.0051
GTR-base	Stage 1	0.0766	0.0542	0.0224	0.9179	0.9322	0.0142
GTR-base	Stage 3	0.0766	0.0575	0.0191	0.9179	0.9280	0.0101

TABLE V : DAE component performance across training stages. MSE and cosine similarity are computed against clean embeddings before and after DAE denoising, with

\Delta

reporting the denoising improvement.

Panel of Figure 8 — Fig. 3 : LID-based geometric evaluation of DAE projection using clean embeddings as the reference set. The two subfigures report mean LID with SEM (Standard Error of the Mean) error bars and median LID, respectively, showing DAE reduces the LID of noisy embeddings back to the clean level.

$\sigma$	BLEU	F1	ROUGE-L	Cosine
0.005	0.4927	0.7823	0.7850	0.9293
0.008	0.4932	0.7807	0.7836	0.9288
0.010	0.4875	0.7717	0.7788	0.9254
0.012	0.4813	0.7729	0.7793	0.9242
0.015	0.4628	0.7578	0.7634	0.9218
0.018	0.4377	0.7421	0.7480	0.9240

TABLE VI : In-domain evaluation of DAEI on GTR-base under different Gaussian noise scales.

Model Inversion Attacks

Model inversion attacks reconstruct private training data or sensitive attributes from a model’s outputs, parameters, gradients, or representations.

See all 21 papers

DeepInvert: Semi-Supervised Embedding Inversion Against Obfuscated Language Models
Aug 5, 2026Zhicong Huang, Cheng Hong, Tao WeiPrivacy-Preserving Language Models Privacy Leakage in Language Models
Ant Group
Shadow Queries for Private Retrieval in Vector Databases
Sep 4, 2026Xinguo Feng, Zhongkui Ma, Zihan Wang +4Privacy-Preserving ML Information Retrieval
The University of Queensland, Brisbane, Australia · City University of Hong Kong, Hong Kong · CSIRO’s Data 61, Sydney, Australia
STAMP: Selective Task-Aware Mechanism for Text Privacy
Mar 12, 2026Fengwei Tian, Payel Bhattacharjee, Heidi Hanson +3Privacy-Utility Tradeoff Privacy-Preserving Language Models
University of Arizona · Oak Ridge National Laboratory · Duke University

Aug 5, 2026J/K move · Enter open · S save

DeepInvert: Semi-Supervised Embedding Inversion Against Obfuscated Language Models

Zhicong Huang, Cheng Hong, Tao Wei

Ant Group

Privacy-Preserving Language Models Privacy Leakage in Language Models

Open paper AskPDF arXiv

Cloud-based language model services routinely process prompts containing sensitive information. Obfuscation-based defenses---including ObfusLM, SentinelLMs, TextObfuscator, and DPNR---mitigate this risk by transforming prompt representations before transmission, offering a lightweight alternative to cryptographic solutions. We show these defenses provide far less protection than previously believed. We present DeepInvert, a semi-supervised embedding inversion attack that recovers original tokens from obfuscated representations with higher accuracy than prior methods. The key insight is that unlabeled obfuscated embeddings retain exploitable semantic structure despite perturbation. DeepInvert combines supervised training on labeled shadow data with a novel unsupervised consistency objective over unlabeled target embeddings, alternating between the two via a mixed training pipeline. Defense-aware adaptations further extend the attack to diverse obfuscation mechanisms across encoder-based and autoregressive architectures. Experiments on nine defenses, five tasks, and four model architectures show that DeepInvert outperforms prior attacks on most defenses. Against ObfusLM, DeepInvert achieves 73.5% top-1 token recovery versus 26.2% for the previous best. Our results reveal a task-dependent tension: obfuscation schemes preserving enough signal for utility also retain sufficient structure for inversion, while schemes resisting inversion collapse utility. On simpler classification tasks, some DP-based defenses can maintain both. We call for a re-evaluation of this defense class.

Abstract

Model Inversion Attacks

Explore similar work

DeepInvert: Semi-Supervised Embedding Inversion Against Obfuscated Language Models

Shadow Queries for Private Retrieval in Vector Databases

STAMP: Selective Task-Aware Mechanism for Text Privacy