Dense text embeddings are widely used in data mining, retrieval, and downstream machine learning systems due to their compact and semantically rich representations, but recent embedding inversion attacks have shown that they can expose substantial information about the original text, leading to serious privacy leakage risks. A common defense is to release perturbed embeddings by adding Gaussian noise, which is simple yet effective against standard inversion attacks and does not significantly degrade embedding utility for downstream tasks. However, it remains unclear whether such noise-protected embeddings are sufficiently safe against adaptive attackers that explicitly account for the perturbation process. In this paper, we study text embedding inversion in a noise-protected setting, where the attacker can observe only noisy embeddings and has no access to clean embedding targets. We first analyze why existing generative inversion methods fail under this setting and identify a "Double Noise Trap", which fundamentally prevents standard generative inversion models from achieving high-quality reconstruction. To address this challenge, we propose DAEI, a denoising-aware embedding inversion pipeline that combines a residual denoising autoencoder with generative text inversion where the denoiser is trained in an unsupervised manner using Stein's unbiased risk estimate to enable denoising from noisy observations alone. Extensive experiments show that DAEI achieves approximately 154% relative improvement in BLEU over the existing generative inversion baseline, while also improving token-level F1 and ROUGE-L by 32--60%. The promising inversion performance of DAEI challenges the prevailing assumption that simple Gaussian perturbation is sufficient to prevent sensitive information leakage from embedding representations.
Figures & tables
Fig. 1 : Geometric intuition of the “Double Noise Trap”. Additive Gaussian noise pushes the target embedding off the clean text manifold, causing vanilla inversion to chase off-manifold directions. DAEI instead projects the noisy representation back toward the clean manifold before inversion.
Fig. 2 : The overall architecture and data flow of our proposed DAEI pipeline. The system consists of four functional components (grey vertical panels) optimized across four stages. (1) Upon obtaining a noisy embedding from the hidden text via the Noisy Embedder, we first apply the DAE (indicated by the shield icons) to reconstruct a denoised embedding z~ . (2) It is then processed by the Inverter to generate an initial hypothetical text x^(k) . (3) During training, the token-level cross-entropy loss ( LCE ) backpropagates directly through the active DAE to achieve joint fine-tuning (grey dashed lines). (4) Finally, the generated text enters an iterative correction loop: it is re-encoded by the embedder and passed through the DAE to yield a denoised hypothesis embedding e~(k) . Both e~(k) and the original denoised target z~ are then fed into the Corrector to predict the next hypothetical text, repeating until the loop terminates to produce the final output.
System
Train embedding
Test embedding
Clean-to-Clean Baseline
Clean
Clean
ZSInvert
Noisy
Noisy
Clean-to-Noisy Baseline
Clean
Noisy
Noisy Inverter Baseline
Noisy
Noisy
DAE-only (ours)
Denoised by stage1 DAE
Denoised by stage1 DAE
DAEI (ours)
Denoised by stage3 DAE
Denoised by stage3 DAE
TABLE I : Main experimental systems. ZSInvert base is training-free; its listed training embedding type refers only to corrector training.
Category
Model
BLEU
F1
ROUGE-L
Cosine Similarity
Clean Upper Bound
Clean-to-Clean Baseline
0.6252
0.8656
0.8704
0.9500
Adversarial Decoding
ZSInvert
0.013
0.1503
0.1117
0.8568
Generative Inversion
Clean-to-Noisy Baseline
0.1348
0.5005
0.5012
0.7785
Noisy Inverter Baseline
0.1916
0.5814
0.5660
0.8349
DAE-only (Ours)
0.3616
0.7206
0.7200
0.9055
DAEI (Ours)
0.4875 (+0.2959)
0.7717 (+0.1903)
0.7788 (+0.2128)
0.9254 (+0.0905)
TABLE II : Main results for in-domain evaluation with GTR-base . Gray rows indicate reference settings that are not available under the current threat model. Improvements in parentheses are absolute gains over the Noisy Inverter Baseline . Best feasible results excluding the clean upper bound are in bold .
Category
Model
BLEU
F1
ROUGE-L
Cosine Similarity
Clean Upper Bound
Clean-to-Clean Baseline
0.4391
0.7622
0.7570
0.9211
Adversarial Decoding
ZSInvert
0.0104
0.1189
0.0894
0.9641
Generative Inversion
Clean-to-Noisy Baseline
0.0846
0.3222
0.3271
0.7081
Noisy Inverter Baseline
0.1249
0.3960
0.4044
0.7520
DAE-only (Ours)
0.2688
0.5854
0.5722
0.8141
DAEI (Ours)
0.3152 (+0.1903)
0.6344 (+0.2384)
0.6324 (+0.2280)
0.8556 (+0.1036)
TABLE III : Main results for in-domain evaluation with GTE-base . Formatting follows Table II .
OOD Dataset
Method
BLEU
F1
ROUGE-L
AG News
Upper Bound
0.1023
0.4296
0.4887
ZSInvert
0.0130
0.1629
0.1142
Baseline
0.0531
0.2749
0.3109
DAEI
0.0791
0.3346
0.3915
Anthropic Toxic Prompts
Upper Bound
0.8114
0.8912
0.9118
ZSInvert
0.0137
0.1451
0.1149
TABLE IV : Out-of-domain evaluation results for DAEI . Upper Bound denotes Clean-to-Clean Baseline, and Baseline denotes the Noisy Inverter Baseline.
Embedder
Stage
MSE ↓
Cosine ↑
Noisy
DAE
Δ
Noisy
DAE
Δ
GTE-base
Stage 1
0.0765
0.0583
0.0182
0.9662
0.9727
0.0065
Stage 3
0.0765
0.0602
0.0163
0.9662
0.9713
0.0051
GTR-base
Stage 1
0.0766
0.0542
0.0224
0.9179
0.9322
0.0142
Stage 3
0.0766
0.0575
0.0191
0.9179
0.9280
0.0101
TABLE V : DAE component performance across training stages. MSE and cosine similarity are computed against clean embeddings before and after DAE denoising, with Δ reporting the denoising improvement.
Fig. 3 : LID-based geometric evaluation of DAE projection using clean embeddings as the reference set. The two subfigures report mean LID with SEM (Standard Error of the Mean) error bars and median LID, respectively, showing DAE reduces the LID of noisy embeddings back to the clean level.
σ
BLEU
F1
ROUGE-L
Cosine
0.005
0.4927
0.7823
0.7850
0.9293
0.008
0.4932
0.7807
0.7836
0.9288
0.010
0.4875
0.7717
0.7788
0.9254
0.012
0.4813
0.7729
0.7793
0.9242
0.015
0.4628
0.7578
0.7634
0.9218
0.018
0.4377
0.7421
0.7480
0.9240
TABLE VI : In-domain evaluation of DAEI on GTR-base under different Gaussian noise scales.
Cloud-based language model services routinely process prompts containing sensitive information. Obfuscation-based defenses---including ObfusLM, SentinelLMs, TextObfuscator, and DPNR---mitigate this risk by transforming prompt representations before transmission, offering a lightweight alternative to cryptographic solutions. We show these defenses provide far less protection than previously believed. We present DeepInvert, a semi-supervised embedding inversion attack that recovers original tokens from obfuscated representations with higher accuracy than prior methods. The key insight is that unlabeled obfuscated embeddings retain exploitable semantic structure despite perturbation. DeepInvert combines supervised training on labeled shadow data with a novel unsupervised consistency objective over unlabeled target embeddings, alternating between the two via a mixed training pipeline. Defense-aware adaptations further extend the attack to diverse obfuscation mechanisms across encoder-based and autoregressive architectures. Experiments on nine defenses, five tasks, and four model architectures show that DeepInvert outperforms prior attacks on most defenses. Against ObfusLM, DeepInvert achieves 73.5% top-1 token recovery versus 26.2% for the previous best. Our results reveal a task-dependent tension: obfuscation schemes preserving enough signal for utility also retain sufficient structure for inversion, while schemes resisting inversion collapse utility. On simpler classification tasks, some DP-based defenses can maintain both. We call for a re-evaluation of this defense class.
Large language models (LLMs) increasingly rely on information retrieval (IR) systems, such as Retrieval-Augmented Generation (RAG), to incorporate domain-specific knowledge without costly re-training. These systems often store pre-computed document embeddings in cloud-based vector databases. However, such embeddings are vulnerable to embedding inversion attacks (EIAs), which can reconstruct their underlying text. Existing defenses, such as adding noise or scaling embeddings, often provide limited privacy or significantly reduce retrieval utility. We propose SHAQ (shadow query generation), a semantic-decomposition and embedding-decoupling defense against EIAs. SHAQ is based on the insight that EIAs rely on the strong coupling between an embedding and its original text. Instead of storing document embeddings directly, SHAQ uses a generative language model to create diverse shadow queries that capture different semantic aspects of each document. These queries are then encoded and stored in place of the original document embeddings, thereby decomposing document semantics and decoupling stored embeddings from the source text. Experiments across diverse IR datasets show that SHAQ substantially improves privacy while preserving retrieval utility, achieving a recovery rate as low as 0.2104, defending up to 19.50% more tokens than baseline defenses, and reaching up to 0.7967 MAP@10 with up to 5.53% utility improvement. These results demonstrate that semantic decomposition and embedding decoupling provide an effective alternative to directly modifying embeddings for defending against EIAs.
Xinguo Feng, Zhongkui Ma, Zihan Wang +4
The University of Queensland, Brisbane, Australia · City University of Hong Kong, Hong Kong · CSIRO’s Data 61, Sydney, Australia
We present STAMP (Selective Task-Aware Mechanism for Text Privacy), a new framework for task-aware text privatization that achieves an improved privacy-utility trade-off. STAMP selectively allocates privacy budgets across tokens by jointly considering (i) each token's importance to the downstream task (as measured via a task- or query-specific representation), and (ii) its privacy sensitivity (e.g., names, dates, identifiers). This token-level partitioning enables fine-grained, group-wise control over the level of noise applied to different parts of the input, balancing privacy protection with task relevance. To privatize individual token embeddings, we introduce the polar mechanism, which perturbs only the direction of embeddings on the unit sphere while preserving their magnitude. Decoding is performed via cosine nearest-neighbor search, aligning the perturbation geometry with the decoding geometry. Unlike isotropic noise mechanisms, the polar mechanism maintains semantic neighborhoods in the embedding space and better preserves downstream utility. Experimental evaluations on SQuAD, Yelp, and AG News datasets demonstrate that STAMP, when combined with the normalized polar mechanism, consistently achieves superior privacy-utility trade-offs across varying per-token privacy budgets.