Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces

Roy Weiss; Benyamin Konstantinov; Eitam Sheetrit; Tomer Simon; Yisroel Mirsky

cs.CRSep 6, 2026

Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces

Authors: Roy Weiss, Benyamin Konstantinov, Eitam Sheetrit, Tomer Simon, Yisroel Mirsky

Organizations: Ben Gurion University Israel · Microsoft Security Israel

Abstract

We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization. Unlike prior attacks that rely on deployment-specific assumptions, such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, our approach targets the detokenizer, a component used in default LLM inference pipelines. To obtain clean signals, we use Flush+Reload on shared tokenizer code to detect when decoding occurs, which lets us perform Prime+Probe at the right moment and isolate token-dependent cache activity. We then apply a clustering-and-language-model pipeline to recover text from noisy cache observations. We evaluate the attack across multiple datasets, hardware platforms, inference frameworks, and model families, and show that it can recover semantically accurate outputs from real-world local LLM deployments, including agentic systems. This vulnerability is particularly significant because the most widely used tokenizer implementations are susceptible to the attack and are embedded in many popular local LLM products and agent frameworks, including systems such as OpenClaw (which we demonstrate), substantially broadening the practical attack surface.

Figures & tables

Figure 1. Tokenizers such as Llama.cpp and HuggingFace Tokenizers are vulnerable to our attack. Because these libraries are widely deployed in local LLM servers, the vulnerability extends to a large ecosystem of downstream applications, tools, and IDE plugins that depend on them (as of August 2026).

Figure 2. Attack workflow illustrating the profiling and exploitation phases and their 4 steps. During profiling, the adversary induces generation to collect traces covering common tokens and trains a predictive model; during exploitation, it uses this model to reconstruct future conversations.

Figure 3. Example cache traces collected during the decode operation of two tokens. Each bar shows the average access latency of a cache set over 10 repetitions.

Figure 4. The trace-to-text reconstruction pipeline. Each trace in the sequence is first encoded into an embedding and assigned to a cluster learned during the profiling phase. The trace is then replaced with the symbol corresponding to that cluster, producing a sequence of symbols. Finally, this symbol sequence is passed to a language model trained to reconstruct the original tokens. Following prior work, we use separate models for the first segment (LLM A ) and subsequent segments (LLM B ).

Dataset	Framework	Hardware	Target	First Segments				Full Paragraphs
Dataset	Framework	Hardware	Target	ASR	$\Phi$	R1-F1	ED $\downarrow$	ASR	$\Phi$	R1-F1	ED $\downarrow$
UltraChat	Llama.cpp	Laptop	Phi-3-mini	57.51%	69.24% $\pm$ 31.59%	77.25% $\pm$ 21.80%	77.97% $\pm$ 20.63%	85.15%	86.68% $\pm$ 14.19%	82.70% $\pm$ 9.69%	82.96% $\pm$ 9.68%
		Laptop	Llama-3	39.71%	54.53% $\pm$ 33.54%	63.99% $\pm$ 23.96%	65.72% $\pm$ 22.18%	72.03%	81.22% $\pm$ 15.67%	72.56% $\pm$ 10.28%	72.08% $\pm$ 10.37%
		Desktop	Phi-3-mini	61.55%	71.62% $\pm$ 30.98%	78.72% $\pm$ 21.51%	79.35% $\pm$ 20.32%	87.42%	87.52% $\pm$ 13.80%	83.02% $\pm$ 9.54%	83.31% $\pm$ 9.56%
		Desktop	Llama-3	44.52%	59.41% $\pm$ 33.28%	69.32% $\pm$ 23.60%	70.18% $\pm$ 22.10%	74.78%	83.19% $\pm$ 15.12%	76.52% $\pm$ 10.21%	76.09% $\pm$ 10.40%
	HuggingFace	Laptop	Phi-3-mini	58.77%	69.43% $\pm$ 31.18%	75.65% $\pm$ 22.18%	77.05% $\pm$ 20.66%	86.11%	86.38% $\pm$ 14.04%	80.23% $\pm$ 9.69%	80.89% $\pm$ 9.59%

Table 1. Reconstruction performance for textual datasets, frameworks, hardware platforms, and target models. All metrics are reported as mean

\pm

standard deviation. Higher is better for all metrics except ED (

\downarrow

Framework	Hardware	Target	First	Full
Llama.cpp	Laptop	Phi-3-mini	90.17%	95.87%
	Laptop	Llama-3	50.13%	49.43%
	Desktop	Phi-3-mini	87.65%	90.56%
	Desktop	Llama-3	81.55%	86.86%
HuggingFace	Laptop	Phi-3-mini	85.19%	88.44%
	Laptop	Llama-3	64.69%	71.35%

Table 2. Code-Alpaca reconstruction performance (ASR) for first-segment and full-output reconstruction.

Panel of Figure 5 — Figure 5. Examples of successful and unsuccessful attacks on $R_{0}$ in UltraChat.

Figure 6. Robustness to additional noise injected into the reconstruction pipeline. We report, for each

p

, the base ASR and the rate of high-fidelity predictions with

\phi\geq 0.9

for both hardware settings.

Figure 7. ASR as a function of targeted and equivalent benign profiling query budgets on Phi-3-mini.

Target	ASR	$\boldsymbol{\phi>0.5}$	R1-F1	ED $\downarrow$
Phi-3-mini	38.40%	52.12%	56.97%	61.07%
Llama-3	31.97%	46.86%	53.87%	57.97%

Table 3. Reconstruction performance (first-segments) on a 13th Gen Intel CPU.

	Model Assumptions		System Assumptions
Attack	Weights	Arch	SMT	Shared Resources
IKWYS ( Gao et al., 2025 )	CPU	None	None	Model data; special
Spill the Beans ( Adiletta and Sunar, 2025 )	CPU	None	None	Model data; special
MoEcho-GPU ( Ding et al., 2025 )	None	MoE	None	GPU counters; special
MoEcho-CPU ( Ding et al., 2025 )	None	MoE	Yes	Expert pages; special
Ours	None	None	Yes	Code only; standard

Table 4. Comparison of deployment assumptions across local LLM side-channel attacks. "Special" denotes non-standard shared resources, while "standard" denotes commonly shared resources in default deployments.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Before	After	Cleaned (%)
Code-Alpaca	121,959	98,540	18.8%
ChatDoctor	112,165	96,084	14.3%
UltraChat	577,747	358,545	37.9%

Appendix

Table 5. Dataset sizes before and after cleaning, with percentage removed.

Panel of Figure 8 — Figure 8. LLM judge prompts for reconstruction evaluation.

Figure 9. Reconstruction fidelity across different numbers of symbol clusters

K

on UltraChat first-segment reconstruction.

Dataset	Framework	Hardware	Target	First Segments						Full Paragraphs
Dataset	Framework	Hardware	Target	$\phi{\geq}0.9$	$\phi{=}1$	R1 $\geq$ 0.9	R1 $=$ 1	ED $\leq$ 0.1	ED $=$ 0	$\phi{\geq}0.9$	$\phi{=}1$	R1 $\geq$ 0.9	R1 $=$ 1	ED $\leq$ 0.1	ED $=$ 0
UltraChat	Llama.cpp	Laptop	Phi-3-mini	41.06%	17.03%	37.65%	16.45%	37.41%	16.28%	55.38%	0.68%	22.12%	0.44%	24.01%	0.44%
		Laptop	Llama-3	23.06%	5.16%	15.97%	4.85%	16.52%	4.81%	34.72%	0.13%	1.31%	0.12%	1.67%	0.12%
		Desktop	Phi-3-mini	44.62%	18.93%	40.86%	18.47%	40.43%	18.36%	58.48%	0.78%	22.45%	0.51%	24.83%	0.51%
		Desktop	Llama-3	28.54%	8.13%	23.49%	7.87%	23.10%	7.79%	41.66%	0.25%	4.94%	0.20%	5.60%	0.20%
	HuggingFace	Laptop	Phi-3-mini	40.23%	13.76%	33.62%	13.16%	34.58%	13.07%	54.08%	0.38%	11.20%	0.27%	14.36%	0.27%

Appendix

Table 6. Reconstruction accuracy across hardware, inference framework, target model, and segmentation configurations. Results are reported for high-fidelity reconstruction thresholds and exact matches under

\phi

, ROUGE-1 (R1), and edit-distance (ED) metrics.

Processor	Class	Generation	Cores (P + E)	Threads
Core i7-6700K	Desktop	6th	4	8
Core i5-6200U	Laptop	6th	2	4
Core i7-7700K	Desktop	7th	4	8
Core i5-7200U	Laptop	7th	2	4
Core i7-8700K	Desktop	8th	6	12
Core i5-8250U	Laptop	8th	4	8

Appendix

Table 7. Representative Intel processors with SMT (Hyper-Threading) enabled by default, across the 6th–14th generations.

\dagger

marks the platforms evaluated in this work.

Panel of Figure 13 — Figure 13. Examples of Successful Paragraph-Level Reconstructions from UltraChat and ChatDoctor

LLM Security

Large language model (LLM) security studies attacks, privacy risks, and defenses across model training, inference, and deployment.

See all 327 papers

Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing
Jul 22, 2026Sadegh Majidi, Niloofar Mireshghallah, Kazem TaramModel Extraction Attacks Language Model Fingerprinting
Purdue University West Lafayette, IN, USA · Carnegie Mellon University Pittsburgh, PA, USA
Secret Stealing Attacks on Local LLM Fine-Tuning through Supply-Chain Model Code Backdoors
Apr 30, 2026Zi Li, Tian Zhou, Wenze Li +3Data Leakage LLM Backdoor Attacks
Nanjing University
Stealing Reasoning Traces from Proprietary LLM APIs
Aug 10, 2026Alexander Panfilov, David Schmotz, Ilia Shumailov +5Model Extraction Attacks Adversarial Attacks
MATS Research · ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems +4

Jul 22, 2026J/K move · Enter open · S save

Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing

Sadegh Majidi, Niloofar Mireshghallah, Kazem Taram

Purdue University West Lafayette, IN, USA · Carnegie Mellon University Pittsburgh, PA, USA

Model Extraction Attacks Language Model Fingerprinting LLM Security

Open paper AskPDF arXiv

This work presents LeakyLMs, a set of attacks that leak proprietary model, architecture, and deployment information from production language models. LeakyLMs is the first to demonstrate that key model and deployment details can be inferred using only token generation timing, even when interacting through remote APIs. LeakyLMs introduces two core attacks. The first attack targets inference optimizations and deployment strategies. For example, our attack detects whether a provider uses speculative decoding, a widely deployed inference-time optimization, and further identifies the context length of the draft model used in the pipeline. Our measurements show that Google Gemini Flash 2.5 uses speculative decoding with a draft context window of approximately 128K tokens. The second attack recovers key architectural properties, including the number of transformer layers, hidden dimension size, and number of attention heads. To achieve this, LeakyLMs builds a detailed and accurate model of token-generation timing on modern NVIDIA GPUs, characterizing how latency scales with model configuration and hardware parameters. The attack then performs a search over the architecture space using this timing model. In experiments with Llama models, the near-correct architectural configuration appears in the top-10 guesses more than 90% of the time.

Abstract

LLM Security

Explore similar work

Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing

Secret Stealing Attacks on Local LLM Fine-Tuning through Supply-Chain Model Code Backdoors

Stealing Reasoning Traces from Proprietary LLM APIs