Existing memory forensics for machine learning systems typically targets static artifacts, such as model weights or prompt logs, rather than the transient computational states of a live inference process. This gap limits an examiner's ability to analyze the specific behavior of a running large language model (LLM) service that cannot be restarted or modified. We present a method for acquiring per-layer residual-stream activations from a live GPU process using a non-modifying debugger, without requiring prior knowledge of the model's architecture parameters or weights. The approach employs a general six-stage process with platform-specific adapters, utilizing either a PyTorch allocator clock or a kernel-launch clock to identify and read tensor allocations from video memory. We validate this acquisition on 33 open-source decoder-only models across nine architecture classes, demonstrating that the recovered bytes are byte-exact and usable for downstream analysis. As a use case, we apply linear probes trained on standard framework activations to the recovered states, successfully distinguishing refusal from compliance with an AUROC of 0.88 on held-out prompts. This work establishes that live, observational access to intermediate hidden states is feasible on named inference platforms, providing a new source of evidence for examining the operational behavior of deployed AI systems.
Figures & tables
Platform
Clock
Identification
Outcome
HF, static KV cache
Allocator
Two-step selector
Byte-exact, 33 models; decode 6 of 7
HF, default (growing) cache
Allocator
–
No adapter: decode 0 of 8
vLLM, eager
Allocator or kernel launch
Selector; or stream class
Byte-exact; stream 24/24, 22/22
Any, name-free content rule
Kernel launch
Additive identity
Failed dev gate: 2 of 9 logs (eager only)
vLLM, default (compile + graphs)
Kernel launch
Kernel-name dataflow rule
Prefill: 31/31, 23/23, 27/27 on 3 fresh models, a duplicate-stop filter fixes long prompts; decode: 6 of 6 runs, every step correct; last layer not claimed in prefill
Prefill byte-exact; decode 10 of 10 runs incl. last layer; quantised Q8_0 and Q4_K_M 12 of 12 runs per pipeline
TABLE I: Adapters built and tested: the platform-specific pair (clock, identification rule), where it worked, and where it did not. Counts are layer ends or layers recovered.
Fig. 1: The pipeline. A debugger attached to an unmodified PyTorch process logs every allocation; a blind selector picks the residual-stream allocations from the log alone; the bytes are read from VRAM; probes are applied to the recovered states.
Test set (n)
Error
False positive
False negative
Tag error
JailbreakBench (71)
15.5% [7.0, 25.4]
13.3% [4.4, 23.9]
19.2% [4.5, 36.4]
26.8%
Other source (157)
4.5% [1.3, 7.6]
6.2% [1.4, 12.1]
3.3% [0.0, 7.1]
8.3%
TABLE II: Refusal-state call from VRAM-recovered vLLM states (Qwen2.5-0.5B, 228 prompts); threshold fixed on the training source; 95% bootstrap intervals; positive means refused.
Measure
Value
Layer ends within 2% (prompts)
142 / 142
AUROC recovered (hook)
0.876 [0.808, 0.929] (0.876)
Balanced accuracy
0.777 [0.696, 0.855]
Accuracy (majority baseline)
0.803 [0.732, 0.866] (0.725)
False positive / false negative rate
0.282 [0.143, 0.436] / 0.165 [0.100, 0.240]
Brier (base rate) / ECE
0.167 (0.199) / 0.160
TABLE III: Experiment A: frozen refusal probe on states recovered from default-configuration vLLM (Qwen2.5-0.5B-Instruct, 142 untouched prompts); positive means refused; 95% bootstrap intervals.
Platform
Token count / hidden size
Layer count
Not derived
vLLM default
6/6 prefill; 6/6 decode
6/6 decode runs (all steps)
model identity
llama.cpp (fp16)
12/12 prefill calls; 10/10 decode
6/6 prefill pairs; 10/10 decode
model identity
llama.cpp (Q8_0, Q4_K_M)
12/12 prefill; 12/12 decode
12/12 prefill; 12/12 decode
model identity
SGLang
6/6 prefill; decode 4/6 (token count wrong on Llama-3.2-1B)
6/6 prefill; 6/6 decode
model identity
TABLE IV: Attributes the acquisition derives from the running process, with no model input, and how often they were exact in the pre-registered tests. Runs are model-prompt runs unless stated.
Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.
The move of LLM inference to edge AI accelerators introduces new physical vulnerabilities. During execution, model parameters and intermediate inference states are repeatedly loaded into and processed on the chip, making them suscep- tible to physical side-channel attacks. In this work, by deploying laser voltage imaging, we show that one can extract LLM assets during inference, namely embeddings, attention, and quantized MLP weights, activations, and other inference states, from localized memories and compute subcircuits. To validate our claims, we perform an attack on an FPGA-based LLM accelerator. Since such accelerators reuse the same buffers and compute subcircuits across addresses, tiles, modules, and layers, reading asset values comes down to probing different memories during inference. We demonstrate full recovery of the targeted values; however, we also establish a methodology to recover asset values even if some weights or bits remain unread. We further derive lower bounds that relate imaging effort to asset dimensions and show that even direct recovery scales linearly with the size of the targeted asset
We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization. Unlike prior attacks that rely on deployment-specific assumptions, such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, our approach targets the detokenizer, a component used in default LLM inference pipelines. To obtain clean signals, we use Flush+Reload on shared tokenizer code to detect when decoding occurs, which lets us perform Prime+Probe at the right moment and isolate token-dependent cache activity. We then apply a clustering-and-language-model pipeline to recover text from noisy cache observations. We evaluate the attack across multiple datasets, hardware platforms, inference frameworks, and model families, and show that it can recover semantically accurate outputs from real-world local LLM deployments, including agentic systems. This vulnerability is particularly significant because the most widely used tokenizer implementations are susceptible to the attack and are embedded in many popular local LLM products and agent frameworks, including systems such as OpenClaw (which we demonstrate), substantially broadening the practical attack surface.
Roy Weiss, Benyamin Konstantinov, Eitam Sheetrit +2
Ben Gurion University Israel · Microsoft Security Israel
As large language models (LLMs) and LLM-based agents are increasingly deployed, monitoring their behavior becomes important for detecting unsafe or unintended actions. Activation probes offer an efficient approach, but existing methods face trade-offs between accuracy, interpretability, and computational overhead. Probes on raw activations lack interpretable inputs, whereas sparse autoencoder (SAE) probes use interpretable features but often rely on additive classifiers and large SAE encoders, limiting interpretability and increasing memory overhead. We introduce GateWatch, a compact, interpretable classifier that expresses its entire decision rule as an explicit Boolean logic over a small number of binarized SAE features. We explore three approaches to constructing these circuits under a specified gate budget. For efficient deployment, GateWatchServe computes only selected SAE features, enables streaming of their binary values across layers during inference, and evaluates multiple GateWatch classifiers in parallel using compact lookup tables. Across 24 tasks on Gemma-2-9B, GateWatch achieves 80.49% mean test accuracy using only 3.8 SAE features and 3.5 logic gates per probe on average, just 1.17 percentage points below full 16K-feature SAE probes, while matching or outperforming them on 10 tasks and reducing probe memory by 99.98%. Inside vLLM, 1,000 concurrent multi-layer, multi-token GateWatch classifiers retain 97-98% of baseline throughput and 99.8% of KV-cache capacity.