From Bytes to Behavior: Request-Level Attribution from Live GPU Hidden States
Organizations: Home Team Science and Technology Agency, Singapore
Abstract
Existing memory forensics for machine learning systems typically targets static artifacts, such as model weights or prompt logs, rather than the transient computational states of a live inference process. This gap limits an examiner's ability to analyze the specific behavior of a running large language model (LLM) service that cannot be restarted or modified. We present a method for acquiring per-layer residual-stream activations from a live GPU process using a non-modifying debugger, without requiring prior knowledge of the model's architecture parameters or weights. The approach employs a general six-stage process with platform-specific adapters, utilizing either a PyTorch allocator clock or a kernel-launch clock to identify and read tensor allocations from video memory. We validate this acquisition on 33 open-source decoder-only models across nine architecture classes, demonstrating that the recovered bytes are byte-exact and usable for downstream analysis. As a use case, we apply linear probes trained on standard framework activations to the recovered states, successfully distinguishing refusal from compliance with an AUROC of 0.88 on held-out prompts. This work establishes that live, observational access to intermediate hidden states is feasible on named inference platforms, providing a new source of evidence for examining the operational behavior of deployed AI systems.
Figures & tables
| Platform | Clock | Identification | Outcome |
|---|---|---|---|
| HF, static KV cache | Allocator | Two-step selector | Byte-exact, 33 models; decode 6 of 7 |
| HF, default (growing) cache | Allocator | – | No adapter: decode 0 of 8 |
| vLLM, eager | Allocator or kernel launch | Selector; or stream class | Byte-exact; stream 24/24, 22/22 |
| Any, name-free content rule | Kernel launch | Additive identity | Failed dev gate: 2 of 9 logs (eager only) |
| vLLM, default (compile + graphs) | Kernel launch | Kernel-name dataflow rule | Prefill: 31/31, 23/23, 27/27 on 3 fresh models, a duplicate-stop filter fixes long prompts; decode: 6 of 6 runs, every step correct; last layer not claimed in prefill |
| llama.cpp, CUDA backend | Kernel launch | Additive identity (prefill); norm-launch rule (decode) | Prefill byte-exact; decode 10 of 10 runs incl. last layer; quantised Q8_0 and Q4_K_M 12 of 12 runs per pipeline |
| Test set (n) | Error | False positive | False negative | Tag error |
|---|---|---|---|---|
| JailbreakBench (71) | 15.5% [7.0, 25.4] | 13.3% [4.4, 23.9] | 19.2% [4.5, 36.4] | 26.8% |
| Other source (157) | 4.5% [1.3, 7.6] | 6.2% [1.4, 12.1] | 3.3% [0.0, 7.1] | 8.3% |
| Measure | Value |
|---|---|
| Layer ends within 2% (prompts) | 142 / 142 |
| AUROC recovered (hook) | 0.876 [0.808, 0.929] (0.876) |
| Balanced accuracy | 0.777 [0.696, 0.855] |
| Accuracy (majority baseline) | 0.803 [0.732, 0.866] (0.725) |
| False positive / false negative rate | 0.282 [0.143, 0.436] / 0.165 [0.100, 0.240] |
| Brier (base rate) / ECE | 0.167 (0.199) / 0.160 |
| Platform | Token count / hidden size | Layer count | Not derived |
|---|---|---|---|
| vLLM default | 6/6 prefill; 6/6 decode | 6/6 decode runs (all steps) | model identity |
| llama.cpp (fp16) | 12/12 prefill calls; 10/10 decode | 6/6 prefill pairs; 10/10 decode | model identity |
| llama.cpp (Q8_0, Q4_K_M) | 12/12 prefill; 12/12 decode | 12/12 prefill; 12/12 decode | model identity |
| SGLang | 6/6 prefill; decode 4/6 (token count wrong on Llama-3.2-1B) | 6/6 prefill; 6/6 decode | model identity |
LLM Auditing
Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.
See all 720 papers