cs.CRNov 13, 2026

From Bytes to Behavior: Request-Level Attribution from Live GPU Hidden States

Authors: Jonathan Pan, Yidi Yuan, Xin Wei Chia, Xinhui Su, Vincent Tan, Brenda Ong

Organizations: Home Team Science and Technology Agency, Singapore

Abstract

Existing memory forensics for machine learning systems typically targets static artifacts, such as model weights or prompt logs, rather than the transient computational states of a live inference process. This gap limits an examiner's ability to analyze the specific behavior of a running large language model (LLM) service that cannot be restarted or modified. We present a method for acquiring per-layer residual-stream activations from a live GPU process using a non-modifying debugger, without requiring prior knowledge of the model's architecture parameters or weights. The approach employs a general six-stage process with platform-specific adapters, utilizing either a PyTorch allocator clock or a kernel-launch clock to identify and read tensor allocations from video memory. We validate this acquisition on 33 open-source decoder-only models across nine architecture classes, demonstrating that the recovered bytes are byte-exact and usable for downstream analysis. As a use case, we apply linear probes trained on standard framework activations to the recovered states, successfully distinguishing refusal from compliance with an AUROC of 0.88 on held-out prompts. This work establishes that live, observational access to intermediate hidden states is feasible on named inference platforms, providing a new source of evidence for examining the operational behavior of deployed AI systems.

Figures & tables

Explore similar work

CardsList
  1. LLMscope: Extracting LLM Assets from Edge AI Chips via Optical Probing

    Aug 26, 2026Dev Mehta, Lily Dukette, William Folan +4LLM SecurityModel Extraction Attacks

  2. Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces

    Sep 6, 2026Roy Weiss, Benyamin Konstantinov, Eitam Sheetrit +2LLM SecurityPrivacy Leakage in Language Models

  3. GateWatch: Low-Overhead LLM Monitoring with Boolean Circuits

    Nov 18, 2026Firas Spanioli, Yaniv DavidLLM Safety MonitoringLanguage Model Probing