Without a restart or code change, a debugger lifts per-layer hidden states from a live GPU, calling refusal at AUROC 0.88
Synopsis
For an online large-model service that cannot be restarted or modified, existing memory forensics recovers only static artifacts such as weights or prompt logs, not the per-layer hidden states that exist briefly on the GPU during a single request. A six-stage pipeline built on a non-modifying debugger uses either a PyTorch allocator clock or a kernel-launch clock to identify and read tensor allocations in video memory, recovering bytes byte-exactly on 33 open-source decoder-only models across nine architecture classes. Linear probes trained on ordinary framework activations, applied to those recovered states, separate refusal from compliance on 142 untouched prompts at an AUROC of 0.876, with a false-positive rate of 0.282.
Interpretation
A running large-model service can yield the hidden-state vector each layer produced for a single request, without stopping the service or changing its code, so that request can be examined after the fact. Earlier memory forensics targeted static artifacts such as weights or prompt logs, or relied on known framework objects to reconstruct a deployed model; the transient states of live inference leave no trace in standard logs or model files. Across 33 open-source decoder-only models spanning nine architecture classes, the recovered bytes matched a synchronized hook reference byte for byte, and the selector was never told the layer count.
The method attaches a debugger that does not modify the target, uses memory-allocation events or kernel launches as a clock, and reads video memory at the moment a tensor has just been written. The allocator clock sets breakpoints on the PyTorch caching allocator's allocation and free paths, logging the size and device pointer of every tensor birth and release, then a two-step rule picks the residual stream out of that log. On Hugging Face and vLLM in eager mode, all 228 prompts for Qwen2.5-0.5B-Instruct were recovered at every layer, with a maximum difference of 0.0 from the hook reference.
Recovered states can be fed directly to existing linear probes to produce a request-level refusal call, not just structural information. The probes themselves are established technique; what is tested is whether states taken from live video memory still carry a readable semantic signal. On 142 prompts never used to build or tune the probe, the frozen probe reached an AUROC of 0.876, balanced accuracy of 0.777 and accuracy of 0.803, above the 0.725 baseline of always answering refused.
Perspective
The demonstrated setting is a single local GPU with ptrace access and a CUDA debugger attachable, decoder-only models up to 3.4B parameters plus one 4-bit 7B model, batch size 1, fixed prompt lengths, and fp32 or bf16 precision (fp16 on vLLM). Structural attributes (token count, hidden size, layer count) were checked item by item in pre-registered platform tests, computational states were compared with hook references under laboratory conditions, and behavioral conclusions come from one 0.5B model and two small held-out sources.
Acquisition is non-modifying but not free: the debugger halts at every allocation, and with the collector attached prefill latency per prompt rose from 9.6 to 10.7 ms in default mode to 157 to 163 seconds, measured on a single GPU. The allocator clock fails in vLLM's default configuration with compilation and CUDA graphs, the kernel-name rule was tested out of sample on only three fresh models, and prefill does not claim the last layer. The authors label the behavioral evidence preliminary: false-positive rate 0.282, calibration error 0.160, accuracy 0.696 on the XSTest safe subset, and only 18 prompts in the HarmBench subset.
