Auditing long-horizon agents at the state transfer: root-cause step accuracy rises from 10.8% to 98.3% across 240 fault-injected trajectories
Synopsis
Security and reliability failures in long-horizon LLM agents often occur where the execution harness moves state, yet conventional traces log prompts and tool calls and omit what crosses those boundaries. A state-transfer audit architecture places an audit point at eight boundaries—observation, context assembly, compaction, memory write, consolidation, action, rollback and delegation—and emits hash-chained records that separate deterministic checks from model-based semantic checks. Across 240 multi-step fault-injection trajectories, full transfer records reached 98.3% violation recall at 0.0% false positives and raised exact root-cause step attribution from 10.8%–16.7% to 98.3%.
Interpretation
The audit target moves from model calls and tool calls to the harness's state transfers, making observation, compaction, memory write, consolidation, rollback and delegation each a recordable boundary. Conventional execution traces log prompts and tool calls but not what crosses these boundaries, so failure attribution and runtime monitors fail on multi-step trajectories. On the Who&When benchmark of multi-agent failure logs, the best of three initial methods identified the failure-responsible step in only 14.2% of cases.
The record emitted at each boundary binds internal context, memory and inference-server cache state to external actions, and the same records serve both post-hoc attribution and runtime controls. Existing observability schemas trace events across the agent lifecycle without tying each record to a failure it detects. Across 240 multi-step trajectories, 120 benign and 120 fault-injected spanning six families, transfer records localized root causes at microsecond deterministic overhead and caught cross-transfer violations missed by lifecycle and tool-only logs.
The implementation wraps each transfer in the harness, logging the hash-chained record and running boundary checks before the state change is applied, with no model changes. Deterministic checks add only logging and comparison, whereas model-based checkers process untrusted context, so the two differ in cost and attack surface. The reference middleware measured deterministic wrapper latency per transfer over 20,000 invocations, with records averaging 524 bytes, matching the estimate of about 0.5 MB per thousand transfers.
Perspective
The demonstrated scope is a reference middleware on controlled fault-injection trajectories: 240 multi-step runs, 120 benign and 120 injected across six fault families, plus deterministic wrapper overhead measured over 20,000 invocations. Broader end-to-end evaluation on live production infrastructure and real multimodal web and desktop interfaces is stated as the next step.
Checking constraint retention needs a model call whose cost and recall on real failures are unmeasured. Web and office tasks lack a test oracle for memory writes. Write-time labelers can themselves be attacked, commercial APIs that hide server KV state prevent direct cache-rollback checks, and audit records hold user data that require explicit retention and access policies.
