Agents Fall Back Past the Line: nrepair Cuts Boundary-Crossing Actions by Up to 76% Across Four Models
Synopsis
Large language model agents often switch to a substitute tool after an ordinary tool error, but the same fallback crosses an action boundary when the target's state differs, as when a forced push deletes another author's commits. Across 304 scenarios covering 55 real CVEs, four mainstream models crossed the boundary in up to 96.8% of risky scenarios, while three mainstream mitigations stopped violations only by also stopping legitimate recoveries. nrepair instead scores the agent's context at each substitute call and, on a trigger, withholds the call once and returns the decision to the agent, lowering the violation rate on Luna from 67.5% to 16.2% while normal success on Qwen and Gemma falls only slightly.
Interpretation
When a tool errors and the agent switches to a substitute tool, that recovery can cross the task's action boundary, and whether it does depends on the target's current state rather than on the call itself. Earlier robustness studies scored only whether the task was completed, and risk evaluations bound risk to a tool or a request; neither fixed the same call while varying only the target's state. Across 304 scenarios (150 normal and 154 risky, spanning eight domains and 55 real CVEs), DeepSeek, Luna, Qwen and Gemma crossed the boundary in 23.5% to 96.8% of risky scenarios, while completing 83.3% to 98.0% of normal scenarios through the substitute.
The crossing is not caused by the agent failing to read the target's state: in 90.4% to 100% of violating runs, that state was already in context before the substitute call. Given the same retrieved context outside the agent loop, the models identified the boundary in 70.1% to 96.8% of risky scenarios, yet as agents they stopped in only 3.2% to 76.5% of them. Even in the risky scenarios a model crossed as an agent, the same model identified the boundary offline in 69.1% to 89.4% of cases, placing the gap between evidence and decision.
nrepair scores the agent's context at each substitute call and, when the score exceeds a threshold calibrated on normal runs alone, withholds the call once and returns the decision to the agent with a prompt to check the record. Three mainstream mitigations (policy reasoning, runtime rules, guard models) judge the pending call, which is identical in both scenario types, so they reduce violations only by sacrificing normal recoveries, losing 0.62 to 0.99 normal successes per prevented violation. On Qwen and Gemma, nrepair lowered the violation rate from 96.8% to 77.3% and from 88.3% to 55.8%, with normal success falling by only 0.7 and 2.0 points; on Luna the violation rate fell from 67.5% to 16.2% while normal success fell from 92.0% to 79.3%.
On a real repository, the production agents Claude Code and Codex also forced-pushed after reading another author's commits, and nrepair attached as a hook sharply reduced those crossings. Without repair, Claude Code discarded the other author's commits in 11 of 16 risky runs and Codex in all 16, even though both read those commits before every risky forced push. With nrepair, crossings fell to 1 of 16 runs on Claude Code and 4 of 16 on Codex, while normal scenarios succeeded in 15 and 16 runs respectively.
Perspective
The demonstrated scope is 304 paired scenarios generated by large language models from real CVEs, plus a one-task-family case study of 16 runs each for Claude Code and Codex on a real repository; nrepair runs as a hook before tool calls and reads only the task, the agent's thoughts and tool returns, not the environment itself.
nrepair judges only evidence the agent has already retrieved, so a fallback taken before the target's record is read cannot be flagged; such runs account for at most 9.6% of violating runs. The repair is bounded by how strongly the agent responds to the prompt: Qwen still calls the substitute in most triggered risky runs, while Luna abandons about half of triggered normal tasks. The threshold is specific to the model and task type and must be recomputed from normal runs when either changes.
