Skip to main content
Search

# AmPermBench

1 related articles

Subscribe to updates
  1. arXiv

    Take the tools away from the agent: intent contracts let a trusted model write every call, but cannot stop it picking the wrong target

    For tool-using LLM agents, intent contracts remove tools from the planning model: at each step the planner submits only a contract with a goal, a verbatim quotation of the user and scope fields, code validates it, and a small trusted model checks it against the user's words and writes the tool call itself. On 96 AmPermBench prompts across three task families, every object changed under contract execution was named in the goal of the contract that produced the change (56/56), against 2/16 and 4/54 for the two review conditions; yet contracts did not reduce unauthorized actions, matching blind review at 10/96, and after environment terms were clarified every acting condition hit a wrong target in 40–49% of runs.

You’re all caught up