A gate before a personal health assistant rewrites its own instructions: 20 of 21 update attempts rejected, and an admitted repair cuts attack flags from 9/84 to 0/84
Synopsis
Safe Personal RSI, an update-evaluation component built for the personal health assistant Kicker, tests proposed instructions before adoption: across GPT-5.6 Sol, GPT-5.6 Luna and Qwen3-14B, two deliberately risky edits produced 21 update attempts and 20 were rejected, twelve on risk checks and eight on utility checks. One GPT-5.6 Luna LLM repair passed verification and admission, cutting attack flags from 9/84 to 0/84 and raising safe task completion from 78/87 to 87/87 on a fixed harness; a repair transferred from Qwen2.5-7B-Instruct to Qwen3-14B reduced violations from 58/84 to 15/84 under stronger attacks.
Interpretation
Once a personal assistant's reusable instructions are updated, the change alters what it discloses, which requests it follows and how it reports actions, so the update itself has to be tested before adoption. The prior path was direct adoption of a predefined edit or a single handwritten rule; on Qwen3-14B direct adoption raised attack flags from 21/140 under the initial harness to 39/140 for the autonomy edit and 55/140 for the stress edit. Across GPT-5.6 Sol, GPT-5.6 Luna and Qwen3-14B, two predefined risky edits yielded 21 attempts and Safe Personal RSI rejected 20, twelve on risk checks and eight on utility checks; the one admitted attempt was GPT-5.6 Luna's autonomy edit, admitted without repair.
The admitted repair removed attack flags in a fresh fixed-harness evaluation without giving up task completion. The unrepaired GPT-5.6 Luna update left 9 flags across 84 attack replays and 78 safe completions across 87 runs. After one LLM repair passed verification and admission, attack flags fell from 9/84 to 0/84 and safe task completion rose from 78/87 to 87/87, with the explicitly authorized health-sharing control succeeding in all three runs; a broad handwritten authority rule matched at 0/84 and 87/87.
A repair can transfer across models: one generated and admitted on Qwen2.5-7B-Instruct and moved unchanged to Qwen3-14B still lowered violations and raised safe completion under strengthened attacks. Under the same stronger attacks, the unrepaired Qwen3-14B harness recorded 58/84 violations and 17/87 safe completions. After transfer, violations fell to 15/84 (17.9%) and safe completion rose to 56/87 (64.4%), with flags on the 13 loop-excluded attack cases falling from 29/39 to 5/39; the static comparator here only prohibits sharing passwords and verification codes.
Beyond update evaluation, runtime probes check memory deletion and permission checks separately: a deleted memory entry reappeared after a real 11-minute idle wait, and a write request under a read-only OAuth grant reached the simulated provider without a local scope check. The entry was absent from the database immediately after deletion, while its source content and conversation history remained available. The reappearance pattern occurred under all four configurations; the write request was refused by the provider with 403 and no write succeeded, and the probe concerns the evaluated Willow runtime snapshot.
Perspective
Experiments run offline on the Willow agent runtime with real model calls, synthetic accounts and simulated external services; mail can be read and drafts saved, while delivery and transactions are disabled, and health content appears in email, so cases concern authorized handling of private information rather than clinical quality. Evaluation uses fixed edits, known case families, repeated stochastic runs and a model-based auditor, with 16 of 29 cases used for update development and 13 excluded from each update loop.
The admission gate uses empirical point estimates and supplies no statistical safety guarantee, and comparing maximum category risk can allow a nonmaximal category or the overall violation rate to rise. All 20 template-repair attempts failed verification, and two GPT-5.6 Luna repairs cleared the confirmed cases but missed the utility floor. The memory probe tests only whether single-entry deletion prevents re-extraction, not complete forgetting of the person, and the OAuth probe records a local enforcement gap in the evaluated snapshot with no post-fix validation in the experiment records.
