A language model edits 65 embedding rows in a robot policy, lifting success on reworded instructions from 74.3% to 87.5%
Synopsis
GPT-6 Astra reads and rewrites the weights of a robot foundation policy directly, without gradients or new demonstrations, and locates one class of failure in the token-embedding layer. On the pi05_libero checkpoint, three independent editing sessions each improved held-out paraphrase success; the strongest changed only 65 embedding rows, raising success from 74.3% to 87.5% on the 335 instructions containing an edited token while the other 430 outcomes stayed unchanged. In 13 further sessions targeting other failures, the editor declined 12 candidate edits, so the demonstrated scope is lexical grounding repair.
Interpretation
A language model can fix one class of instruction-understanding failure by changing very few embedding rows of a robot policy: the skill already exists, and the edit changes which wording reaches it. Earlier weight-space programming demonstrations covered games and language models, not robot foundation policies; such policies complete a task under the canonical instruction but pick the wrong object or task when given a synonym or description. On the pi05_libero checkpoint, the strongest candidate C changed 65 embedding rows and moved success on all 765 held-out language variants from 660 to 704, or 86.3% to 92.0%, with 48 repairs and four regressions; the 335 instructions containing an edited token rose from 74.3% to 87.5%, and the other 430 outcomes did not change.
The edit rewrites selected source-token embedding rows toward task-anchor words, a remapping at the lookup level that leaves vision, attention and action-expert tensors untouched. An editing session receives weight tensors, a persistent CPU Python workspace and a budgeted forward tool, so it can inspect tensors and write replacements but cannot train the policy, change the deployment program or fit with gradients. Candidate C changed 132,768 scalar entries across 65 rows of the 257,152-row, width-2,048 embedding table, or 0.00367% of the exported checkpoint; moving the same 65 rows by matched distances in random directions reached only 85.2%.
The editor's retain-or-decline decisions are a second result: it withholds edits that fail its own checks. Beyond the language paraphrases, 12 of 13 final candidates were reverted or declined across sessions on FastWAM, LIBERO-Plus, LIBERO-PRO and LIBERO-RECOVER. The sole retained FastWAM value-projection candidate moved success from 48.0% to 48.7% on a 700-pair tagged sample, exact p=0.542, with no significant held-out gain.
Perspective
The demonstrated scope is two released checkpoints in simulation: the language-paraphrase repair on pi05_libero, and the decline record on FastWAM, LIBERO-Plus, LIBERO-PRO and LIBERO-RECOVER; the three language sessions share one held-out population and seed 7, so their independence lies in the search rather than in environmental coverage.
The authors call the obstacle-versus-ability distinction a productive hypothesis rather than a proven boundary: visual perturbations, recovery states and task changes produced no retained, independently confirmed repair under the tested budgets, which does not prove no suitable weight edit exists. A full-replacement control using the editor's own discovered lexicon already explains much of the gain, and its direct difference from the editor is not significant at 0.05; runtime rendering variation can also produce closed-loop discordances even when fixed-input computations are unchanged.
