Skip to main content
Back to timeline
arXivSource publication:

A robot that improves without touching its weights: RPG lifts success on 22 manipulation tasks from 28.6% to 95.0%

Synopsis

RPG treats a robot execution system as a shared skill library plus a system prompt that can be revised repeatedly, practicing in simulation without updating model weights. It first identifies manipulation capabilities in an offline dataset and builds simulation practice tasks, then uses privileged execution and video-based diagnosis to rewrite skills, keeping a change only after cross-task validation. On held-out initializations of 22 manipulation tasks, success rises from 28.6% after the first practice round to 95.0% after 15 rounds, above ASPIRE at 75.5% and CaP-Agent0 with GPT-6 Astra Pro at 60.0%.

Interpretation

A robot execution system can keep rewriting its shared skill library and system prompt through simulation practice without updating model weights, raising task success substantially on held-out initializations. Earlier language-model robot agents either encoded perception and control in fixed programs or kept a multimodal model in the control loop for step-by-step reasoning, with skills and prompts typically developed and maintained by hand. Across 220 evaluations on ten held-out seeds for each of 22 manipulation tasks, RPG rises from 63/220 (28.6%) after round 1 to 209/220 (95.0%) at round 15, above ASPIRE at 166/220 (75.5%) and CaP-Agent0 with GPT-6 Astra Pro at 132/220 (60.0%).

Practice is driven by a contrast between two executing agents: one uses only observations available at deployment, the other additionally sees simulator state, and differences between their execution records localize failures. Failure diagnosis does not rest on a single trajectory; it compares the runtime agent's failed segments with the privileged agent's executions and with related videos from the offline dataset. With both the Video Analyzer and the Privileged Agent enabled, full RPG reaches 172/220 successes (78.2%) after five practice rounds; removing the Video Analyzer drops it to 114/220 (51.8%) and removing the Privileged Agent to 112/220 (50.9%).

A candidate revision must pass a cross-task regression gate before it joins the shared system, and the merged system is retested and rolled back if it fails. One edit to a shared skill can help one task and hurt another, so success on a single task is not enough grounds for retention. A candidate must raise mean success while no task loses more than one success out of five; in a controlled comparison that changes only the skill library with the runtime fixed, four tasks go from 26/40 to 37/40 successes.

Perspective

The demonstrated scope is 22 manipulation tasks for a two-arm YAM robot in MuJoCo simulation, plus ten trials on each of three physical tasks after a common calibration and hardware-adaptation procedure; physical deployment does not use the Privileged Agent or the Video Analyzer.

The evaluation curve is not strictly monotonic: successes drop by one at rounds 6 and 14, revisions that passed the five-seed development checks but did not improve every result on the ten held-out seeds. Towel folding accounts for seven of RPG's 11 failures, and the stricter folding variant stays at 0/10 across all 15 rounds.