Frontier models drive real robot arms: Reins gets GPT-6.1 Sol to 88% success on five long-horizon tasks
Synopsis
Frontier multimodal models can plan and call tools, yet they struggle to drive real robot arms through long-horizon, contact-rich manipulation, blind while the arm moves and while the model thinks. Reins lets an off-the-shelf model operate a dual-arm robot through 3D perception tools and atomic actions that report contact and grip, chains actions with stopping conditions, and has a second model instance watch the cameras and report what changed. On five real tasks, GPT-6.1 Sol succeeds in 88% of episodes from natural-language prompts with no task-specific code; without the harness it never lifts the correct mug in the shell game and takes 8 times as long.
Interpretation
An off-the-shelf frontier multimodal model can drive a real dual-arm robot through long-horizon manipulation without robot training, using only a tool interface. Frontier models used as robot policies had mostly been evaluated in simulation, and real-hardware attempts moved a Franka and a bimanual AgileX in 2–4 cm steps, mostly pick-and-place; on RoboDojo's real Sweep Blocks, ten vision-language-action models trained on 100 demonstrations each never succeed. Across five real tasks with five episodes per condition, GPT-6.1 Sol succeeds in 88% of episodes; without the harness it never lifts the correct mug in the shell game, closing the gripper 6–9 times in 60 calls over four episodes without lifting a mug, and once lifting the wrong one.
The model writes actions as chains that carry check conditions, executing several steps in one call and stopping at the first step whose condition fails. The comparable prior harness runs its action chunks open-loop, without checking each step; here every step carries a condition that must hold after it runs, plus an optional repeat count. A chain holds up to eight tool calls and sixteen runs, with the condition checked after each step; all three frontier models open the drawer every time, and they separate on the shell game, whose scene changes while the actor thinks.
A second instance of the model acts as a narrator, watching the cameras continuously and sending back a text entry describing what changed in each window. A prior self-monitoring agent checked its own execution at 0.2 Hz, slower than a narrator that watches frame by frame. The narrator watches the head camera at 1 frame/s, and each window lasts as long as the previous call, so the newest reported frame is 1–2 latencies old; Sol's narrator answers in 1–6 s and sees a shuffle in 8–11 windows, while Astra's and Opus's take 3–40 s and see it in 2–4.
Perspective
The demonstrated setting is a tabletop platform with two 6-DoF AirBot Play arms and RGB-D cameras on the head and each wrist, on the shell game, drawer, towel, shirt and sweep tasks, with five episodes per condition and success judged by the operator. Task knowledge enters only through natural-language prompts, and contact is sensed only from joint current and gripper width.
A slower narrator reports fast changes later, and the newest reported frame is 1–2 call latencies old; Opus's narrator never tracks the red object in five episodes, and its two wins were guesses. Evaluation covers one platform with five episodes per condition, and contact sensing is limited to joint current and gripper width.
