Skip to main content
Back to timeline
arXivSource publication:

General AI models drive a robot arm untrained: up to 44% on LIBERO-10, but extra reasoning only helps some

Synopsis

A University of Texas at Dallas team let six frontier AI models control a Franka Panda arm in closed loop on the ten long-horizon LIBERO-10 tasks, using only camera images, robot state and three tool calls, with no robot-specific training. Across 18 configurations and 1,800 episodes, GPT-6 Astra reached the highest success rate at 44% under high reasoning effort, with Gemini-4 Argon close at 43%; raising reasoning effort generally improved success, but by amounts that varied by model and at higher API cost.

Interpretation

Frontier general-purpose AI models can act directly as closed-loop manipulation policies without any robot-specific training: a model sees only two camera images and a 12-value robot state, and issues tool calls that specify end-effector pose and gripper commands. Earlier demonstrations used different tasks, robots and control interfaces, making systematic comparison difficult; LIBERO-10 is a common evaluation set for robot-learning methods such as Octo, OpenVLA and π0.5. Six models were evaluated at low, medium and high reasoning effort with 100 episodes per configuration, giving 18 configurations and 1,800 episodes, all zero-shot under a task-agnostic system prompt.

Each turn the model issues exactly one tool call: move_to gives a target pose and gripper command, while done and give_up end the episode. An adapter interpolates each move_to target into a straight-line trajectory at 20 Hz and hands it to LIBERO's operational space controller, with task success checked after every control step. Evaluation covers all ten LIBERO-10 tasks with the first ten initial states each, under limits of 80 model calls and 900 control steps per episode.

Success rates rise with reasoning effort, but not for every model. GPT-6 Astra improved from 28% at low effort to 44% at high, Gemini-4 Argon from 34% to 43%, GPT-6.1 Sol from 24% to 33%, and Claude Opus 5.5 from 9% to 33%; Claude Opus 5 instead fell from 12% to 6%. GPT-6 Astra and Gemini-4 Argon at high effort cost roughly $7–8 per episode in API calls, while GPT-6.1 Sol and Claude Opus 5.5 reached 33% at roughly $1.6–2.0 per episode.

Perspective

The evaluation covers the ten long-horizon LIBERO-10 tasks with the first ten initial states each, on a Franka Emika Panda arm with a parallel-jaw gripper, run zero-shot entirely in simulation under a system prompt without task-specific hints or demonstrations.

The failure explanations are generated by the models themselves and may not accurately explain the causes of failure; the authors treat them as useful hypotheses. Commanded and achieved displacements also diverge, with achieved displacement averaging about 64–72% of the commanded displacement.