Skip to main content
Back to timeline
arXivSource publication:

An open 9B vision-language model goes from 0/100 to 84/100: EmbodyAgent chains black-box distillation, SFT and online RL into one post-training stack

Synopsis

EmbodyAgent turns an open-weight 9B vision-language model into a closed-loop robot agent that writes and runs Python control code. Untrained, Qwen3.5-9B failed all 100 held-out initial states across ten LIBERO Spatial tasks on output format alone, and supervised fine-tuning on teacher trajectories reached only 14/100; online outcome-based reinforcement learning on the agent's own sampled episodes raised that to 84/100 and cut median model calls per test episode from 36 to 11. All three stages share one execution interface exposing only raw observations and seven-dimensional end-effector control, with no action decoder.

Source-provided article image: EmbodyAgent: Post-Training Embodied VLM Agents via Distillation, SFT, and RL
arXiv

Interpretation

An open-weight 9B vision-language model can operate a robot in closed loop purely by writing Python programs, reaching 84/100 on held-out states from a starting point of zero. Under the same tool schema and prompt without post-training, Qwen3.5-9B failed all 100 held-out states, with every episode ending in a format-invalid output; supervised fine-tuning on teacher trajectories alone reached 14/100. Evaluation covers ten LIBERO Spatial tasks with disjoint per-task splits of 30 training, 10 validation and 10 test initial states; the 100 test states never enter synthesis or training and are re-scored after every RL round with greedy decoding, full episode budgets and no sampling-side safeguards.

All three training stages share one think-then-code output contract: each turn the model first states a plan for loop structure, action batching and observation timing, then emits an executable Python program. Earlier code-as-policy agents typically compose calls from a hand-engineered library of manipulation primitives, whereas this runtime exposes only env.observation(), returning primary and wrist camera images plus robot proprioception, and env.action(a), advancing one simulator step with a seven-dimensional end-effector command. Stage one uses the frontier vision-language model Astra Medium as a black-box teacher, drawing 1,620 attempts on training states and filtering by the environment success predicate to 100 successful episodes (ten per task), exported as 1,475 explanation-and-program targets; stage two trains Qwen3.5-9B with both vision encoder and language parameters trainable for ten epochs.

Online outcome-based reinforcement learning closes the imitation distribution shift and compresses decision frequency at the same time. Supervised fine-tuning only saw teacher-generated trajectories, while at test time the student must condition on its own programs, execution feedback and the resulting observations; RL optimizes directly on the student's own sampled episodes with no teacher and no learned critic. After 64 rounds held-out success reaches 84/100, temperature-one training rollout success rises from 8.4% to 87.7%, all ten tasks improve with nine at 8–10/10 and task 4 moving from 0/10 to 4/10, and median model calls per test episode fall from 36 after SFT to 11.

Perspective

The demonstrated scope is ten LIBERO Spatial simulation tasks, a teacher corpus of 100 synthesized episodes and one 9B student model, evaluated on a fixed set of 100 held-out initial states. The authors list extension to broader manipulation suites, real-robot execution, and controlled ablations isolating each stage and representation component (explanations, program structure, observation scheduling) as next work.

Each training stage reflects a single run without multi-seed replication, so the headline score on 100 test states carries a binomial standard error and each task contributes only ten states. Task 4 remains at 4/10, showing that outcome-based RL under sparse rewards does not remove per-task difficulty ceilings; fixing 100 evaluation states buys round-by-round comparability at the cost of coarse initial-state coverage.