Skip to main content
Back to timeline
arXivSource publication:

Robot agents stop asking the LLM at every primitive: ChunkAct compiles memory into action chunks and reaches 83.8% on LIBERO-Pro

Synopsis

In agentic robotics, where an LLM agent sits in the control loop, every perception or motion primitive costs a separate round of reasoning, and a single pick-and-place averages 24.2 calls and 2.2M input tokens. ChunkAct compiles memory offline into executable action chunks, so one call runs a chain of primitives and returns control only on failure. On LIBERO-Pro it reaches 83.8% overall task success, 5.0 points above Harness VLA, with 2.9 fewer calls and 4.4 and 2.3 fewer input and output tokens.

Source-provided article image: ChunkAct: Action Chunking for Agentic Robotics
arXiv

Interpretation

The planner in a robot agent moves from one query per primitive to one query per decision: a single call launches a chain of perception and control primitives that runs to completion on its own and hands control back only when it fails. Earlier agentic robotics systems had the LLM observe the scene, consult memory and issue a single tool call at every step, so calls grew linearly with the number of primitives and input tokens grew quadratically. On LIBERO-Pro, ChunkAct reaches 83.8% overall success, 5.0 points above the agentic Harness VLA; the two share the same planner, VLA primitive, perception and memory, and differ only in how memory is used.

Action chunks are compiled from memory before deployment, fixing the stage order of a sub-goal type, the primitive assigned to each stage, parameters conditioned on object class, fallback strategies and per-stage verification, so execution reads no memory at run time. Previously memory was retrieved at test time and read into context, and memory reads plus perception queries made up most calls, with each result appended so the context kept growing over an episode. On LIBERO-Pro, ChunkAct averages 8.3 calls, 0.5M input tokens and 3.7k output tokens per episode against 24.2, 2.2M and 8.5k for Harness VLA; with reasoning disabled, ChunkAct reaches 80.0% success and Harness VLA 56.0%.

The planner keeps only what cannot be compiled: understanding the task and scene, and after a chunk fails, re-grounding the target or falling back to primitives. Previously the planner served as both reasoner and executor, treating every step alike whether or not it required judgment. On LIBERO-Pro, the chunks alone reach 57.5% success under a scripted driver that makes no LLM call, against 83.8% for ChunkAct with reasoning, a gap that corresponds to the gain from failure recovery.

Perspective

Evaluation centers on the eight suite and perturbation combinations of LIBERO-Pro, each with 10 tasks and 10 seeds per task; on LIBERO-Plus each task is run with a single seed under each perturbation axis. The planner is GPT-5.6-sol, the VLA primitive is an RLinf-fine-tuned checkpoint, segmentation comes from SAM 3, and the budget is 100 LLM calls per episode.

The chunk library is compiled on LIBERO-Pro and includes task-specific memory; the zero-shot LIBERO-Plus results indicate it does not overfit to those tasks, but that conclusion is limited to the seven perturbation axes. With reasoning disabled, ChunkAct's success stays stable while Harness VLA degrades sharply, a contrast that reflects how heavily each system relies on the planner.