Harness 2 lets agents rewrite their own runtime scaffolding without ground-truth answers, lifting GLM-5's WorkBuddy-Bench average by 11.5 points in one step
Synopsis
Google Cloud AI Research released Harness 2, a plug-and-play framework for recursive agent harnessing that automatically rewrites the prompts, memory, tools and orchestration around a frozen model across an open-world task stream with no ground-truth answers. A contrastive Propose–Probe–Compose step compares executions of the same task before and after an edit, and lessons from those edits update a continual improvement harness for later tasks. On WorkBuddy-Bench, JobBench and Harvey's Legal Agent Benchmark, a single improvement step beat the base harness in all four executor–harness configurations, with gains of up to 11.5 points, 4.5 points and 11.8 percentage points.
Interpretation
An agent's performance on open-world tasks depends not only on the underlying model but on its execution harness, the prompts, memory, skills, tools and orchestration around it; Harness 2 lets that layer be rewritten during deployment without changing model weights. Earlier automatic harness optimization selected the best configuration against fixed training tasks, unit tests or validation feedback before deployment, whereas open-world tasks are typically unseen during harness design and carry no ground-truth answers or reliable verifiers to guide online edits. Across three professional benchmarks, WorkBuddy-Bench, JobBench and Harvey's Legal Agent Benchmark, and four executor–harness configurations, a single improvement step beat each base harness, with maximum gains of 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench and 11.8 percentage points in LAB Partial pass rate.
Each improvement consists of Propose, Probe and Compose: the pipeline first runs a reference execution, then proposes one candidate aimed at general agentic deficiencies and one aimed at the current domain's knowledge, and executes both on the same task under matched model and environment configurations. The behavioral contrast between a candidate and the reference execution indicates whether the edits mitigate the targeted deficiency, leave the behavior unchanged or introduce regressions; it does not verify task correctness, because execution stochasticity and shared model biases limit the reliability of that evidence. Composition reads the reference execution, the candidate harnesses, their diffs against the current harness and all task-local execution evidence, and may merge supported changes, revise conflicts, adopt one candidate or keep the current harness; the composed harness is then executed and its decisions recorded.
Perspective
Evaluation covers the Code, Web and Office domains of WorkBuddy-Bench, the end-to-end professional tasks of JobBench, and the per-criterion Partial pass rate of Harvey's Legal Agent Benchmark, with Gemini 3.5 Flash and GLM-5 as executors, OpenCode and Codex as initial harnesses, and Claude Opus 4.8 as the default harness editor. Each benchmark's task stream starts from an empty improvement library and updates it after each task.
Behavioral contrasts are limited by execution stochasticity and shared model biases, so they evidence whether edits mitigate a targeted deficiency rather than task correctness. Recursion does not uniformly close gaps between initial harnesses: on WorkBuddy-Bench Gemini's gap falls from 2.1 to 0.4 while GLM-5's rises from 0.2 to 1.3. On LAB Partial, Meta-Harness still leads for Gemini 3.5 Flash under Codex.
