Skip to main content
Back to timeline
arXivSource publication:

Long-horizon research agents stop carrying their chat: a harness holds the state, and kernel tuning saves over 84% of tokens

Synopsis

In long-horizon automated research, more inference does not by itself produce more progress: agents replay growing histories, duplicate one another's work, or stop experimenting while tokens keep burning. Stateless language agents move research state out of the conversation into a harness that stores candidate solutions and measured outcomes and rebuilds a role-specific context for every invocation; across software engineering, kernel optimization and algorithm design they reach the best final result on every task and match the strongest kernel baseline's final performance with over 84% fewer tokens.

Source-provided article image: Stateless Language Agents: Scaling Long-Horizon Automated Research
Figure 1 · arXiv

Interpretation

Long-horizon research systems now let agents carry no conversation, with research state held by an external harness and each invocation seeing only a context rebuilt for its role. Earlier agent frameworks kept each agent in one long-running session, leaving the research history inside its own conversation and resuming that same session after a restart. On Anthropic kernel optimization, all three stateless-agent runs beat all nine baseline runs at 1B tokens, with the worst run at 1122 cycles against 1191 for the best baseline.

What each agent sees becomes an explicit design choice rather than a by-product of an accumulating conversation. Previously the context grew with the session, mixing candidate solutions, evaluation results and failed attempts into one history. The harness rebuilds an evidence summary across search directions for the planning role and gives each execution role only its task, assignment, retained candidate and local feedback; from shared checkpoints, removing either context mechanism lowers mean progress.

The planning role turns evidence into concrete experiments and assigns them to parallel execution roles at small coordination cost. Previously parallel agents chose their own experiments and could implement the same feature repeatedly. The planning role consumes 0.24–0.51% of tokens and 1.20–2.29% of model cost, against 8.39–10.70% of tokens and 6.14–7.35% of cost for SwarmResearch's Shepherd.

Perspective

The demonstrated scope is three task categories with executable evaluators: FrontierSWE software engineering, Anthropic and SOL-ExecBench kernel optimization, and FrontierCS algorithm design, at budgets up to 1B cumulative tokens. Goals that are vague or slow to evaluate remain untested. Because long-horizon runs are costly, most configurations use a single run; only Anthropic kernel optimization and the checkpoint ablations are replicated. Comparisons are matched on tokens, and cost estimates exclude the compute for running and evaluating experiments.

The ablations change what each agent sees, not whether it keeps its conversation across invocations. For the planning role the open question is decision quality, not cost; on FrontierSWE 15 workers are never the best width, though fewer trials per lane may also explain this. The best worker count varies with task, token budget and time constraint, and cheaper worker models outperform stronger ones at matched cost on some tasks but not others.