Why coding agents burn money: repeated runs of the same task can differ 30-fold in token use
Synopsis
On SWE-bench Verified, eight frontier models ran 500 real GitHub issues through the OpenHands agent framework, four independent runs per issue. Agentic coding consumed roughly 3,500 times the tokens of single-turn code reasoning and 1,200 times those of multi-turn code chat, with cost driven mainly by input tokens fed back repeatedly rather than by output tokens. Token use varied widely across runs of the same task, the most expensive run costing about twice the cheapest, and higher token spend did not correspond to higher accuracy.
Interpretation
Agentic coding tasks cost far more tokens than ordinary code tasks, and the money goes mostly to the input side. Earlier code reasoning and code chat tasks typically involved single-turn solving or multi-turn discussion, without tool calls or repository exploration. Across the 500 SWE-bench Verified problems, agentic coding averaged about 3,500 times the tokens of single-turn code reasoning and 1,200 times those of multi-turn code chat, with a much higher input/output ratio even when token caching was enabled.
The agent carries the full conversation history unchanged into each round, so prior context is resent to the model and input tokens accumulate with every round. The OpenHands agent framework lets the model read repositories, call tools and execute commands autonomously until the task ends, with no human intervention. Eight frontier models (Claude Sonnet-3.7, Sonnet-4, Sonnet-4.5, GPT-5, GPT-5.2, Qwen3-Coder-480B-A35B-Instruct, Kimi-K2 and Gemini-3-Pro) each ran four times on the 500 problems, and all token metrics are averaged over the four runs.
High token spend does not mean more accurate solving; the costliest runs tend to repeat views and edits of the same file. After ranking the four runs of each problem by token cost into minimum, lower, upper and maximum cost groups, accuracy rose slightly from the minimum to the lower group and then saturated at higher cost levels. Mixed-effects regression showed that repeated file views and repeated file modifications were significantly more frequent in the upper and maximum cost groups than in the minimum cost group, indicating that some expensive runs are inefficient back-and-forth exploration.
Perspective
The conclusions rest on the OpenHands agent framework and 500 real GitHub issues from SWE-bench Verified, with four runs per issue for each of eight frontier models; the authors note that the model coverage is still only part of the agentic model landscape and release the experimental pipeline for replication and extension.
The authors state that compute cost limited the study to eight models, so validation on a wider range of architectures and agent designs remains open; pre-execution self-prediction correlates only weakly to moderately with real usage (up to 0.39), all models systematically underestimate actual token consumption, and precise instance-level prediction remains an open problem.
