Skip to main content
Back to timeline
Microsoft ResearchSource publication:

TokenEconomizer lets coding agents learn from past sessions: 18.4%–30.9% lower cost across 277 real bug-fixing tasks

Synopsis

Developers use AI coding agents in the same codebases day after day, yet agents often start each session from scratch, rediscovering familiar facts, loading unused tools and repeating unsuccessful approaches. TokenEconomizer, an open-source plugin from UC Berkeley, Microsoft Research and Microsoft Security, turns completed session logs into guidance for later sessions without modifying the agent or its underlying model. Across 277 real bug-fixing tasks in six repositories, it cut average cost by 18.4%–30.9% for five agent and model combinations, saving in all 30 repository-combination pairs while task success rates stayed broadly stable.

Source-provided article image: TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient
Figure 1 · Microsoft Research

Interpretation

An open-source plugin can make an AI coding agent cheaper the longer it works in one repository, without changing the agent itself or the model it calls. Earlier cost-cutting efforts mostly targeted individual tasks or relied on a fixed saving strategy; three popular open-source plugins tested each lowered costs in some repositories but not others. On 277 SWE-Bench tasks from six repositories with five AI coding agent and model combinations, average cost fell 18.4%–30.9%, with savings in all 30 repository-combination pairs and no performance loss; by contrast, lowering the model's reasoning effort saved similar or larger amounts but caused substantial performance degradation.

After each session the plugin reads the session log, updates what it knows about the repository, and acts through interfaces the agent already provides. It relies on three mechanisms: micro-skills that record repository-specific knowledge as one-line notes, tool pruning that removes tools repository history suggests the agent is unlikely to use, and failure-loop intervention that suggests a different approach when an agent keeps hitting the same error. A micro-skill note enters the starting context only after several sessions confirm the pattern and is retired if later evidence contradicts it; tool pruning always keeps core tools for running commands and editing files, and removed tools can be restored; a judge model decides whether the agent is truly stuck, and the open-source plugin alerts the developer rather than stopping the work itself if the agent is still stuck after the note.

Savings grow as experience accumulates and carry over to sequential web development. Early tasks can cost more while the plugin is still learning, but cumulative costs fell below the default in every repository by the sixth task. Costs were 16.0%–21.2% lower on sequential web development in Web-Bench; on security auditing, where every task is in a new repository, costs were still 6.4%–19.6% lower; each of the three mechanisms contributes savings on its own, and combining all three delivers the largest reduction.

Perspective

The demonstrated scope is 277 tasks from six SWE-Bench repositories with five AI coding agent and model combinations, plus two additional settings, sequential web development on Web-Bench and security auditing; each repository's evaluation starts with no prior session history. The plugin targets developers and researchers using existing AI coding agents and is designed to run alongside them without modifying their underlying models.

Savings grow as experience accumulates, early tasks can cost more while the plugin is still learning, and cumulative costs fall below the default only by the sixth task. In the open-source plugin, if the agent is still stuck after a suggested alternative, it alerts the developer rather than stopping the work itself. In security auditing every task is in a new repository, and the cost reduction there is 6.4%–19.6%, lower than in same-repository settings.