Skip to main content
Back to timeline
arXivSource publication:

Coding agents get a complexity receipt mid-run: erosion falls by up to 24.6 points, held-out correctness holds

Synopsis

Coding agents that write code autonomously over long runs are guided mainly by whether tests pass, so complexity accumulates as they extend their own work. MITL injects a short message after each code-changing turn, naming changed functions whose cyclomatic complexity exceeds a limit and changed lines the agent's own tests skip, giving it a chance to refactor or add tests first. Across three long-horizon benchmarks and four models, complexity feedback significantly lowered erosion, with a pooled reduction of 7.8 to 24.6 percentage points of starting erosion, while every per-model held-out correctness interval included zero.

Source-provided article image: Metrics-in-the-Loop: Complexity Feedback Reshapes Agent-Written Code
arXiv

Interpretation

The complexity concentration of code written by a coding agent during a long autonomous run can be pushed down while the code is being generated, rather than caught in later review. Agents previously steered mainly by whether tests pass, and passing tests do not reveal accumulating complexity or gaps in test coverage, so those weaknesses persist as an agent keeps extending its own code. On Commit0 and SlopCodeBench, complexity feedback significantly lowered erosion relative to the reminder-only control for all four models, with pooled intervals excluding zero and reductions of 7.8 to 24.6 percentage points of starting erosion.

The mechanism inserts a short report after each turn that changes code, naming functions over the complexity limit and changed lines the tests do not execute. Cyclomatic complexity counts the independent paths through a function, and more paths make a function harder to follow and to test; the report lists changed functions above a threshold of 6 in descending order and asks the agent to bring them under the limit. The experiments cover four models, DeepSeek V4.1 Flash, GLM-5.3 Flash, Claude Haiku 5.5 and GPT-6 Luna, on 25 Commit0 libraries, 35 SlopCodeBench problems and 35 SWE-EVO transitions, compared against reminder-only, no-message and coverage-only arms.

Perspective

The demonstrated scope is Python code, one harness, four models and one token budget per benchmark, with feedback delivered turn by turn during the agent's run. The authors propose a next step: a new benchmark of unseen specifications with long sequences of change requests, measuring change correctness and cost for code written with and without feedback.

Complexity feedback lowered erosion, but no model's held-out correctness interval excluded zero, and on SlopCodeBench a joint mixed model gives an odds ratio of 0.82 with interval 0.70 to 0.96, so a small cost of the combined arm on that benchmark cannot be ruled out. The authors also note that reports combine limits and measurements between turns, leaving target, content and timing effects confounded, and that current benchmarks have near-saturated pass rates or are dominated by between-library variance, so they cannot yet show a correctness payoff from better structure.