EvoCodeAgent turns the architecture diagram into a project state that code can rewrite, lifting unit-test pass rates by up to 53.5% under GPT-5
Synopsis
Project-level code generation asks a model to produce a complete multi-file repository from requirements, with consistent cross-file dependencies, yet existing methods treat the architecture design as a one-time blueprint that drifts away from the implementation. EvoCodeAgent replaces that blueprint with a project state updated throughout generation, reconciles architecture and code dependencies using evidence from verification tools, and abstracts past repair trajectories into reusable repair experience. On the DevBench and CodeProjectEval project-level benchmarks under GPT-5, unit-test pass rates reach 60.48% and 27.73%, up to a 53.5% improvement over the strongest baseline.
Interpretation
Project-level code generation no longer treats the architecture design as a one-time blueprint; the architecture is continuously rewritten by generated code and verification results. Earlier approaches either generated code directly from requirements without an explicit architecture stage, or built a structured intermediate representation during early planning and then used it as a fixed blueprint. On the two project-level benchmarks of different scales, DevBench and CodeProjectEval, EvoCodeAgent reaches unit-test pass rates of 60.48% and 27.73% under GPT-5, up to a 53.5% improvement over the strongest baseline.
The system maintains a project state initialized from the product requirement document, UML diagrams and architecture design document, recording structural nodes, predicted relations, realized relations, artifact snapshots and verification feedback together. The skeleton stage infers predicted relations from import statements, cross-file symbol matching and semantic similarity between code elements, and the code stage extracts realized relations from generated code by static analysis; realized relations progressively dominate context retrieval in later iterations. On CodeProjectEval under GPT-5, the average project is 19,376.06 tokens, while context retrieved through co-evolution averages 6,560.93 tokens, 33.9% of the total.
Judge agents no longer rely only on the LLM's own assessment of intermediate artifacts; they call structure, interface, missing-file and skeleton-coverage verifiers and write explicit evidence back into the project state. Earlier multi-stage frameworks mainly had the LLM judge intermediate artifacts on its own under long context, without checkable evidence. On CodeProjectEval under GPT-5, removing these tools drops the unit-test pass rate from 27.73% to 10.88% and produces the largest fall in architecture consistency.
Artifact snapshots and verification outcomes before and after each revision are compared and abstracted into repair experiences made of an applicable scenario, a corrective strategy and an observed outcome, retrieved later by error message. Earlier iterative feedback was kept as local diffs or recent revision history tied to specific files and error traces, so similar issues in new files had to be rediscovered. On CodeProjectEval each task accumulates 26.4 experiences on average with a 31.9% retrieval rate; after retrieval, 35.3% yield improvement, 61.8% show no observable effect and 2.9% regress.
Perspective
Evaluation is limited to the Python subset of DevBench (10 tasks, 2.2 files and 276 lines of code on average) and the 18 Python tasks of CodeProjectEval (12.7 files and 2,388.6 lines of code on average), with DeepSeek-v3.2 and GPT-5 as backbones and identical PRD, UML and architecture-design inputs. The authors list extension to other programming languages as future work.
The authors list limited dataset coverage, possible bias from backbone model choice, and degraded generation when requirements are incomplete or contradictory as threats to validity. Experience retrieval depends heavily on similarity between the current failure and past cases, and most retrieval events show no observable improvement.
