AnalogLayoutWorld gives analog layout agents a public proving ground: 91.1% of attempts yield GDS, but only 21.3% pass every acceptance check
Synopsis
General coding agents write code well, but their reliability on long-horizon engineering tasks governed by physical constraints remains unclear. AnalogLayoutWorld turns analog circuit layout into 40 expert-reviewed fixed-netlist tasks with an open EDA toolchain and independent physical, electrical and layout-quality scoring. Across 13 model-harness configurations, 91.1% of attempts produce GDS layouts but only 21.3% meet all acceptance criteria; the in-house Novara system posts the highest aggregate score, 0.8154.
Interpretation
Analog layout design becomes a publicly reproducible agent evaluation environment in which agents start from a fixed transistor-level netlist and decide for themselves how to build, verify and revise a layout. Earlier LLM layout methods concentrated on selected circuit families and specialized workflows, and some systems carried proprietary dependencies, so the reliability and reproducibility of autonomous layout design lacked broad, objective evaluation. The 40 tasks cover amplifiers, references, power-management circuits, sampling circuits and oscillators, and every circuit passes pre-layout simulation to establish a verified electrical baseline.
Evaluation separates handing in a file from actually completing the design: submitted GDS is re-checked by an independent process after the agent stops, and agent-reported results do not determine the score. Producing a layout artifact alone does not establish task completion, and violations of geometric and morphological constraints account for most of the gap between simulation pass rates and complete acceptance. Across 13 model-harness configurations, 91.1% of attempts produce GDS but only 21.3% meet all acceptance criteria, and the GPT configurations reach substantially higher physical and electrical verification rates than the other general-purpose agents.
Systems differ in which aspect of layout quality they lead, and the aggregate ranking reflects both verified circuit behavior and engineering quality. GPT-6 Astra leads in DRC and all-case post-layout simulation, while Novara leads in LVS and credited engineering quality. Novara ranks first with an aggregate score of 0.8154, followed by GPT-6 Astra at 0.7483; GPT-6 Astra reports 11.2M tokens per task on average, against 34.5M tokens and a 0.4779 aggregate score for GLM-5.3.
Perspective
The evaluation covers 40 fixed-netlist tasks in the SKY130 process and examines only physical implementation with predefined devices and specifications; topology selection and transistor sizing are outside the task scope, and results characterize performance within the included circuit families and scale range. Acceptance is defined by design rule decks, device models, extraction settings and electrical test cases, providing reproducible rule-based evidence under the stated conditions rather than foundry sign-off or a guarantee of silicon performance; the study includes no silicon measurements.
The layout-quality metrics are an initial approximation designed with analog layout engineers and may have incomplete coverage or assign low scores to otherwise sound layouts; the authors plan to refine them through expert review and community feedback. Verification progress differs substantially across configurations, and many trajectories still lack a recovered passing self-check within the displayed budget; under a fixed budget, the realized benefit of test-time scaling depends on reasoning efficiency and on converting environment feedback into verified improvements.
