Skip to main content
Search

# A2Z GameSpec-Bench

1 related articles

Subscribe to updates
  1. arXiv

    A2Z GameSpec-Bench tests coding agents on 100 long-form GDDs: near-99% verifiable rate, but top overall GDD Fidelity only 77.0

    The work introduces A2Z GameSpec-Bench, a benchmark of 100 long-form game design documents (50 Small and 50 Big, averaging 14,085 and 26,297 tokens) that turns each GDD into a fixed Dependency-Aware Contract and evaluates coding agents' end-to-end game development faithfulness through source-code inspection, scenario-based replay, and adaptive playtesting, finding that compilable and runnable outputs do not imply faithful implementation (e.g., Claude-Fable-5.1 reaches a 98.7% verifiable rate but an overall GDD Fidelity of 77.0), that dependency context raises coverage of recorded playtest violations from 71.1% to 80.2%, and that requirement-specific feedback improves overall GDD Fidelity by 10.9% relative to self-revision after two rounds on Big GDDs.

You’re all caught up