Agents must learn unfamiliar software by using it, then rebuild it: the best system fully reproduces under a quarter of SWE-Recreation's requirements
SWE-Recreation turns "understand by using, then rebuild" into a measurable task: an agent sees only rendered pixels and acts through pointer and keyboard, with no DOM, source, logs or back-end. The benchmark spans 54 real applications (28 websites and 26 Android apps) and 1,446 hidden requirements; across five frontier systems, the best reaches 0.530 weighted fidelity and at most 22.9% fully reproduced requirements under application-equal averaging.