Skip to main content
Search

# TestHelix

1 related articles

Subscribe to updates
  1. arXiv

    Passing one reference solution is not enough: TestPrism gives each task 10 implementations, and the best coding agent reaches only 28.00% joint success

    A Nanjing University team built TestPrism, a benchmark of 300 test tasks and 3000 labeled candidate implementations, to check whether generated tests actually separate correct code from incorrect code. Its main metric, the Joint Success Function, requires a test to fail on the initial program, accept every valid implementation and reject every invalid one; across fourteen baseline coding-agent configurations the best score is 28.00%, while single-reference success reaches 59.67%. The team also introduces TestHelix, which combines heterogeneous test-and-repair pairs, peer cross validation and recursive self improvement to raise that metric.

You’re all caught up