Skip to main content
Back to timeline
OpenAI NewsSource publication:

Astra for Law tops Harvey LAB-AA after update: 20.8% full credit on 120 legal assignments, 0.03 hallucinations each

Synopsis

OpenAI updated its legal model configuration Astra for Law and opened it for access requests to every ChatGPT Enterprise customer. The configuration builds on GPT-6.2 Astra and adds an expanded legal search index plus instructions for researching a question, weighing the authorities and writing the answer. On 120 Harvey LAB-AA assignments it earned full credit on 20.8%, above GPT-6.2 Astra at 15.0% and Muse Wonder Full at 13.1%, with 0.03 material hallucinations per assignment, tied for the lowest rate with GPT-6.

AI-generated editorial illustration: Astra for Law leads in legal intelligence

Interpretation

Astra for Law now ranks first on Harvey LAB-AA, a benchmark that asks a model to take a legal assignment from start to finish and meet every requirement without inventing a material claim. The configuration had previously been available only to a small group of firms, including Cooley, Ropes & Gray and Sullivan & Cromwell. On Artificial Analysis's 120-assignment version spanning 24 areas of law, Astra for Law earned full credit on 20.8% of assignments, versus 15.0% for GPT-6.2 Astra and 13.1% for Muse Wonder Full.

The configuration pairs a frontier model with an expanded legal search index and instructions for researching a question, weighing authorities and writing the answer, rather than training a separate legal model. Since September the legal search index has grown 34%, adding more than 80 million new URLs, including jurisdictions such as the UK, Australia and Canada. The index now spans over 310 million URLs and covers more than 99.9% of published U.S. precedential case law; on Harvey's own LAB, GPT-6 Astra met every requirement on 4.6% of assignments, GPT-6.2 Astra on 14.6%, and the legal configuration on 16.2%.

Perspective

The demonstrated scope is the 120 Harvey LAB-AA assignments, the LAB: USCL brief-writing evaluation co-developed by Harvey and Grubel, and U.S. federal and state law questions written by practicing lawyers on Vals AI's Legal Research Bench; readers should treat it as benchmark results disclosed in a vendor announcement.

In the supplied-precedents version of LAB: USCL, Astra for Law scored 1,883 against Opus 5.5 at 1,882, effectively even, and the charted uncertainty ranges overlap; on the Vals AI bench the full-credit rate rose from about 46% to 53.8%, leaving nearly half the questions without full credit.