Skip to main content
Search

# RCP-nDCG@10

1 related articles

Subscribe to updates
  1. Cohere Labs

    Cohere proposes RCP-nDCG@10: a calibrated AI judge replaces fixed answer keys, matching human preference in 77% of 289 blind contests versus 52% for conventional nDCG

    Cohere introduces RCP-nDCG@10, a retrieval evaluation methodology in which a calibrated AI judge grades every retrieved document against the same explicit relevance rubric and combines rubric answers with group-wise comparisons and calibration, so relevant documents absent from the answer key can still earn credit; in a blind study with 46 annotators, 273 queries and 289 head-to-head contests, reviewers sided with RCP-nDCG 70% of the time where the two metrics named different winners, and across all contests RCP-nDCG matched reviewer preference 77% of the time versus 52% for conventional nDCG, with the gain driven mainly by crediting relevant documents the answer key missed (19 percentage points).

You’re all caught up