Skip to main content
Back to timeline
Hugging Face - BlogSource publication:

AI2's XYBench tests 8,000+ real questions to see whether AI assistants catch a user's mistaken premise

Synopsis

AI2, with Carnegie Mellon, NVIDIA and Johns Hopkins, released XYBench, a benchmark of 8,115 real questions drawn from Stack Overflow, Stack Exchange, hand-curated examples and WikiHow, testing whether AI assistants recognize the mistaken premise behind a user's request. On Stack Overflow questions, the five evaluated models offered a solution to the stated request in 75–86% of responses but a solution to the underlying goal in only 59–71%; even when the user's intent was supplied, misconception identification did not reliably rise.

Source-provided article image: XYBench: Helping with the problem behind the question
Hugging Face

Interpretation

XYBench labels the approach a user asks about separately from the goal the user wants to reach, so it can measure whether an assistant corrects a mistaken premise in its answer. Earlier assistant evaluations mostly target well-specified questions with the right approach, while XYBench collects cases where the asker has already settled on an approach and the real problem lies elsewhere. The benchmark holds 8,115 questions: 1,557 from Stack Overflow, 184 from Stack Exchange, 102 manually curated, and 6,272 generated from WikiHow guides.

Each response is scored on three aspects: whether it contains solutions to the stated request and to the intended request, where its emphasis falls, and whether it steers the user away from the mistaken approach. Claude Sonnet 4.5 serves as the automated judge, classifying individual suggestions in each response and assessing the answer's overall emphasis. Two human reviewers independently assessed 500 model responses; agreement with the judge ranged from 97.5–100% on whether a solution appeared, 84.6–93.3% on misconception identification, and 74.2–79.8% on pragmatic emphasis.

On Stack Overflow questions, models more often answered the approach the user literally asked about than the goal behind it. Solutions to the stated request appeared in 75–86% of responses, compared with 59–71% for solutions to the underlying goal. On the same Stack Overflow questions, models focused primarily on the underlying goal in only 35–41% of responses and on the stated request in 47–57%.

Supplying the user's intent alongside the question raised the rate of goal-focused solutions but lowered misconception identification. On Stack Overflow, Sonnet 4.6's rate of including a solution to the underlying goal rose from 59% to 67%, while its misconception-identification rate fell from 47% to 34%. Across the sources with human reference answers, every model in this setting still trailed humans on misconception identification.

Perspective

Evaluation runs in a single-turn setting with one response per question; questions are mainly English and mainly procedural problems such as cooking and home repair. The authors propose testing which training methods help models act on inferred intent, and extending the benchmark to other languages, cultures and conversational settings.

The authors note that supplying intent helped but did not consistently close the gap with human responses, suggesting that detecting intent is only part of the challenge and models also need to act on it when it diverges from the stated request. The benchmark currently covers English-language queries and mainly procedural problems.