Skip to main content
Back to timeline
发表出处待核验Source publication:

Diagnosis without a finished case file: Microsoft system scores 83.2% on NEJM cases against 36.1% for generalist physicians

Synopsis

A Microsoft team recast language-model diagnosis as a sequential encounter: given only a chief complaint, the system decides what to ask and which tests to order while continually revising its differential. It built an interactive benchmark, SDBench, and reached 83.2% accuracy on New England Journal of Medicine clinicopathological conference cases and 80.3% on MIMIC-IV-ED emergency department admissions, against 36.1% and 46.8% for generalist physicians, at lower total cost. These results come from a controlled benchmark setting.

AI-generated editorial illustration: Opens in a new window Opens an external website Opens an external website in a new window

Interpretation

Evaluation of language-model diagnosis moves from a single complete case summary to a chief complaint, with the system choosing what to ask and test next. Earlier clinical AI evaluations often relied on written vignettes, whereas real patients do not arrive with a tidy write-up of key symptoms and laboratory findings. On New England Journal of Medicine clinicopathological conference cases the system reached 83.2% accuracy versus 36.1% for generalist physicians; on MIMIC-IV-ED emergency department admissions the figures were 80.3% and 46.8%.

The system simulates a panel of physicians, continually revising the differential diagnosis and selecting cost-effective tests. The process runs on an interactive benchmark, SDBench, with test costs priced from a hospital chargemaster, so accuracy and workup cost are counted together. Evaluation covers two settings, journal case conferences and emergency department admissions, and the system is more accurate than human responders in both at lower total cost.

Perspective

The results come from a controlled benchmark setting: journal clinicopathological conference cases and MIMIC-IV-ED emergency department admissions, compared against generalist physicians working without the system.

The authors state that these findings were obtained in a controlled benchmark setting; the evaluation does not assess therapeutic appropriateness, safety, or longitudinal management, and clinician-AI workflows are identified as future work.