MedPhysBench puts medical physics hand calculations and error checks on the exam table: four models near perfect on calculation, yet one accepts a correction it always applies itself
MedPhysBench turns medical physics hand calculations and the checking of a colleague's work into an automatically graded benchmark: ten seeded generators cover radiotherapy, diagnostic imaging, nuclear medicine and radiation protection, reference answers come from code implementing the governing protocols, and grading only converts units and applies a tolerance band. On 1,200 held-out test items, 12 models scored 33.3% to 100.0% on calculation, with four at 99.3% or above; five models detected at least 96.7% of planted errors with at most 4% false alarms in both checking formats. The same model can compute correctly and check incorrectly, so the two abilities need separate measurement.