Alibaba's PURE splits identity matching into 14 diagnostics: ten multimodal models span 47.1% to 86.0%, with rule judgment weakest
Alibaba researchers released PURE, a diagnostic benchmark for multi-granularity multimodal identity matching, with 1,457 multiple-choice questions across 14 tasks in four dimensions: Perception, Understanding, Reasoning and Judgment. Questions come from real e-commerce record pairs and were verified by two domain experts. Across ten recent multimodal large language models, overall scores range from 47.1% to 86.0%, with Understanding highest on average and Judgment lowest, showing that aggregate matching accuracy hides whether models struggle to interpret evidence or to apply matching criteria.