Pathology zero-shot prediction moves to weight generation: MI-Hyper tops MI-Zero by 1.85 AUC points across 308 tasks
Synopsis
Zero-shot whole-slide image prediction has mainly pooled patch-text similarities, the route MI-Zero takes. MI-Hyper instead uses a hypernetwork to generate the aggregation and classification weights of a multiple instance learning model directly from task and class descriptions, first training the generator jointly with a shared MIL basis on mixed pathology tasks and then refining the generator with the basis frozen. Across 308 tasks from 52 pathology datasets, mean AUC, balanced accuracy and macro-F1 exceed MI-Zero by 1.85, 1.17 and 0.57 percentage points, and some molecular prediction tasks gain more than 20 points in both AUC and balanced accuracy.
Interpretation
Zero-shot whole-slide prediction no longer relies only on pooling patch-text similarities; task descriptions now generate the weights of a multiple instance learning predictor. MI-Zero, using the same frozen image and text encoders, pools patch-class-text similarities into slide-level predictions with a predefined operator, so changing descriptions only changes similarity scores. On 308 tasks from 52 pathology datasets and 160,989 task-bag records, MI-Hyper reaches mean AUC 58.24 and balanced accuracy 45.66 against 56.38 and 44.49 for MI-Zero, gaps of 1.85 and 1.17 percentage points.
The generator is first trained with a shared multiple instance learning basis using source labels, then refined with that basis frozen, so text fixes task weights before patch features enter. A hypernetwork generates model parameters from conditioning information; here the conditioning is task context and class descriptions, with a class-name branch producing the attention query and a shared class decoder outputting each class's classifier row and bias. Training runs two stages of 15 epochs each over five folds of 112 semantic groups, target task IDs are held out from source optimization and checkpoint selection, and at deployment parameters are generated once per text condition and reused across cases.
Perspective
Evaluation covers 308 dataset-endpoint tasks from 52 pathology datasets and 160,989 task-bag records with fixed five-fold checkpoints, and the held-out target is new task IDs rather than an entirely unseen patient population.
Cases can recur in other source tasks: 100,573 query records match patient identifiers or feature files in the source training pool and 4,873 in the source validation pool, while 55,543 have no recorded match, so the results measure transfer to held-out task IDs.
