Skip to main content
Back to timeline
arXivSource publication:

Brain MRI can stay at the hospital: SCAN adapts a frozen vision-language model with local prototypes, lifting epilepsy AUC by up to 25.4%

Synopsis

Multi-site brain MRI models usually need centralized scans or repeated federated training; SCAN instead has each new site build local class prototypes on a frozen 3D medical vision-language model and share differentially private class statistics. At seven unseen sites, training-free local prototypes raised AUC at three epilepsy sites from 0.598, 0.625 and 0.508 to 0.684, 0.729 and 0.762, and mean AUC matched a classifier trained at each site (0.757 vs. 0.759). Under differential privacy at ε = 4, new-site contributions changed existing-site AUC by at most 0.2 percentage points, though gains from sharing were limited.

Source-provided article image: SCAN: Site-Aware Collaborative Privacy-Preserving Adaptation of Vision-Language Models for Neurological Disease Classification
Figure 1 · arXiv · Page 2

Interpretation

A new site can improve neurological disease classification using a small set of locally labeled scans, without retraining the vision-language model or uploading patient scans. Federated learning avoids centralizing raw images but requires repeated rounds of model updates and training infrastructure at every site, and shared updates can still leak patient information. Across seven unseen sites, training-free local prototypes raised AUC at three epilepsy sites from 0.598, 0.625 and 0.508 to 0.684, 0.729 and 0.762, and by 3.0% at AD site 62, with mean AUC of 0.757 versus 0.759 for a logistic regression trained at each site.

Sites exchange only noised class embedding sums and counts, from which a receiving site rebuilds prototypes and adjusts the frozen model's predictions. Each site keeps exact local class prototypes for its own diagnosis, while sharing adds Gaussian noise once to class embedding sums and counts, calibrated with the analytic Gaussian mechanism to satisfy (ε, δ)-differential privacy without repeated noising or retraining. The evaluation covers 7,641 T1-weighted brain MRI scans from 67 sites in 12 countries spanning Alzheimer's disease, stroke, epilepsy and controls; the noise scale is 1.53 at ε = 4 and 5.28 at ε = 1, with δ = 10⁻⁵.

Perspective

The evaluation covers 7,641 T1-weighted brain MRI scans from 67 sites in 12 countries, with Alzheimer's disease, stroke, epilepsy and healthy controls as diagnostic categories, scored by disease-versus-control AUC at seven unseen sites; each unseen site randomly holds out 20% of each class as a support set and evaluates on the rest. The authors state that each site should validate adaptation on held-out scans, particularly with small support sets, and that further validation is needed before clinical deployment.

The authors note that only one vision-language model (3D DenseNet121 with ClinicalBERT) and one support/evaluation split were evaluated, and other backbones and splits may behave differently; the differential privacy guarantee is per scan and equals patient-level privacy only when each patient contributes one scan; the vision-language model and source cache are held by a trusted coordinating site and are unprotected in the current version. Gains from cross-site sharing were limited, with recipient-site improvement of at most 0.7 percentage points at ε = 4.