General vision-language models cannot place cardiac structures: EchoLens recalls 87% across 21 structures after 136-patient training
Synopsis
On standard adult transthoracic echocardiograms, general and medical vision-language models and an open-vocabulary detector recalled 3% or fewer of cardiac structures, scoring below a box that never sees the image. EchoLens, a dedicated detector trained on 5,912 expert-annotated structures and 4,219 Doppler signals from 136 patients, recalled 87% on the internal test set, and its box centres fell within the reference as often as a blinded expert reader's (95.7% versus 95.3%), covering 21 structures across 11 views and 11 spectral Doppler signal types.
Interpretation
Where a cardiac structure lies in an ultrasound frame has to be learned, and does not emerge from large-scale pretraining. General and medical vision-language models and an open-vocabulary detector previously interpreted whole frames or studies without naming the structures behind their outputs; on the same frames they recalled 0% to 3.1% of reference structures. On the internal test set, counting one box per structure at an IoU of 0.5 or higher, EchoLens recalled 86.5% (95% CI 84.0 to 89.0), while all three comparison models scored below a per-class prior box that never sees the image (49.1% fitted on validation, 53.0% fitted on the test set itself).
The detector outputs each structure as a labelled box with a confidence score, and those confidences are summed by view to assign the frame's view. The method uses DEIMv2, a DETR-style detection transformer: a 640 × 640 frame passes through a ViT-Tiny+ backbone distilled from DINOv3 in parallel with a lightweight convolutional spatial tuning adapter that extracts fine detail, a hybrid encoder fuses the features, the 300 highest-scoring positions become object queries, and a decoder predicts a class and a box for each query. B-mode and Doppler detectors were trained separately on 1,035 B-mode frames and 1,806 Doppler frames from 136 patients (132 with Doppler); on external MIMIC-IV-ECHO, two physicians independently judged 81% to 92% of its detections correct across all 11 views, and the Doppler signal type was identified in 83% of frames.
Perspective
The demonstrated scope is 11 views and 11 spectral Doppler signal types of the standard adult transthoracic examination, with development data from a single center and 136 patients of median age 73 years, 85% of them men; external evaluation used physician-reviewed MIMIC-IV-ECHO samples and 50 CAMUS patients. The authors propose testing next whether EchoLens can generate grounding labels across large unannotated archives and whether such labels improve the accuracy and verifiability of generated reports.
Performance drops sharply at strict overlap thresholds (AP75 of 0.18), which the authors attribute to box extent being a convention experts do not reproduce rather than to the detector missing structures; the tricuspid valve has the lowest skill relative to the prior, and mitral regurgitation was assigned to tricuspid regurgitation in 7 of 13 frames with the reverse never occurring; the comparison models were open-weight versions not fine-tuned on the authors' annotations, so the comparison shows that localization does not arise from pretraining, not that EchoLens would outperform a vision-language model trained for the task.
