Skip to main content
Search

# External Validation

9 related articles

Subscribe to updates
  1. Oral resistome benchmark flips its winner: Hyena leads on held-out clusters, but MoR holds up on external databases

    Sequence models for oral resistome proteins lacked an evaluation that rules out homology-driven memorization. The study clustered 26,324 proteins into 1,437 clusters at 40% sequence identity, trained eight causal architectures under one protocol, and validated externally on CARD and NCBI AMRFinderPlus. Hyena reached the lowest held-out perplexity at 14.93, but only four of eight architectures beat the residue-composition baseline of 17.99 externally, where parameter-matched MoR led at 17.40 and the Transformer degraded by 27%.

  2. Parkinson's EEG microstates fail to travel: cohort origin is decodable at AUC 0.966 in the same feature space

    Expanding resting-state EEG A/B/C/D microstate sequences into static, directed-transition, flow and complexity features separated Parkinson's disease from controls within a development cohort at ROC-AUC 0.823, but locked testing on an independent OpenNeuro cohort fell to 0.513, near chance. In that same feature space, a classifier predicting dataset origin reached ROC-AUC 0.966, showing the two cohorts' feature distributions differ sharply. The study therefore evaluates within-cohort discrimination and cross-cohort transportability separately, and argues for explicit distribution-shift checks before microstate features are treated as reproducible electrophysiological markers.

  3. A wrist-worn single-lead ECG now flags 13 structural heart conditions, supervised by routine echo reports

    A Peking University-led team aligned routine single-lead ECG with physician-authored echocardiography reports to train AnyECG-Echo, a screening framework that resolves 13 fine-grained structural heart disease subtypes at once. Earlier AI-ECG models mostly required 12-lead input, were validated within a single health system, or needed per-condition manual labels. Across internal, temporal and geographically independent external cohorts totaling 25,222 patients, the framework reached high AUROC for high-impact subtypes such as reduced left ventricular systolic function, global heart enlargement and mitral stenosis, and retained usable discrimination in the external cohort.

  4. Artificial Intelligence in Ophthalmology: From Diagnostic Accuracy to Clinical Application

    This paper assesses why high-performing artificial intelligence systems for ocular image processing seldom translate into improved patient outcomes, locating the central problem in the disparity between pixel-level performance metrics and their clinical significance, naming data bias, domain shift, and label noise alongside the lack of prospective randomized deployment trials as primary obstacles, and outlining a path through stringent external validation, established decision criteria, ongoing surveillance in real clinical practice, transparent reporting standards, and deliberate human-factors engineering, with the goal of converting algorithmic accuracy into meaningful diagnostic precision for glaucoma, diabetic retinopathy, and macular conditions (specifically diabetic macular edema and

  5. Real-World External Validation of Artificial Intelligence-Based Full-Vessel Segmentation for Intracoronary Optical Coherence Tomography

    This retrospective, single-center external validation study enrolled 100 consecutive patients undergoing clinically indicated OCT and used the previously developed OCT-AID algorithm to perform automated pixelwise full-vessel labeling of 2560 analyzable frames, comparing it frame by frame against an expert manual reference standard; agreement was excellent for calcified plaque identification (κ=0.88) and quantification (intraclass correlation coefficients 0.79–0.93), close to interobserver variability, reasonable for lipid plaque identification and quantification (κ=0.68; lipid arc intraclass correlation coefficient 0.79; minimum fibrous cap thickness intraclass correlation coefficient 0.

  6. Research Square

    Cross-cohort evaluation of thyroid ultrasound deep learning across different outcome definitions: a duplicate-controlled benchmark

    This preprint trained five backbones under duplicate-controlled image-level partitioning (387 thyroid ultrasound images and 1,332 fine-needle aspiration cytology blocks from 385 public cases, labelled by postoperative diagnosis) and applied the frozen models to TN3K (n = 1,228, dataset-provided benign/malignant labels) and to a DDTI endpoint derived from radiologist TI-RADS categories (n = 637), finding ultrasound ensemble AUROC 0.825 and ResNet-50 0.888 on TN3K versus 0.477 and 0.437 against the TI-RADS-derived endpoint, with a cytology benchmark AUROC of 0.986 (0.970–0.998) against 0.733 for the ultrasound ensemble, while noting that cohort, acquisition and endpoint changed together so the contrast cannot be attributed to label definition alone.

  7. ENDOVISTA-ENT: Development and Preliminary Validation of an Integrated Artificial Intelligence System for Quality-Control Assistance and Lesion Recognition During Nasopharyngolaryngoscopy

    In this retrospective two-center study of 2 365 patients (1 562 from the First Affiliated Hospital of Sun Yat-sen University and 803 from Ruijin Hospital, Shanghai Jiao Tong University School of Medicine), the authors developed ENDOVISTA-ENT, an integrated AI system with Model 1 for inside/outside-body image determination, Model 2 for recognition of 11 standard anatomical sites, and Model 3 for lesion localization and benign/malignant classification, reporting internal/external accuracies of 99.44%/99.84% and 96.09%/94.73%, malignant-lesion AUCs of 0.986/0.968, early nasopharyngeal, laryngeal, and hypopharyngeal cancer AUCs of 0.878-0.932, an increase in six physicians' overall interpretation accuracy on 200 pathologically confirmed cases from 78.50% to 88.20% with AI assistance (χ²=40.

  8. Anesthesiology

    Development and External Validation of a Multimodal Artificial Intelligence Mortality Prediction Model of Critically Ill Patients Using Multicenter Data

    Using 203,434 ICU admissions from 2001 to 2022 across more than 200 hospitals in the MIMIC-III, MIMIC-IV, eICU, and HiRID databases, the study developed and externally validated a multimodal deep-learning model that predicts subsequent inpatient mortality from time-invariant variables, time-variant variables, clinical notes, and chest x-ray images available within the first 24 h of ICU admission; with structured data alone the model reached an AUROC of 0.92 (95% CI, 0.90 to 0.93), an AUPRC of 0.53 (95% CI, 0.49 to 0.57), and a Brier score of 0.19 (95% CI, 0.18 to 0.20), external validation across eight eICU institutions yielded AUROCs of 0.84 to 0.92, and in the subgroup with both notes and imaging, adding text and images raised the AUROC modestly from 0.87 (95% CI, 0.85 to 0.89) to 0.

  9. Prediction Models for In-Hospital Delirium Using Routinely Collected Electronic Health Record Data: Systematic Review

    This systematic review searched PubMed, MEDLINE, Embase, PsycINFO, and Web of Science from inception to November 11, 2025, and included 29 studies that developed, validated, or evaluated multivariable prediction models using routinely collected electronic health record or administrative data to predict acute mental status deterioration during adult hospital admissions, all operationalized as delirium; using CHARMS and TRIPOD/TRIPOD-AI for data extraction and PROBAST for risk of bias, it found that the evidence clustered into four overlapping prediction tasks (admission or early-stay risk stratification, perioperative or postoperative prediction, dynamic intensive care unit prediction, and external validation or workflow evaluation of existing tools), that most studies were retrospective co

You’re all caught up