GenVinci turns routine H&E slides into molecular maps: serial human skin sections yield 3D virtual transcriptomes
Synopsis
Spatial transcriptomics preserves molecular and positional information but is costly, heterogeneous across platforms and mostly limited to two-dimensional sections, while routine H&E slides are widely available yet carry no direct molecular readout. GenVinci is pretrained on ImageOmicsNet-300M, a corpus of roughly 300 million cellular and spatial profiles with H&E sections, learns cells as components of tissue scenes, and aligns pathology-model image features to predict spatial gene expression from H&E. On held-out datasets from five cancer types it reaches higher average gene-expression correlation than baselines retrained per dataset, without dataset-specific retraining, and is used to reconstruct 3D virtual transcriptomes from serial human skin sections.
Interpretation
A routine H&E slide can now be converted into spatially located gene-expression reads and assembled into a three-dimensional tissue scene, rather than remaining a morphological description of a single 2D section. Earlier histology-to-transcriptome predictors were typically trained on relatively small datasets and struggled to generalize beyond the tissues or disease conditions on which they were developed, while spatial transcriptomics itself is costly, heterogeneous across platforms and largely restricted to 2D sections. On held-out spatial transcriptomic datasets spanning five cancer types and two experimental platforms, GenVinci-2DST was trained once and evaluated zero-shot, reaching average Pearson correlations of 0.45, 0.42 and 0.33 for the top 50 highly variable genes, top 50 highly expressed genes and all genes, versus 0.35, 0.33 and 0.22 for HEST, the strongest baseline retrained per dataset.
The model learns cells as components of tissue scenes instead of treating each cell as an independent transcriptional entity. Existing single-cell and spatial foundation models were mostly trained on dissociated single-cell data without explicit spatial or morphological grounding, which limited the biological contexts to which they transfer. The pretraining corpus ImageOmicsNet-300M holds 300.2 million transcriptomic profiles and 4,623 H&E sections across two species, 82 tissue types and 55 transcriptomic technologies; the encoder takes modality, assay, organism and tissue context directly, and training combines masked gene prediction, masked expression prediction and gene expression reconstruction.
H&E-predicted expression is used to reconstruct 3D virtual transcriptomes of human skin and to localize the cell type where disease genetic risk is enriched. A single 2D section fragments continuous multicellular niches into many disconnected patches and under-resolves the interfaces between compartments. Across 605 quality-controlled serial sections from nine female donors aged 20 to 71, the largest 3D connected component of the epidermis-associated niche held 86.9% (23-year-old donor) and 75.1% (71-year-old donor) of its cells, whereas the same cells split into 26 to 67 disconnected components within each section; against adjacent STARmap sections, cell-type composition agreed at a mean Pearson correlation of 0.79.
Perspective
The demonstrated scope is zero-shot gene-expression prediction on held-out spatial transcriptomic data from five cancer types, validation against same-section STARmap in mouse tongue and kidney, and 3D virtual transcriptome reconstruction from serial abdominal skin sections of nine female donors compared with adjacent STARmap sections; the authors note that these virtual transcriptomes are predictions with modest accuracy at near-cellular scale, lower accuracy for sparse immune cells and lowly expressed genes, and sensitivity to slide-level staining.
The authors' stated qualifications include: ImageOmicsNet-300M reflects biases in available data across tissues, species, diseases and technologies; GenVinci does not encode spatial coordinates during pretraining, so 3D reconstructions arise from registration and section-wise prediction; without ablations under matched training data, the contributions of transcriptomic supervision, the spatial refinement module and biological-context embeddings are not isolated, and gains over baselines may partly reflect pathology foundation-model features and the larger training corpus; senescence mapping relies on weak supervision anchored mainly on p21 detection and an operational threshold, may capture quiescence, differentiation or stress programs, and requires orthogonal validation at the protein or SA-β-gal level; skin cell types were assigned by reference-based label transfer and depend on the reference; aging analyses involved one animal per age group per organ and nine female donors from one sun-protected site, so age trends are exploratory; genetic mapping relied on relative enrichment against two negative-control traits because absolute significance was not calibrated on dense inputs, was validated on four adjacent rather than identical sections from two donors, and did not recover the immune axis from H&E.
