Skip to main content
Back to timeline
bioRxivSource publication:

Single-cell embeddings drop gene attention: scUNVEIL, pretrained on about 92 million cells, reaches 80.6% macro F1 on fine-grained PBMC subtypes with a linear readout

Synopsis

scUNVEIL reframes single-cell RNA-seq pretraining as measurement enrichment: estimating what the same measurement would reveal at greater UMI depth, using a residual multilayer perceptron on count vectors. Pretrained without labels on about 92 million human cells, it reached 80.6% macro F1 on linear evaluation of 58 PBMC subtypes in the Hao et al. 2021 dataset, above 70.1% for the strongest evaluated foundation-model comparator, scFoundation, while generating embeddings for all 161,764 cells in 1 minute 41 seconds on one CPU. That dataset was excluded from pretraining, so the result is zero-shot transfer.

Source-provided article image: scUNVEIL: Learning scRNA-seq Cell Embeddings Through Measurement Enrichment
Figure 1 · bioRxiv · Page 4

Interpretation

Single-cell RNA-seq pretraining does not need attention over thousands of gene tokens; a residual multilayer perceptron reading a count vector directly yields cell embeddings that a linear classifier can use. The dominant prior approach is transformer-based foundation models such as scGPT, scFoundation, Geneformer and UCE, which are expensive to train and deploy, while independent evaluations found them matched or beaten by simpler baselines on benchmarks such as cell-type annotation. On the Hao et al. 2021 PBMC reference dataset with 58 subtypes, 161,764 cells, 8 donors and 100 donor splits, the 2048-dimensional scUNVEIL embedding reached 80.6±2.1% macro F1, 79.7±2.8% balanced accuracy and 91.6±0.6% overall accuracy, the highest of the evaluated methods on all three metrics.

The model treats each cell as one fixed-length count vector, and a single forward pass returns both a reusable cell embedding and a measurement-enriched gene-expression estimate. During training, each cell's observed UMIs are randomly split into retained inputs and held-out targets, and the empirical gene-frequency distribution of the held-out molecules serves as a soft target under categorical cross-entropy, equivalent to predicting the gene identity of one randomly held-out UMI. Pretraining used about 92 million Homo sapiens cells and 60,000 gene features from the CELLxGENE Census, with no labels, donor metadata or target-depth conditioning; the model has 16 residual blocks and an embedding dimension of 2048.

The measurement-enrichment objective itself accounts for much of the gain, not nonlinear capacity alone. A linear-embeddings variant removes all residual blocks, keeping only the input projection, a final layer normalization and the output projection. That linear variant still reached 76.8±1.8% macro F1, above every non-scUNVEIL method, while a 2048-component PCA fitted on the same 92 million-cell corpus reached 69.1±2.5% macro F1.

Perspective

The demonstrated scope is label-free pretraining on about 92 million human cells, zero-shot linear annotation evaluation on the Hao et al. 2021 PBMC dataset excluded from that corpus, and four supplementary tasks with externally measured labels; the authors propose prospective shallow-sequencing studies to establish achievable sequencing savings.

The authors state that measurement enrichment is an inference procedure, not a new physical measurement, and cannot recover cell-specific features that are neither observed nor predictable from the pretraining population; it does not remove technical distortions such as batch effects, and predicted expression values are intended for hypothesis generation rather than as confirmatory evidence of differential expression. PCA narrows the advantage on morphology and lineage-recovery tasks, and at 1% retention mean R² is only 0.039 for cell area and 0.038 for circularity; the supplementary tasks were chosen after exploratory screening and do not estimate performance on unselected phenotypes.