Gene regulation beyond the single cell: Multicellular Transformer takes a whole tissue sample as input and predicts receptor-ligand pairs zero-shot
Synopsis
Single-cell foundation models usually take one cell at a time, so their gene representations lean toward cell-intrinsic regulation. Multicellular Transformer instead takes a set of cells from the same tissue sample as input, so each gene's representation carries its expression across that sample together with the sample's cellular composition. On known receptor-ligand pairs from CellChatDB it reaches a zero-shot AUROC of 0.71, against 0.65 for the second-best method, scGPT, and the gain holds across cell-cell contact, secreted signaling and ECM-receptor interaction types.
Interpretation
The unit of gene representation moves from a single cell to a set of cells drawn from the same tissue sample, so each gene's vector reflects both its expression across all cells in that sample and the sample's cell-type composition. GeneFormer, scGPT and scPRINT take a single cell as input and emphasize cell-intrinsic regulation; SCENIC and CellOracle likewise focus on intracellular regulatory networks, while cross-cell methods such as DIALOGUE rely on correlation-based modules. On known receptor-ligand pairs from CellChatDB, Multicellular Transformer reaches a zero-shot AUROC of 0.71 against 0.65 for the second-best method, scGPT, and the gain holds across cell-cell contact, secreted signaling and ECM-receptor interaction types.
Each gene is split into two summed parts: an expression latent vector that varies with expression, and a sample-independent gene function vector learned from scratch. The expression latent vector comes from a shared two-layer feed-forward network whose shared weights limit overfitting; during pretraining its input units are fixed to specific cell types, for example units 1–20 to CD8 T cells and units 21–32 to macrophages. Pretraining used more than 4,200 samples from over 3,500 donors and more than 14 million cells across more than 30 tissue types; the model has a 12-layer encoder and nearly 60,000 genes, and cells are iteratively sub-sampled per sample during training to limit overfitting.
The pretrained gene vectors feed several downstream tasks without task-specific retraining. Traditional cell-cell communication methods rely on predefined databases to score known ligand-receptor pairs across cell populations; here only cosine similarity between pretrained gene vectors is used to test whether any arbitrary gene pair is likely to take part in a cross-cell process. In systemic lupus erythematosus and COVID-19 data, interferon-stimulated genes such as IFI44 and OAS1 cluster with interferon-pathway genes only in disease samples; in lung cancer, zero-shot in silico knockouts give a Pearson correlation of 0.33 (p-value below 1×10⁻¹⁰) with observed effects in the DepMap SW 1573 cell line.
Perspective
The demonstrated scope is pretraining on more than 4,200 samples from over 3,500 donors and more than 14 million cells, with downstream analyses on public lung cancer, glioblastoma, systemic lupus erythematosus, COVID-19 and multi-tissue immune atlas data; spatial co-localization was checked on Xenium and Visium datasets excluded from training. The intended reader is a researcher who wants to infer cross-cell regulation from single-cell transcriptomes.
Newly proposed receptor-ligand interactions still require experimental validation; the model currently indicates whether genes may interact rather than how, and pinpointing which two cell subsets interact relies on post-hoc techniques such as Integrated Gradients; input units are fixed to predefined cell types, which constrains zero-shot generalization to entirely novel, unannotated cell types and requires sampling strategies to handle shifts in cell-type proportions; biases in public dataset coverage, protocols and annotations may affect generalization to underrepresented contexts.
