GenNA makes natural-language annotations the generation condition: with 10-nucleotide prefixes it beats Evo2-7B and GENERator-v2 on sequence recovery
Synopsis
Nucleotide generative models have mostly been steered by sequence prefixes or structured labels, which express complex gene-function semantics poorly. GenNA places natural-language prompts such as species, gene name and functional annotation at the start of the sequence and is pretrained from scratch on a corpus spanning 2221 eukaryotic species and about 416 billion characters, using a unified cross-modal tokenizer to model sequence and annotation autoregressively together. On a sequence-recovery benchmark with only 10-nucleotide prefixes, full-annotation GenNA reached mean alignment identities of 0.465 for RNA and 0.455 for DNA, above Evo2-7B and GENERator-v2; its advantage narrows as prefixes lengthen.
Interpretation
Nucleotide sequence generation gains a control channel conditioned on natural-language functional annotations, so the model can propose candidate sequences from a gene-function description with almost no sequence context. Earlier generative models relied mainly on sequence prefixes, local context or predefined structured attributes with limited expressivity; natural language as the primary generation condition had been comparatively underexplored. On a sequence-recovery benchmark of 2000 human mRNAs and 1000 genomic DNA sequences, with 10-nucleotide prefixes, full-annotation GenNA reached mean alignment identities of 0.465 for RNA and 0.455 for DNA, above Evo2-7B at 0.440 and 0.439 and GENERator-v2 at 0.433 and 0.432.
The model places annotations, structural tags and nucleotides in one autoregressive sequence, so annotations keep constraining generation rather than sitting only at the input. The approach trains a 6000-token cross-modal byte-pair encoding tokenizer that maps frequent functional phrases, XML-style tag fragments and local nucleotide patterns into a shared discrete token space, then predicts the next token under a causal language-modeling objective. Across 100 randomly sampled sequences during coding-sequence generation, prompt tokens received a median attention ratio of 0.673 (interquartile range 0.599 to 0.744), versus 0.028 (interquartile range 0.017 to 0.045) for the upstream 5′ untranslated region.
The annotation gain is larger for RNA than for DNA, indicating that the annotation condition couples with different strength to different molecule types. On the same recovery benchmark, full annotation consistently outperformed gene identity only and no annotation. At the 10-nucleotide prefix, mean alignment identities were 0.465, 0.452 and 0.433 for RNA and 0.455, 0.446 and 0.437 for DNA; the annotation contribution rose from 0.013 to 0.061 across RNA prefix lengths and from 0.009 to 0.018 for DNA.
Perspective
The demonstrated scope is computational: the sequence-recovery benchmark uses 2000 human mRNAs and 1000 genomic DNA sequences, and generation experiments cover tRNAs, rRNAs and histone-coding sequences, judged by tRNAscan-SE, Rfam covariance models and physicochemical-property distributions. The authors propose extending the model to more complex downstream tasks through parameter-efficient fine-tuning such as low-rank adaptation.
All generated results currently lack systematic wet-lab validation, so structural compatibility and the plausibility of histone-coding sequences still require experimental confirmation. Generated distributions are more concentrated than natural reference distributions, a distribution sharpening the authors partly attribute to temperature scaling and top-p sampling truncating low-probability tails, and they call for more refined sampling calibration. Tokenization strategy and context length leave room for improvement, and how scaling laws affect the performance ceiling has not been systematically explored.
