Skip to main content
Back to timeline
bioRxivSource publication:

A 35M-parameter protein language model, DynamicsPLM, adds molecular-dynamics supervision and matches sequence models 86 times larger on ProteinGym

Synopsis

DynamicsPLM attaches residue-level and residue-pair biophysical quantities from molecular dynamics simulations as extra prediction targets to a 35M-parameter ESM-2 encoder, so the model returns both an evolutionary plausibility score and a structural dynamics disruption score from sequence alone. Sequence language models had relied on scaling parameters to absorb structural constraints implicitly, while dynamics-only models discarded evolutionary statistics. On the 201 proteins of ProteinGym, the model reaches a mean Spearman correlation of 0.398, ranks first among models with at most 100M parameters, and matches or exceeds sequence models up to 86 times its size.

Source-provided article image: DynamicsPLM: A Dynamics-Enriched Small Protein Language Model for Protein Variant Effect Prediction
Figure 1 · bioRxiv · Page 5

Interpretation

A protein language model with only 35 million parameters, reading sequence alone and never a structure, places among the top of all 47 sequence models on ProteinGym for ranking mutation effects and first within the small-model group. Sequence language models had mainly absorbed structural constraints implicitly by scaling to hundreds of millions or billions of parameters, and at inference they saw nothing of the folded protein's physical dynamics. On 201 proteins and 2,413,913 substitution variants in ProteinGym, DynamicsPLM reaches a mean Spearman correlation of 0.398 and a median of 0.414, ranking 19th of 47 overall and first among models with at most 100M parameters, ahead of larger models including Tranception L, RITA XL, Progen3 and xTrimoPGLM-1B-CLM.

On folding stability, the small model's ranking accuracy approaches the largest sequence model, which has roughly 86 times its parameters. Stability assays report biophysical properties tied to folding, a task where sequence signal alone is comparatively weak. On the 66 stability assays in ProteinGym, DynamicsPLM reaches a mean correlation of 0.528 and a median of 0.507, second of 47 models behind only the 3B-parameter VespaG, and first among models with at most 100M parameters, 0.09 above the ESM2-35M backbone and 0.05 above ESMDance.

The model works by running two output heads that predict molecular dynamics quantities alongside the original masked-language-modeling objective, then subtracting the two scores into one mutation-effect score at inference. Dynamics-only models learn physical quantities but lose evolutionary statistics, while multimodal structure models require experimentally or computationally derived 3D structures at inference. Training uses molecular dynamics data for 64,403 proteins, 61,959 after deduplication, clustered at 50% sequence identity into 58,832 training and 3,127 held-out evaluation proteins; each residue carries 50 residue-level properties and each residue pair 13 interaction features, with the final configuration weighting language-modeling loss at 0.7 and dynamics loss at 0.3.

Perspective

Evaluation covers 201 proteins of at most 1,022 residues and 2,413,913 substitution variants in ProteinGym, plus ΔΔG ranking over 412 proteins and 527,794 substitution variants in the Tsuboyama dataset; insertions and deletions are excluded, and the model takes only sequence as input.

On viral proteins ESMDance reaches a mean correlation of 0.311 against 0.268 for DynamicsPLM, because rapid viral sequence divergence weakens the language-modeling component, and fusing it with the dynamics score falls below the dynamics score alone; on the Binding category, with only 12 assays, DynamicsPLM shows no statistically significant advantage over either baseline. The authors note that the fixed-weight combination always folds in the evolutionary term and propose learning the mixing ratio from sequence features such as homolog depth as the next step.