Skip to main content
Back to timeline
arXivSource publication:

Calpha lets a general chat model decide when to fold: mean TM-score 0.808 on single chains, still behind the direct baseline on long chains

Synopsis

Calpha wires protein folding into a general chat model: Qwen3.5-9B reads a sequence with ESM2 residue features, emits a <structure> token when asked to fold, and conditions a frozen SimpleFold-3B head on its own residue hidden states to produce all-atom coordinates. Under a fixed folding request this configuration reaches mean TM-scores of 0.808 on FoldBench-334, 0.635 on CASP16 domains and 0.900 on a CAMEO-style PDB set, but on 300–600-residue chains the mean paired differences are −0.044, −0.160 and −0.025, all below the direct ESM2→SimpleFold-3B baseline.

AI-generated editorial illustration: Calpha: Toward Native Biomolecular Intelligence in a General-Purpose Language Model

Interpretation

A general language model can answer questions about a protein and decide on its own when to fold within the same chat turn, producing single-chain coordinates from its internal residue states without calling an external tool. Earlier biomolecular foundation models mapped a fixed input to a fixed output and took no natural-language instruction; in the three existing ways of connecting LLMs to folders, language reasoning stayed outside the predictor or produced text descriptions rather than coordinates. Under a fixed folding request, a frozen SimpleFold-3B head that receives protein information only through Qwen residue states reaches mean TM-scores of 0.808, 0.635 and 0.900 on 334 FoldBench-334 chains, 155 CASP16 domains and 200 CAMEO-2025 chains, with failed predictions scored zero.

Folding is triggered by the model itself in one forward pass: the language head is unchanged, and emitting the <structure> token invokes the structure head while other outputs continue as text. The trigger test covers five folding-request phrasings and one protein question that should be answered in text, with one response per eligible sequence and phrasing. Four request types elicited the trigger on every evaluated example, and the fifth elicited it on 95.5%–98.2% across the three sets; false triggers on the text question ranged from 0 to 1.4%.

The structure head receives residue hidden states from nine of Qwen's 32 layers (every fourth layer, width 4096), mixed and projected by an 8.4M-parameter conditioning front-end before entering SimpleFold-3B. Stage two freezes the language model and the pretrained 3B head and trains only this front-end, for 20,000 steps at crop 256 and then 10,000 steps at crop 512, on the same 84,658 PDB chains. With the 3B head frozen and only the 8.4M-parameter front-end trained, FoldBench-334 reached 0.801; unfreezing the head from that configuration made all six measured checkpoints lower, peaking at 0.792 at 5,000 steps and falling to 0.771 by 20,000 steps.

Perspective

The results cover single chains or isolated domains under one fixed folding-request template, scored by TM-score on alpha carbons only; complex prediction, structure as input, side-chain accuracy, all-atom steric quality, inference latency and peak inference memory were not evaluated. The 155 CASP16 domains correspond to 72 distinct sequences and are predicted in isolation, which differs from the official whole-target protocol.

At 100–299 residues the paired intervals include zero on all three sets, and the CASP16 interval extends to a loss of 0.030 TM; at 300–600 residues all three intervals fall below zero, but the two systems also differ in trainable weights, training duration and crop length, so the comparison measures their combined effect. The CAMEO-2025 no-homolog bucket holds only 16 entries, nine of which belong to three groups above 95% sequence identity. After protein training, accuracy on the MMLU-Pro subset fell from 77.4% to 70.3%. Whether paraphrased requests preserve coordinate accuracy, and whether an instruction or a line of reasoning can steer the predicted conformation, remain unmeasured.