Structure confidence picks the wrong proteins: 97% of high-confidence designs expressed, but only 16.7% bound terbium
Synopsis
In designing rare-earth-binding variants of the lanthanide-binding protein lanmodulin, candidates selected on structural confidence expressed at 97% yet only 16.7% bound terbium, against 83.3% for candidates from a function model trained on measured metal-binding data. No structural-confidence metric consistently ranked binders above non-binders within the three design panels, while the function model's prediction gave an ROC AUC of 0.79 to 0.92 in every panel. The comparison holds one protein family fixed, and the function model was trained largely on lanmodulin variants.
Interpretation
Using structural confidence as the selection filter produced the design set with the lowest binding rate among the rare-earth-binding proteins. Structural-confidence scores had been used as design filters that rank candidates without requiring measurements, which is why scores such as pLDDT appear in design pipelines. Of 93 structure-confidence-selected designs, 90 expressed and 15 bound terbium, 16.7% (95% CI 10.4 to 25.7); of 100 function-selected designs, 96 expressed and 80 bound, 83.3% (74.6 to 89.5), an odds ratio of 25.0 between the two panels.
The function model predicts binding strength directly from sequence, metal ion, buffer and assay conditions, and generates new sequences from that prediction. This route skips predicting a three-dimensional structure and then deriving function from a hand-built structural rationale, using measured dissociation constants as training labels instead. The model was trained on roughly 2,500 proteins measured in-house, heavily weighted toward lanmodulin variants; across 297 designs screened against terbium and dysprosium, the function model's prediction reached an ROC AUC of 0.79 to 0.92 for separating binders from non-binders within each panel.
On the campaign's design objective, terbium/dysprosium selectivity, the function-guided designs sat above the natural lanmodulin ortholog set. The ortholog set had a median selectivity of 1.26, with wild-type lanmodulin near 1.4. Function-selected designs had a median selectivity of 1.70 and outranked a randomly drawn ortholog in 83.9% of pairs across 149 orthologs; co-generation-selected designs had a median of 1.38 and structure-confidence-selected designs 1.42, the latter not distinguishable from the ortholog set.
Perspective
The claim is bounded to the lanmodulin protein family: the function model was trained largely on lanmodulin variants and evaluated on lanmodulin variants, so what is shown is that in-domain measured-function data outranks in-domain zero-shot structural confidence. The three panels also differ in generative method, so part of the binding-rate gap reflects how far each method travels from a scaffold already known to work. The measurement data and analysis code are proprietary to Aether Biomachines and not publicly available.
The binding-rate comparisons between panels were specified after the measurements were collected and are reported without multiplicity correction, unlike the prespecified selectivity analysis. The structure-confidence panel contributes only 15 binders, so its within-panel rank correlations are underpowered. The terbium/dysprosium selectivity ratio requires both metal measurements to resolve, so the selectivity analysis has fewer proteins than the binder counts.
