Skip to main content
Back to timeline
arXivSource publication:

Molecule-language models get a structure lesson: SAGA trains only, lifting retrieval R@1 by up to 7.82 points

Synopsis

Molecule-language models learn from paired structure-text contrastive objectives, yet they treat a structurally related nonmatch and an unrelated molecule as the same negative class. SAGA uses Centered Kernel Alignment during training to align pairwise relationships among neural molecular embeddings with fingerprint Tanimoto similarity, adding no parameters and requiring no fingerprints at inference. On MolCA and Atomas, every reported molecule-text retrieval metric improves, with Atomas-L text-to-molecule R@1 rising from 43.15 to 50.97, and non-exact retrieval shifts toward closer structural analogues.

Source-provided article image: Classical structural priors improve neural molecular representations in molecule language models
arXiv

Interpretation

The embedding space of molecule-language models begins to organize along the continuous gradient of chemical similarity rather than only by paired versus unpaired. Earlier binary contrastive objectives placed structurally related nonmatches in the same negative class as unrelated molecules, leaving the similarity continuum between exact identity and unrelated compounds unused. On ChEBI-20, all three backbones improve on every reported retrieval metric in both directions: MolCA text-to-molecule R@1 rises from 31.48 to 35.82, Atomas-B from 33.56 to 40.44, and Atomas-L from 43.15 to 50.97.

The approach adds a relational alignment term to the training loss so that pairwise similarity patterns among neural embeddings match those among fingerprints. Alignment is measured with Centered Kernel Alignment (CKA), which is insensitive to dimensionality, orthogonal transformations and isotropic scaling, so fingerprints need not correspond coordinate by coordinate with embeddings and no trainable parameters are added. MolCA computes CKA within mini-batches of 64 molecules; Atomas keeps a fingerprint queue parallel to its feature queue and computes CKA over 128 samples while the training batch size stays 16.

The retrieval gain is not only better paired ranking; non-exact hits also move toward structurally closer analogues. After removing each query's exact target, MolCA's mean Tanimoto similarity of the top-3 non-exact neighbours rises from 0.235 to 0.311 in text-to-molecule and from 0.211 to 0.272 in the reverse direction. MolCA retrieves closer top-3 neighbours for 76.3% of validation queries (one-sided Wilcoxon signed-rank test) and 73.8% in the reverse direction, with the same trend for Atomas-B and Atomas-L in both directions.

Perspective

The conclusions rest on two-dimensional fingerprint priors and are evaluated on ChEBI-20 retrieval, MoleculeNet scaffold-split property prediction and generation benchmarks across the MolCA and Atomas model families. For property prediction the molecular encoder is frozen and only a two-layer head is trained, so the results reflect information carried by the representation itself.

The structural prior is not beneficial for every endpoint: Atomas-B drops on HIV and Atomas-L on MUV, and 5 of the 20 ADMET datasets show positive but non-significant correlations. Generation and description results come from single-run ablations, and the authors note it is not established that the same prior is optimal for tasks dominated by three-dimensional conformation or target-specific binding.