Skip to main content
Search

# Contrastive Learning

34 related articles

Subscribe to updates
  1. bioRxiv

    No spike sorting required: C3PO learns interpretable population states straight from raw spike events, from milliseconds to hours

    C3PO models neural spikes directly as a marked point process, skipping spike sorting and binning, and uses contrastive prediction over event history to infer a low-dimensional context state. On synthetic data the learned context forms a ring organized by latent phase and recovers tuning curves for 48 phase-tuned neurons; on several published recordings it recovers known variables such as position, head direction and theta phase, and maps population features directly onto single neurons.

  2. arXiv

    Diffractive optical networks stop prescribing output patterns: weak supervision lets handwritten scripts self-organize, cutting training steps by about 82%

    Diffractive optical neural networks usually assign each class to a fixed detector region and train the light field to reproduce that preset output. A team at the Singapore University of Technology and Design and the National University of Singapore instead used script-level coarse labels and a modified contrastive objective that constrains only relations among samples, letting optical representations of handwritten Devanagari, Arabic and English characters cluster into three groups on their own. At a shared training endpoint the three-layer network reached 93.4% mean test accuracy against 84.7% for supervised optical networks, and needed about 18% of their training steps to reach a stable 84.5% test accuracy.

  3. bioRxiv

    Spatial transcriptomics stops swapping only cell-level states: SpaAxFormer keeps each gene's state and beats Impeller on DLPFC imputation L1

    Spatial transcriptomics models usually exchange representations only between cells or spots, with gene information arriving through the attached expression profile. SpaAxFormer keeps every cell–gene entry as its own state, first letting genes within a cell reference each other, then exchanging the same gene's state across spatial neighbors, with a separate cell-level summary. On six DLPFC slices its mean L1 imputation distance was lower than Impeller's, for example 0.239 versus 0.254 on slice 151510; on Stereo-seq and Slide-seq V2, Impeller had lower mean L1 and RMSE.

  4. arXiv

    CroCoMAE moves SAR-optical alignment from whole images to patches, with the largest gains on semantic segmentation

    SAR and optical remote sensing images are often treated as largely redundant views, and earlier cross-modal self-supervised methods aligned them only at the whole-image level, leaving fine-grained complementary structure unused. CroCoMAE combines image-level contrastive learning with patch-level masked reconstruction inside a ViT, splitting each patch embedding into a shared subspace aligned across sensors and an unconstrained unshared subspace. On the EuroSAT-SAR scene-classification and DFC2020 semantic-segmentation benchmarks it beats uni-modal self-supervised baselines and retrained cross-modal frameworks in most configurations, with the largest gains on segmentation.

  5. bioRxiv

    BacLLM folds UniProt annotation knowledge into a protein language model, so sequence-only input can call resistance genes and rank phage candidates

    BacLLM injects semantic representations of UniProt annotations, encoded by a large language model, into an ESM-2 sequence encoder through cross-modal contrastive learning, so inference reads amino acid sequences alone. Across six resistance-prediction benchmarks, BacLLM (4B) beat Ankh, ESM-2, ProteinBERT and ProTrek in mean F1 under both logistic regression and an RBF-kernel support vector machine; on the SHV-1 β-lactamase P0AD64 it recovered six of eight experimentally supported resistance-attenuating sites among the top 65 ranked positions, while ESM-2 recovered none. The text branch is removed after training, so these gains come from sequence input, not annotation input.

  6. arXiv

    PACE-ECG pairs a patient's own ECGs across time: 41 times fewer parameters, higher external heart-failure AUROC

    ECG foundation models are usually compared on in-domain interpretation tasks, leaving their transfer to longitudinal prognosis across healthcare systems uncertain. PACE-ECG treats two recordings from the same patient within 60 days as a contrastive positive pair and adds waveform- and report-derived multitask supervision, with an encoder of 2.21 million parameters. On MIMIC-IV and two external HEEDB cohorts, its frozen representations predict post-myocardial-infarction all-cause mortality and incident heart failure with higher AUROC than DeepECG-SSL, with the largest gains in external heart failure.

  7. bioRxiv

    π-SPECFormer learns from 291 million mass spectra and flags six mutation peptides missing from databases in lung adenocarcinoma

    Conventional database searching assigns high-confidence peptides to only about 20% to 31% of acquired spectra, leaving the rest unread. π-SPECFormer was pre-trained on 291 million tandem mass spectra in two stages: self-supervised learning of fragmentation patterns, then refinement with peptide–spectrum match groupings. It beat task-specific baselines in de novo sequencing, spectrum clustering and modification detection, and in 103 lung adenocarcinoma samples it produced nine high-confidence mutation peptides, six of them absent from the public databases examined.

  8. bioRxiv

    DNA-LLM2Vec turns causal DNA decoders into bidirectional encoders, with frozen-probe gains that shift by task and backbone

    DNA-LLM2Vec converts pretrained causal DNA decoders into frozen bidirectional encoders in stages: it opens bidirectional context, repairs the resulting mismatch with masked next-token prediction, then adds contrastive views encoding strand and positional invariance. Across two backbones, DNAGPT and HyenaDNA, and seven biological task families, the benefit is not uniform but depends on task structure and backbone; crop contrastive learning helps most for extended-context families such as enhancer–promoter interaction and Virus-COVID, reaching frozen Avg-EPI MCC of 27.5% and 26.7%.

  9. medRxiv

    Alzheimer's cohort stops sorting patients by diagnosis alone: TRACE lifts 24-month phenotype purity 3.1 points over the strongest baseline

    TRACE maps longitudinal visit records to interpretable progression phenotypes and lets the number of phenotypes exceed the number of diagnostic classes. In the label-matched setting on the ADNI cohort it achieves the highest purity, NMI, ARI, silhouette index, accuracy and macro-F1 at the current, 12-month and 24-month horizons; at 24 months it exceeds the strongest baseline by 3.1 and 5.8 percentage points in purity and silhouette index. With the phenotype count freed, it splits a new phenotype out of the cognitively-normal- and MCI-dominant groups, one whose cognitive-scale trajectories sit in between while its hippocampal-volume trajectory coincides with the MCI group.

  10. Medical CLIP stops reading only high-level semantics: splitting texture and attribute branches lifts multi-abnormality accuracy by 10.2%

    Medical CLIP has mainly aligned high-level semantic features, leaving texture, entity order and attributes poorly captured and limiting fine-grained tasks. Deco-CLIP splits both image and text into two branches: one ViT pretrained by CT-CLIP for semantics, one MAE-pretrained ViT for texture, with a text encoder that forks in its last three layers, shares keys and values, and uses distinct queries, plus threshold-adaptive sample mining and a regularized attention mechanism. On CT-RATE zero-shot tasks, multi-abnormality detection accuracy improves by 10.2% and report-to-volume retrieval recall by 16.2%.