Skip to main content
Search

# Joint-Embedding Predictive Architecture

16 related articles

Subscribe to updates
  1. arXiv

    Pruning 1.6% of neurons cuts name inference in fine-tuned LLMs by up to 7.4 points, with no retraining

    Large language models fine-tuned on proprietary records memorize names and other personal data, and existing privacy-neuron editing inspects only the leaking model, with no PII-free reference to separate identity neurons from useful context. JEPA-Prune trains a public model on surrogate and masked views of the same records, never on the original personal values, and uses a cross-model representation probe, integrated gradients and a scale-invariant specificity score to locate PII-responsive neurons, which are zeroed before deployment without retraining. Across three PII datasets and five models, pruning 1.6% of neurons lowers PII inference accuracy by up to 7.4 and extraction recall by up to 13.9, while unseen-text perplexity stays within 16% of the unpruned model for most LLMs.

  2. arXiv

    JEPA world models can memorize backgrounds and forget actions: motion-guided contrastive regularization lifts planning success from 52% to 78% under randomized backgrounds

    When a JEPA-style world model is regularized only through its embedding distribution, the encoder can treat the fixed background of each episode as its main content and suppress the state changes produced by robot-object interaction, a failure the authors call static collapse. The new method contrasts the predicted latent transition with transitions to other observations from the same episode, and uses optical-flow similarity as soft targets so candidates with similar motion receive similar supervision. Planning success rises from 52% to 78% over LeWM on OGBench-Cube with randomized backgrounds and from 14% to 54% on LIBERO-Object, while four other visual-control benchmarks stay comparable.

  3. arXiv

    Frozen world models skip per-step backprop: WARP-JEPA swaps planning for learned proposals, speeding long-horizon episodes 9.6–18.8×

    Hierarchical Joint-Embedding Predictive Architectures (H-JEPA) previously planned by running gradient descent on actions at test time, and the iterative solver consumed 96.6%–97.7% of episode runtime on long-horizon tasks. WARP-JEPA keeps the world models frozen and replaces online optimization with learned multi-candidate trajectory proposers, scoring candidates with the frozen predictor inside the generation loop. Across 30 frozen two-, three- and four-level models on four benchmarks, it speeds long-horizon episode inference by 9.6–18.8× (2.9–3.0× on Cube) and raises task success by up to 33.3 percentage points.

  4. arXiv

    Cross-modal remote sensing retrieval stops settling for one point estimate: CAPE-JEPA lifts F1@5 by 13.74 points on BEN-14K cross-modal directions

    Cross-modal remote sensing retrieval must match semantically identical scenes between synthetic aperture radar and multispectral imagery, yet earlier JEPA-style methods regress a single deterministic latent for each masked patch and assume every region is equally predictable. CAPE-JEPA instead outputs a per-patch Gaussian mean and variance, trains it by negative log-likelihood, and adds adversarial cross-modal distributional regularization plus in-training contrastive learning. On BEN-14K and fMoW it beats baselines including X-JEPA, SkySense and CROMA across all eight retrieval directions, gaining up to 13.74 points over X-JEPA on BEN-14K cross-modal directions and up to 15.59 on fMoW, with 407.30M parameters versus 432.72M.

  5. arXiv

    Same Login Screen, Opposite Labels: LGWM Checks the Transition in 17 ms, Not the Screen

    For GUI agents, the safety signal may lie not in the screen but in the transition that produced it: the same credential screen can be a legitimate result of tapping Sign in or a hijacked result of tapping View order. LGWM skips rendering the next screen and predicts its representation directly in the observation encoder's space, reducing verification to a vector comparison; on RSWT-Bench, where each credential screen appears pixel-identically under both labels, the training-free consistency score reaches 0.987 AUC at 17 ms per decision, on par with the strongest closed-source vision-language models and about ten AUC points above generative GUI world models.

  6. arXiv

    Fluid world models can read physics but not steer it: pre-registered tests split decodable, prominent and steerable

    On two-dimensional decaying Navier–Stokes flow, three pre-registered studies separate three properties often conflated as evidence that a latent has learned physics: decodability, prominence and steerability. A joint-embedding predictive architecture (JEPA) beats an untrained network of the same architecture on three probe targets yet fails the pre-registered criteria for organizing its latent around fluid quantities; autoencoders that render the field make enstrophy prominent in their slowest unsupervised directions, and adding VICReg lowers that prominence in every comparison while raising linear decodability; a routed design yields a label-free loudness dial in all 8 trained models and a spectral-shape dial in only 3 of 5 fresh seeds, below the pre-registered bar of 4.

  7. Single-cell perturbation prediction gets a unified base: DCAF ties data pairing, scoring and modeling into one framework

    Single-cell perturbation experiments are costly and low-throughput, yet computational prediction has been hampered by inconsistent evaluation and by models that do not reliably beat simple baselines. DCAF combines data pairing, an evaluation criterion and a prediction model in one framework: entropy-regularized optimal transport recovers a pre-perturbation state for each perturbed cell, log fold changes of non-silenced genes drive scoring, and a JEPA structure encodes control and perturbed cells jointly. Across cross-gene, cross-cell-type and cross-region settings, DCAF-model beats GEARS, scGPT, linear and mean baselines on R² and MSE of log fold change for non-silenced genes.

  8. arXiv

    World models stop training a controller separately: LeWAM folds action prediction into one pass, lifting contact-rich success from 28.6% to 89.7%

    LeWAM trains visual representation, action-conditioned latent dynamics, and a stochastic policy in a single stage, raising planning success from 28.6% to 89.7% on contact-rich tasks and from 10.9% to 46.2% on long-horizon tasks while planning 32.3 times faster.

  9. Brain MRI foundation model drops the single-sequence habit: 1.55M scans, up to 15% few-shot gain on gliosis and MS

    A New York University team released Neuro-JEPA, a multimodal foundation model that encodes T1w, T2w and FLAIR brain MRI contrasts in one architecture, pretrained on 1,551,862 scans from 428,647 studies. It outperformed BrainIAC, VoCo and NeuroVFM overall across 47 downstream tasks in three health systems and 12 public datasets while using less than one-third of NeuroVFM's pretraining data, and reached up to 15% higher few-shot AUROC on label-scarce tasks such as gliosis and multiple sclerosis, with roughly 15% on the NACC Alzheimer's disease task.

  10. arXiv

    CT pretraining favors what context can predict: pathology signal fades in hidden-target prediction while scan-level calls barely move

    A ViT-B/3D encoder trained on 113,611 thoraco-abdomino-pelvic CT volumes, TAP-JEPA, matches state-of-the-art CT encoders on thoracic and abdominal classification benchmarks; replaying its exact masking, teacher and predictor shows hidden-target prediction lowers mean Skill from 0.388 to 0.271 for pathology but only from 0.921 to 0.873 for anatomy. Pathology moves 34.7% toward matched controls along the teacher direction while controls move 13.6% toward pathology, with larger shifts farther from visible context, and deleting all final-layer pathology tokens changes the decision margin by at most 0.45% in four of seven scan-level probes.