Jump to content
[dismiss]
Molecupedia is a parody encyclopedia written by Molecular Reality Corporation. It is not Wikipedia and is not affiliated with the Wikimedia Foundation. About

Joint embedding predictive architecture

Page protected
From Molecupedia, the encyclopedia of very small things
Self-supervised neural network design that learns by predicting the embedding of one part of an input from another
This article is about the machine-learning architecture. For the company's essay applying it to ionic current, see The Nanopore World Model.

A joint embedding predictive architecture (JEPA) is a design for self-supervised learning in which a network learns from unlabeled data by predicting the embedding of one part of an input from the embedding of another part. The loss is computed between representations, not between pixels or samples, so details that cannot be predicted are free to drop out of the representation. Yann LeCun proposed the idea in 2022 as a component of a world model for autonomous agents.[1] Researchers at Meta AI built I-JEPA for images in 2023, V-JEPA for video in 2024 and V-JEPA 2 in 2025, and the recipe has been tried on electrocardiograms and other signals.

A JEPA is easy to break. If the encoder and the predictor return the same vector for every input, the prediction error is zero and nothing has been learned. Preventing this collapse is the main engineering problem, and a collapsed model reports a loss that looks excellent.

Origins

Joint embedding is older than the name. Siamese networks, in which two copies of one network with shared weights embed two inputs so that the distance between the embeddings says whether the inputs match, were used in 1993 to verify signatures.[2] LeCun's position paper of 2022 had a different aim. It argued that an agent needs a model of the world that can predict the consequences of actions, and that such a model should predict abstract representations, so that the details of the world that cannot be predicted can be left out. It proposed a hierarchy of such predictors under the name H-JEPA, and it reported no experiments; they began with I-JEPA the following year.[1]

Architecture

A diagram of a joint embedding predictive architecture, with context and target encoders, a predictor and a loss between embeddings, above a masked autoencoder with a decoder and a loss in signal space
A joint embedding predictive architecture compared with a masked autoencoder, drawn for a signal in time. The JEPA predicts the embedding of the hidden stretch and compares it with the target encoder's embedding; the autoencoder reconstructs the hidden samples. The drawing is schematic.

A JEPA has three trained parts. The context encoder turns the visible part of an input into embeddings, and the target encoder does the same for the hidden part to give the answer. Given the context embeddings and the position of the hidden part, the predictor outputs a guess at the target embeddings. Training minimizes the distance between guess and answer, usually as a squared error, and no decoder ever maps embeddings back to the input:

ℒ=\lVert gϕ(fθ(xcontext), p)−sg(fθ¯(xtarget))\rVert2,

where f is the encoder, g the predictor, p the position of the target and sg the stop-gradient, which blocks learning signals from flowing into the target. In I-JEPA the target encoder's weights are an exponential moving average of the context encoder's, so the answer changes slowly.

I-JEPA predicts the representations of several target blocks of an image from a single context block. Its authors found the masking strategy to be the main design choice: the target blocks must be large enough to be semantic, and the context block informative enough, which means spread across the image. With these choices a ViT-Huge/14 trained on ImageNet with 16 A100 GPUs in under 72 hours, using no hand-crafted data augmentations.[3] V-JEPA applied the method to video as the sole objective, with no pretrained image encoder, text, negative examples or reconstruction.[4]

Two neighboring families mark the boundaries. A masked autoencoder hides patches of an image and reconstructs the missing pixels, so the model must account for every detail, predictable or not.[5] A contrastive method such as SimCLR pulls together the embeddings of two views of one image and pushes apart those of different images.[6] What a JEPA takes from them is the embedding space of the second and the masking of the first; what it predicts is the embedding of the hidden part.

Collapse

Joint-embedding methods share a trivial solution in which the encoder outputs a constant, and each family prevents it in its own way. Contrastive methods use negative pairs. BYOL trains an online network to predict a slowly updated copy of itself and needs no negatives.[7] VICReg adds an explicit penalty on the variance of the embeddings along each dimension, and a term that decorrelates the dimensions.[8] Total collapse is one failure among several. In dimensional collapse the embeddings span a smaller space than the one available, and it occurs in contrastive learning too.[9]

The loss does not reveal collapse. The authors of a 2022 study of joint-embedding methods point out that, with no input reconstruction to look at, there are no "visual cues of unsuccessful training", and that the loss values are "non informative". Their remedy is the effective rank of the embeddings, which can be computed without labels and tracks downstream performance.[10]

Family

Models in the JEPA family
Year Model Data Result
2023 I-JEPA ImageNet images A ViT-Huge/14 trained in under 72 hours on 16 A100 GPUs[3]
2024 V-JEPA 2 million videos With a frozen backbone, 81.9% on Kinetics-400, 72.2% on Something-Something v2 and 77.9% on ImageNet-1K[4]
2024 ECG-JEPA Ten public ECG databases, over 1 million records An AUC of 0.945 on the PTB-XL all-statements task, ahead of invariance-based and generative pretraining[11]
2025 TS-JEPA Standard time-series datasets Matched or passed baselines on classification and forecasting[12]
2025 V-JEPA 2 Over 1 million hours of video 77.3% on Something-Something v2; a version post-trained on under 62 hours of unlabeled robot video planned pick-and-place tasks zero-shot on robot arms in two laboratories[13]

LeCun's company, AMI Labs, is built around world models of the kind the 2022 paper described.[14]

For signals in time

The argument for using a JEPA on a noisy recording is that noise defeats reconstruction. The TS-JEPA authors begin from it: methods that reproduce masked parts of the input, they write, "can be vulnerable to the presence of noise or confounding variables", and they move the task to latent space.[12] White noise cannot be predicted, so a model that predicts samples carries an error floor equal to the noise variance whatever its size, and for Gaussian noise of 12.8 pA rms, the thermal noise of a 1 MΩ resistor in a band of 10 kHz, that floor is 164 pA2 per sample (see Johnson-Nyquist noise). A JEPA is not asked to reach it, because its embedding may leave the noise out.

At Molecular Reality

Main article: The Nanopore World Model

The company's plan to train a JEPA on the ionic current through a nanopore is set out in an interactive essay of 2026 in the voice of Keltar. The essay calls the decision to predict in a learned space "the single best decision in this whole stack", because "A 200 kHz current trace is mostly thermal noise, capacitive ringing, and the amplifier's Bessel filter talking to itself." It warns that video JEPAs come with a slowly updated teacher network, that a home-made version for current traces "does not get that for free", and that without one "the predictor collapses every signal to one constant vector and reports a beautiful, meaningless loss of ~0."[15]

Around the JEPA, the plans put a Conformer for the current, a Perceiver to fuse current, voltage and microscope video, and a language model to narrate the results.[16] Before any of it is built, the essay proposes a test: pretrain on public nanopore recordings and check whether the frozen embeddings beat five classical numbers, "for ~$50 of GPU". The company's guide files the resulting world model of the signal under plans: "The model itself is still ahead of us."[17] The wider program is described in Machine learning at Molecular Reality and Machine learning on nanopore signals.

See also

References

  1. ^ a b LeCun, Yann (June 27, 2022). "A path towards autonomous machine intelligence, version 0.9.2". OpenReview.
  2. ^ Bromley, J.; et al. (1993). "Signature verification using a "Siamese" time delay neural network". International Journal of Pattern Recognition and Artificial Intelligence 7 (4): 669-688. doi:10.1142/S0218001493000339.
  3. ^ a b Assran, M.; et al. (2023). "Self-supervised learning from images with a joint-embedding predictive architecture". arXiv. doi:10.48550/arXiv.2301.08243.
  4. ^ a b Bardes, A.; et al. (2024). "Revisiting feature prediction for learning visual representations from video". arXiv. doi:10.48550/arXiv.2404.08471.
  5. ^ He, K.; et al. (2021). "Masked autoencoders are scalable vision learners". arXiv. doi:10.48550/arXiv.2111.06377.
  6. ^ Chen, T.; et al. (2020). "A simple framework for contrastive learning of visual representations". arXiv. doi:10.48550/arXiv.2002.05709.
  7. ^ Grill, J.-B.; et al. (2020). "Bootstrap your own latent: a new approach to self-supervised learning". arXiv. doi:10.48550/arXiv.2006.07733.
  8. ^ Bardes, A.; Ponce, J.; LeCun, Y. (2021). "VICReg: variance-invariance-covariance regularization for self-supervised learning". arXiv. doi:10.48550/arXiv.2105.04906.
  9. ^ Jing, L.; et al. (2021). "Understanding dimensional collapse in contrastive self-supervised learning". arXiv. doi:10.48550/arXiv.2110.09348.
  10. ^ Garrido, Q.; et al. (2022). "RankMe: assessing the downstream performance of pretrained self-supervised representations by their rank". arXiv. doi:10.48550/arXiv.2210.02885.
  11. ^ Weimann, K.; Conrad, T. O. F. (2024). "Self-supervised pre-training with joint-embedding predictive architecture boosts ECG classification performance". arXiv. doi:10.48550/arXiv.2410.13867.
  12. ^ a b Ennadir, S.; et al. (2025). "Joint embeddings go temporal". arXiv. doi:10.48550/arXiv.2509.25449.
  13. ^ Assran, M.; et al. (2025). "V-JEPA 2: self-supervised video models enable understanding, prediction and planning". arXiv. doi:10.48550/arXiv.2506.09985.
  14. ^ Anna Heim (January 23, 2026). "Who's behind AMI Labs, Yann LeCun's 'world model' startup". TechCrunch.
  15. ^ "Keltar's Critique v2: The Nanopore World Model (Hardened)". Molecular Reality Corporation. June 2026.
  16. ^ "Company records". Molecular Reality Corporation. October 2026.
  17. ^ "Keltar's Field Guide to Molecular Reality". Molecular Reality Corporation. August 2026.
Retrieved from "https://molecularreality.com/wiki/Joint_embedding_predictive_architecture"
Last edited 1 hour ago by Translocation event