Self-supervised pretraining is increasingly used to improve transfer on small, label-limited molecular property prediction tasks. However, commonly used contrastive objectives can be sensitive to how views of a molecule are constructed, which limits their robustness.
The new paper introduces M-JEPA, a predictive self-supervised learning method for molecular graphs. Instead of contrasting augmented views, M-JEPA learns by predicting missing or latent information, an approach intended to avoid the fragility introduced by view construction.
To assess transfer, the authors propose a scaffold-shift evaluation on the Tox21 benchmark, which measures performance on molecules whose scaffolds differ from those seen during pretraining. The work positions predictive self-supervision as a more robust alternative to contrastive pretraining for label-limited molecular tasks.