Molecular machine learning models are often evaluated with structure-disjoint splits, which separate training and test molecules so that chemically similar structures do not appear in both sets. According to a new arXiv preprint (arXiv:2610.12694v1), the standard way of building these splits is not as fixed as it seems. The construction rule is set once at the scale the library happened to have: choose a pairwise distance threshold, then assign whole structures to one side or the other.
Because the threshold and assignment are tied to a particular library, the preprint argues that the split itself drifts when the library changes size or composition. A split that looks challenging at one scale may become easier or harder at another, making it difficult to compare results across studies that use different molecular libraries.
The preprint's title, "Critical Mass: Structure-Disjoint Splits Drift With the Libraries They Are Built On," points to this dependency. Based on the abstract, the main claim is about the construction rule rather than a specific benchmark result. No conflicting sources were reviewed.