Sparse autoencoders (SAEs) have become a standard tool for interpreting neural network activations, but they are usually trained on massive datasets—hundreds of millions to billions of tokens. A new preprint on arXiv asks whether that scale is truly necessary, and whether more data always leads to more reliable features.
The authors trained four SAEs to study how data volume affects the learned features. Their results point to diminishing returns: beyond a certain point, adding more activation data does not meaningfully improve the features. More strikingly, they find that an SAE can behave functionally well while still failing to reproduce the same features when retrained—meaning that good performance on reconstruction or downstream tasks does not guarantee that the interpretable units are stable.
This distinction matters for interpretability research. If features are not reproducible, conclusions drawn from a single SAE may be fragile. The preprint is a reminder that scale alone is not a substitute for careful evaluation of feature consistency, and that the field may need to develop better metrics for what makes an SAE trustworthy.