Sparse autoencoders are a popular tool for interpreting language models by breaking activations into sparse, human-readable features. A new preprint on arXiv (2610.21240) warns that these features are not as stable as they might seem.

The authors show that when a prompt is transformed in ways that preserve the underlying request, the SAE redistributes activity across features. As a result, observing a feature change on one prompt does not necessarily indicate a meaningful shift in the model's behavior or intent.

The finding has implications for interpretability research that relies on SAE feature tracking. The paper suggests that researchers need to account for this redistribution when drawing conclusions from feature-level analyses.