Sparse autoencoders (SAEs) represent model activations with sparse, linearly decoded features. A new paper on arXiv examines whether the downstream effects of these features are additive—that is, whether the effect of multiple features equals the sum of their individual effects.
The paper argues that this additive assumption does not necessarily hold. To measure the discrepancy, the authors introduce SAE Additivity Residuals (SAR), which quantify the gap between observed downstream behavior and what would be expected from linear composition of features. The preprint is available on arXiv under identifier 2610.21261.
By quantifying where additive composition fails, SAR could help researchers assess when feature-based explanations of model behavior are trustworthy. This is especially relevant for interpretability methods that rely on SAEs and assume linearity.