Vision foundation models encode rich semantic structure, but most tools only test pre-defined concepts. Researchers from Ohio State and collaborators asked whether sparse autoencoders (SAEs) could instead surface unknown visual patterns directly from a model's representations, without concept-specific labels. They paired SAEs with heuristic feature ranking to score candidate features by how distinctly they separate groups in the data.
The team tested the approach in three stages. On the ADE20K scene dataset, SAEs beat k-means, PCA, and SemiNMF across most metrics for rediscovering known concepts. On FishVista, the same pipeline recovered fine-grained fish anatomical features with no supervision. On Heliconius butterflies, a ranking based on the Mann-Whitney U statistic consistently highlighted real diagnostic keys that distinguish mimic subspecies.
The authors stress that these experiments validate feature surfacing through rediscovery, not the discovery of previously unknown phenomena. They frame the work as a methodological prerequisite: a way to generate candidate visual features that scientists can then investigate. Prospective open-ended discovery remains future work, but the results suggest SAEs could help move vision models from confirmation to exploration.