Deep neural networks often learn shortcuts, relying on spurious associations such as acquisition artifacts or patient demographics rather than the intended signal. Before deployment, models should be audited for such reliance, but current methods—like linear probes and concept activation vectors—ask only whether a concept is decodable from a layer's representations. The authors of a new arXiv paper argue this framing is misleading, because decodability reflects not just concept reliance but also correlations among concepts and outcomes in the auditing dataset.
Their proposed method, ICON (Independent Canonical cONcepts), reframes shortcut auditing as a variance decomposition problem. Instead of asking whether a concept is decodable, ICON quantifies the share of a layer's variance explained by each concept conditional on all other concepts and the outcome, while also reporting the variance left unexplained. Because these explanations are comparable across layers, the resulting trajectories can distinguish concepts that actually drive predictions from those that are merely encoded in the representations.
The authors validated ICON on simulated data with known ground truth, where it recovered concept importance more accurately than seven baseline methods. In real-world applications involving skin-cancer classification and neuroimaging, ICON separated learned shortcuts from merely correlated concepts, with independent validation checks confirming the results. This offers a more rigorous way to audit models before deployment, potentially reducing the risk of failures caused by spurious correlations.