SHAP has become a standard tool for attributing importance to features in predictive models, but its explanations are local by nature. To get global feature importance, practitioners typically average local SHAP values, for example by computing mean absolute SHAP or mean squared SHAP. Yet, as a new arXiv paper notes, there have been no methods for performing statistical inference on these aggregated quantities.
The paper, by Justin Whitehouse, Ayush Sawarni, and Vasilis Syrgkanis, treats the SHAP curve as a nuisance function estimated from data. For p-th powers of SHAP with p at least 2, the authors construct a debiased estimator that combines U-statistics with Neyman orthogonal scores for functionals of nested regressions, and show it is asymptotically normal. For the harder case where p is between 1 and 2, the target parameter is not twice differentiable, so they build debiased U-statistics for a smoothed alternative and carefully tune the smoothing temperature to obtain inference for the true, unsmoothed quantity.
The authors also introduce a Neyman orthogonal loss for learning the SHAP curve through empirical risk minimization, and they discuss excess risk guarantees for commonly used function classes. Together, these results provide a way to attach confidence statements to global SHAP-based feature importance, which could make feature selection and model explainability more rigorous in practice.