Over-parameterized neural networks generalize surprisingly well, but learning theory still lacks a consensus on why. A new arXiv paper reframes this puzzle around compression: is there a notion of compression that actually implies generalization?
The paper's abstract says existing answers measure complexity at different levels, ranging from weights to further levels of model structure. The authors take a minimum description length perspective, suggesting that the right compression measure may be the one that captures how succinctly a model can describe the data.
Because only the abstract was available for this digest, the full argument is not summarized here. Based on the announcement, the paper's contribution is to examine whether MDL can serve as the missing link between compression and generalization.