PAC-Bayes theory offers a principled way to bound generalization error, but its standard form places priors over the Euclidean parameter space, causing the relative entropy term to grow with the network's dimension. For today's overparameterized deep networks, that dependence makes the bounds vacuous. A new paper on arXiv addresses this by turning to Dirichlet processes, a nonparametric prior framework.
Instead of working directly in the high-dimensional weight space, the paper constructs priors that allow the bound to be independent of the number of parameters. The abstract reports that this yields dimension-free PAC-Bayes generalization guarantees for deep neural networks, though the full derivation and experimental validation are not included in the announcement.
Because the source is only an abstract, we cannot verify the mathematical details or compare with other approaches. The contribution, if correct, would be a step toward tighter, more usable generalization bounds for large models.