Differentially private learning often relies on preconditioners like KFAC to speed up convergence, but building these preconditioners from private data can leak information. A new arXiv preprint explores a data-free alternative: construct the Kronecker factors of KFAC from synthetic probes rather than from the actual training data.

The proposed method, called DP-KFC, validates a probe by checking whether its eigenvalue decay matches that of the private factors. The authors argue that this spectrum-matching criterion is not enough. They provide a random-matrix account that turns the validation criterion into a more stringent condition, suggesting that matching only the spectrum can miss important structure.

The work is theoretical and appears as an arXiv preprint, so it has not yet been peer-reviewed. Still, it points to a concrete limitation in current data-free preconditioning approaches and offers a new framework for designing better validation rules in differentially private optimization.