Neural networks are known to exhibit spectral bias: they tend to learn low-frequency components of a target function before high-frequency ones. This behavior, also called the F-principle, has been confirmed in numerous experiments, but a complete theoretical explanation has remained incomplete. A new paper on arXiv proposes a framework based on orthogonal polynomial dynamics to account for this ordering in shallow networks.
The authors model the training process as an expansion in orthogonal polynomials, showing how the network's error decomposes into sinusoidal components that are learned at different rates. Their analysis suggests that the dynamics naturally favor lower-frequency modes early in training, matching the observed spectral bias. This is presented as a step toward a rigorous understanding of why this ordering occurs, rather than just a description of it.
Because the abstract is the only source, the details of the mathematical derivation and the exact conditions under which the result holds are not fully available here. Still, the work offers a potentially useful lens for interpreting a well-known empirical pattern, and it may inform future research on network design and training schedules. The paper is available on arXiv under identifier 2610.21964.