Understanding how stochastic gradient descent learns internal representations in wide neural networks remains a central problem, especially when the target function is itself a network of the same architecture. In the regime where width and input dimension grow proportionally, standard approaches require tracking a diverging number of overlaps between learned and target features. Zhou, Barbier, and Goldt instead develop a mesoscopic description that characterizes the distribution of these overlaps, reducing the dynamics to a few order parameters in the early stages of learning.

The authors show that early training is well described by a Gaussian approximation, where a small set of collective variables captures how the network learns the target's linear and quadratic components. This description breaks down at a predictable timescale, marking the onset of specialization, when individual learned features begin to strongly correlate with specific target features. In extensive-rank networks, specialization is accelerated by a quadratic shielding mechanism: learning the linear and quadratic parts of the target filters gradient fluctuations, sharpening the signal for feature alignment.

The activation function plays a key role in shaping the specialization pattern, and the paper characterizes how learned features refine after specialization begins. Together, these results clarify when large neural networks admit a simple statistical description and how collective learning gives way to individual feature discovery, offering a new theoretical handle on representation learning in wide networks.