A new arXiv paper studies classification dynamics under small initialization and identifies a pronounced non-monotone behavior of accuracy, which the authors call the accuracy well. The abstract describes accuracy rising rapidly early in training, subsequently dropping, and only much later.

The authors note that this behavior is non-monotone, meaning accuracy does not improve steadily over time. The paper is available on arXiv under the identifier 2610.20439.