A recent arXiv preprint, announced as a replacement, develops strong Gaussian approximations for constant-learning-rate stochastic gradient descent (SGD). The work is framed in the style of KMT-type results, which provide tight probabilistic bounds on how closely a stochastic process can be approximated by a Gaussian process. The authors state that they work with heavy-tailed noise, a departure from many classical analyses that assume light-tailed or sub-Gaussian conditions.
The abstract indicates two key assumptions: the number of iterates grows, and the learning rate is constant. This is notable because constant learning rates are common in practice but harder to analyze than decaying schedules. The paper appears to focus on the bias and trajectory inference of the SGD iterates under these conditions, though the full details are not included in the abstract.
As a single source, there are no conflicting findings to compare. The contribution is theoretical, and the abstract does not describe experiments or empirical validation. Readers interested in the precise conditions and the form of the approximation will need to consult the full manuscript on arXiv.