Stochastic gradient descent is usually analyzed as an optimization algorithm, but a new paper treats it as a nonlinear time series. The authors model constant-step-size SGD as an iterated random function system, a perspective that lets them track the algorithm's distribution over iterations rather than just its convergence in probability.

The central result is geometric moment contraction, which gives convergence in ordinary Euclidean distance. This is stronger than earlier Wasserstein-distance results and handles the non-stationarity caused by a fixed starting point. From this, the authors prove a quenched central limit theorem, an invariance principle, and alpha-stable convergence for averaged SGD, all independent of initialization.

The paper also examines how the algorithm depends on the step size by differentiating the SGD recursion with respect to it. This leads to an interpolation scheme that reconstructs the full trajectory, and to a Richardson-Romberg extrapolation using multiple step sizes that reduces bias and pushes averaged estimates closer to the global optimum. The authors add an online inference method and a bias-reduced variant, with confidence intervals whose coverage approaches the nominal level as n grows.