A new paper on arXiv examines the last iterate of stochastic gradient descent (SGD) with a constant step size, rather than the more commonly analyzed averaged iterate. The authors consider smooth convex objectives in the interpolation and low-noise regimes, where the variance of the stochastic gradients at the optimum is small.
The abstract indicates that the work establishes near-optimal convergence rates for the final iterate in these settings. This is significant because practitioners often use the last iterate in practice, and theoretical guarantees for it are stronger and more directly applicable than those for averaged solutions.
As the source is a single abstract, no comparison with other work is possible. The full details of the proof and exact rates are not provided in the abstract, but the title and summary point to a tight bound in the smooth interpolation regime.