Gradient descent with a large learning rate is known to succeed in training neural networks even when the step size violates classical smoothness-based conditions. Most theoretical explanations for this behavior, however, have been limited to linear models or other simplified settings.

The new preprint takes a step beyond that by analyzing a minimal ReLU classifier trained on non-linearly separable data. The authors study convergence and implicit bias both at and beyond the 'edge of stability' — the point at which the learning rate exceeds the threshold suggested by smoothness-based analysis.

The abstract is truncated in the source, so the full results are not available from the announcement. What is clear is the intended scope: extending the theory of large-learning-rate gradient descent from linear models to a simple nonlinear ReLU network.