Sharp minima are often associated with poor generalisation in deep neural networks. Sharpness-Aware Minimisation (SAM) improves generalisation by minimising a combination of the training loss and a sharpness measure.
But SAM uses a fixed perturbation size when estimating sharpness. The new paper, titled 'One Size Does Not Fit All', argues that this is a limitation: the appropriate perturbation should depend on the local curvature of the loss surface. The proposed method, curvature-aware adaptive perturbation, adjusts the step size based on the geometry of the loss landscape.
This adaptation could allow the optimiser to avoid sharp minima more effectively than a fixed-radius SAM. The abstract suggests that the method is designed to improve generalisation, though the full details of the implementation and experimental results are not included in the available snippet.