Soft spectral clipping (SSC) has been used to restrain the largest singular values of gradients during LLM pretraining. The arXiv paper notes that the benefit of this operation depends on both the current gradient scale and the chosen clipping threshold, making it sensitive to settings that can shift during training.

To address that, the authors introduce CAST (Clipping with Adaptive Scales and Thresholds). Rather than applying fixed clipping, CAST tracks gradient history and adjusts the scale and threshold accordingly. This makes it a history-adaptive extension of SSC, intended to keep the regularisation effect consistent as training progresses.

Since the announcement is the only source, there are no independent results or conflicting findings to compare. The abstract does not report experimental details, so the practical gains of CAST remain to be evaluated.