Recent work has suggested that switching optimizers between pretraining and later training stages can hurt model performance, but the effect has not been studied systematically. A new arXiv preprint takes up that question by examining the transition from AdamW to Muon, two optimizers used in large language model training.
According to the abstract, the paper investigates the "mismatch" between these optimizers as a concrete instance of the broader phenomenon. Because only the abstract is available from the source, the article does not specify the magnitude of the cost or the exact training stages covered.
With a single source, there are no independent findings to compare. The main contribution described is the attempt to move from suggestive evidence toward a more systematic understanding of optimizer-switching costs.