Adam is a widely used optimizer for training large language models, prized for its optimization performance and stability. But its element-wise estimation of the first and second moments of gradients comes with a substantial memory cost, which becomes a bottleneck as model sizes grow.
The new arXiv paper, titled AdamPM: Memory-Efficient LLM Optimization via Moment Pooling, proposes a way to address this overhead. The abstract frames moment pooling as the key idea, suggesting a more memory-efficient approach to LLM optimization. However, the abstract does not describe how the pooling works or report measured memory savings.
Since this is the only source, there is no independent comparison to make. The paper is announced as new on arXiv, but the abstract should be read as an early signal rather than a full evaluation of the method.