The scaling relationship between compute and performance in language models has been widely studied. A new paper on arXiv shifts attention to memory scaling, proposing a method called COUNT that adds embeddings for n-grams to scale memory use.

The approach is designed to work on host memory, and the authors say it enables training-free adaptation. That means models could be adapted to new tasks without additional training, by leveraging counted n-grams.

The abstract notes that models are increasingly being designed to scale memory use as well as compute. COUNT appears to be a step in that direction, though the full details of the method and its evaluation are not included in the abstract.