A new arXiv preprint asks a practical question for anyone deploying language models that keep learning online: how do you stop the model from slowly eroding its own base capabilities? The authors frame this as a problem of representation drift, and they investigate where in a modern LLM continual updates should be applied.
The study specifically contrasts LoRA-style parameter updates with n-gram memory, measuring how each choice changes the model's internal representations. The abstract suggests the goal is to identify which part of the model can absorb new information without destabilizing what it already knows.
Because the source is only a truncated abstract, no findings or numbers are reported here. The paper is listed as a new arXiv announcement, so the full results are not yet available from this source.