Large language models acquire new skills through fine-tuning, but that process can destroy capabilities they already have. This problem, known as catastrophic forgetting, is a central obstacle to continual learning in LLMs, according to a new arXiv paper.

The paper, titled "Towards Continual Learning: Mitigating Forgetting With Subspace Tuning," proposes subspace tuning as a way to address the issue. The abstract suggests that recent work connects to this direction, though the full method and results are not described in the available text.

Because the source is a single arXiv abstract, the details of how subspace tuning works and how well it performs remain unclear. The paper's contribution appears to be a targeted approach to preserving old knowledge while adding new skills, but further reading of the full manuscript would be needed to assess its claims.