A new arXiv paper tackles a core problem in personalizing large language models: deciding what to remember from user feedback. When a user corrects an LLM, the feedback may reflect a long-term preference or just a fix for that specific request. The authors argue that blindly retaining all feedback can cause negative transfer, where irrelevant corrections degrade performance on later tasks.

To address this, they propose a transfer-aware approach that learns which feedback is worth keeping as a reusable skill. The idea is to selectively retain only the feedback that generalizes across requests, rather than storing every correction. This shifts the focus from simply accumulating data to curating it for future usefulness.

The work highlights a key tension in personalization: more feedback is not always better. By making retention decisions explicit, the method aims to improve both the efficiency and reliability of lifelong learning in LLMs. Further details on the mechanism and experimental results are available in the full preprint.