LLM unlearning aims to erase specific knowledge from a model without harming its overall capabilities. According to a new preprint on arXiv, the usual evaluation metrics—target suppression and clean retain utility—may not tell the whole story. The paper describes a local failure mode where a model performs well on standard retain prompts but degrades on prompts that are adjacent to the forgotten data.
To address this, the paper proposes RA^2 (Retain-Anchored Attraction). The approach is designed to preserve what it calls forget-adjacent utility: the model's ability to handle related content that should remain intact. By anchoring on retained data, the method aims to prevent the unlearning process from causing collateral damage in the neighborhood of the target.
This is a single source, so there are no conflicting findings to compare. Its contribution is to draw attention to this evaluation blind spot and offer a specific mechanism for mitigating it. As with any preprint, the results have not yet been peer-reviewed.