Machine unlearning is the task of removing targeted information from large language models after training. The goal is to ensure that specific data no longer influences model outputs. But according to a new arXiv paper, how we verify that removal may be missing the point.
The paper notes that existing evaluations often measure forgetting only on a designated benchmark distribution. In other words, a model is considered "unlearned" if it performs poorly on a specific test set designed to probe for the targeted information. The authors argue that this approach captures only behavioral forgetting.
Their central claim, reflected in the title, is that behavioral forgetting is not privacy. Passing a benchmark does not necessarily mean the underlying information is gone in a meaningful privacy sense. The abstract is truncated in the source, so the paper's proposed alternative is not detailed here, but the critique is clear: current evaluation methods may be measuring the wrong thing.