Multimodal large language models (MLLMs) are trained on vast datasets that may include sensitive, copyrighted, or unsafe material. When prompted, these models can reproduce such content, raising legal and ethical concerns. Existing unlearning methods often remove too much, degrading performance on benign tasks.

A new framework called CUSE, described in an arXiv preprint, aims for fine-grained unlearning. It combines counterfactual guidance—using altered examples to steer the model away from specific outputs—with on-policy self-distillation, a technique that helps the model retain its original capabilities on unrelated content. The approach is designed to remove target information while preserving non-target content, even when the two are closely intertwined.

The significance lies in the granularity: rather than scrubbing entire topics or modalities, CUSE targets specific pieces of information while leaving the rest of the model's knowledge intact. This is particularly important for practical deployment, where models must remain useful after unlearning. The preprint is available on arXiv under the identifier 2610.14763.