Long-term memory for AI assistants requires a gate that decides when a new utterance should retire an old stored claim. The authors of a new arXiv preprint argue that existing benchmark labels are too coarse for this task: they mark a pair of statements as changed or not changed, but that binary signal cannot tell a gate whether the new utterance actually supersedes the old claim.
The paper pre-registers an evaluation of a "system-one" model—a fast, heuristic model—as this retirement gate. The title's distinction between "suggested" and "superseded" points to the core issue: a new utterance may suggest a different fact without definitively replacing the stored one, and a gate that acts on mere suggestion would retire memories prematurely.
Because the abstract is truncated, the full results are not yet available, but the framing highlights a measurement gap in memory-editing benchmarks. If the gate is to be evaluated properly, the authors imply, labels must capture the retirement decision itself, not just the presence of a change.