A new arXiv paper, "First Passage Attention," addresses a blind spot in augmented softmax attention. Many attempts to add forget gates to attention rely on summing decay contributions along the key–query path. That design makes the accumulated decay monotonic, meaning the model can only progressively forget older context — it cannot pick out and erase specific intermediate information while preserving what comes before or after.
The authors argue this is a fundamental restriction for tasks where selective context removal matters. Their proposed method, First Passage Attention, is positioned as an alternative that avoids the monotonic accumulation constraint, though the abstract does not detail the full mechanism.
The paper is announced as a new arXiv submission, so it has not yet undergone peer review. The abstract alone leaves open how the first-passage formulation is implemented and evaluated, but the motivation is clear: current forget-gated attention is too rigid for selective erasure.