An arXiv preprint asks a deceptively simple question: how much of its prompt does an autoregressive transformer remember after generating t tokens? The authors consider two different prompts followed by the same continuation text, so any difference in the model's hidden states must come from the original prompt. They prove a 'conditional no-hair theorem' showing that these differences vanish as t grows — the model's internal state becomes independent of the prompt.

Borrowing the term from black-hole physics, where a 'no-hair theorem' says a black hole forgets everything but mass, charge, and angular momentum, the result formalizes a kind of memory loss in transformers. The theorem is conditional, meaning it applies under specific assumptions about the model and inputs, which the abstract does not fully spell out.

Because this is a preprint, it has not yet been peer-reviewed. Still, the result offers a theoretical lens on a practical phenomenon: long generations may drift from the original instruction or context.