Looped transformers are designed to reuse a single block across many iterations, which lets them keep computing after training ends. In principle, that flexibility is useful: the model can spend more steps on harder inputs. But a new preprint from arXiv suggests this convenience comes with a catch.

According to the abstract, recurrent reasoners of this kind are prone to 'overthinking.' Iterating beyond the depth seen during training can take a model that had the right answer and push it toward a wrong one. The paper's title frames the problem more sharply: learning to compute is not the same as learning to retain or reuse what has already been computed.

The distinction matters for anyone relying on looped transformers for tasks that benefit from extra inference-time computation. More loops are not automatically better, and the model's training signal apparently does not guarantee that it will preserve earlier correct states. The full details are in the paper, but the abstract alone is a warning that variable compute has its own failure modes.