Looped transformers are a class of models that apply the same block repeatedly, rather than stacking distinct layers. This design raises a practical question: if you train a block at a certain depth, can you simply run it for more iterations at test time to get deeper reasoning? A new arXiv preprint takes up this question in the specific context of all-pairs graph connectivity.
Using that task as a testbed, the authors investigate how far a block trained at one depth can be safely extrapolated to greater depths. The abstract as provided focuses on the question rather than the answer, so no results are reported here. The framing suggests the answer is not unlimited, and that the work is a step toward understanding when looped transformers can generalize beyond their training depth.
The study appears to target a core limitation of parameter-efficient transformer designs: knowing when a fixed block can be reused more times than it was trained for. For tasks like graph connectivity, where iterative computation is natural, this question is especially relevant.