Looped large language models reuse the same parameters across multiple recurrent steps, trading parameter storage for extra computation. However, as the arXiv abstract notes, most such models are trained from scratch, and pretraining with looped depth is expensive.
The UpLoop method is proposed to address this. Instead of training a looped model from scratch, it upcycles an already pretrained non-looped model into a looped one using a gated loop. This post-training conversion aims to get the benefits of looped architectures without the high pretraining cost.
The abstract is brief and does not provide experimental details or comparisons. As a single source, there are no differing accounts to reconcile; the description here follows the abstract's claims.