A new preprint on arXiv, titled LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocation, explores how looped Mixture-of-Experts (MoE) models can be made more efficient. Looped MoE models combine parameter reuse with sparse computation, aiming for better performance under matched compute budgets. The abstract notes that prior work has shown different loop visits can take on different roles, even when they share the same parameters.

The authors propose LatentLoop, a method that allocates compute asymmetrically across loop iterations. Rather than treating each pass through the loop identically, the model appears to vary how much computation is spent on each visit. This is positioned as a way to scale looped MoE models further, building on the observation that loop visits are not functionally equivalent.

The abstract is brief and does not include experimental details or quantitative results. As an arXiv preprint, the work has not yet undergone peer review. The full paper would presumably describe the allocation strategy and its effects on performance, but those specifics are not available from the abstract alone.