Multi-token prediction (MTP) has become a popular way to improve language model training, but existing methods are "auxiliary by design": they attach extra modules to the backbone and require an additional pass for each added prediction horizon. That makes them increasingly expensive as the number of horizons grows.
The new arXiv paper, ECHO, proposes a different approach called "efficient causal self-recursion." Rather than stacking modules for each horizon, the method aims to achieve multi-horizon training within the backbone itself. The abstract argues this goes beyond auxiliary MTP, though the full technical details are left to the paper.
Since this is a single preprint, there are no conflicting sources to compare. The paper is listed as new on arXiv under ID 2610.24766.