A recent arXiv preprint proposes a formal way to study autoregressive token generation, framing it as a circuit called a sequential sampler. In this model, each new symbol is sampled based only on the preceding symbols and possibly a small amount of retained state. The abstraction is designed to capture the essential structure of autoregressive language models while making their computational limits tractable.

The abstract indicates that the authors prove results about what such samplers can achieve, though the full list of findings is not included in the available text. The framing suggests a focus on inherent limitations: with restricted state, certain kinds of generation may be impossible or require many more steps.

Because only the abstract is available here, the article does not detail specific theorems or bounds. The significance lies in the model itself, which could help researchers reason about the theoretical boundaries of current generation methods rather than relying only on empirical observation.