Language models sometimes repeat verbatim until they hit the generation limit. A new arXiv preprint approaches this failure from the exit side: once a span has already been repeated n times, the authors ask, can the model still get out? The focus is not on how the loop starts but on whether escape remains possible as the loop continues.
The authors study conditions under which repetition locks in, looking specifically at truncation sampling and bounded penalties. They also point to a blind spot in DRY, a method referenced in the title of the work. The paper is a preprint and has not yet been peer-reviewed.