A new preprint on arXiv argues that comparing truncation samplers—the rules that decide which tokens a language model may output—is best understood as a measurement problem. The authors observe that almost every deployed truncation rule is nested, meaning the rules form a hierarchy rather than a set of fundamentally different alternatives.
The motivation is practical. The paper notes that settling a single comparison between two truncation samplers has cost six thousand A100-hours, and there is no way to know in advance how far apart two samplers can be. This makes rigorous benchmarking extremely expensive and suggests that many reported differences may be artifacts of how the comparison is set up.
The authors frame the issue as a measurement problem: if the rules are nested, then the choice of evaluation metric and test distribution can determine which sampler appears to win. The preprint does not provide a full solution, but it suggests that future work should focus on measurement methodology rather than on pitting individual rules against each other.