Researchers behind a new arXiv preprint argue that the usual intuition about sampling temperature is incomplete. While it is known that larger sample budgets benefit from higher temperatures in large language models, the field has lacked a clear account of what temperature tuning is actually optimizing and why it works. The authors propose treating decoding as a pass@k maximization problem, where success is measured by whether at least one of k sampled outputs meets the desired criterion.
Under this framing, the optimal temperature is not a fixed property of the model or task but depends on the sampling budget. The paper introduces the idea of a 'tilted temperature' as the optimal decoder, suggesting that the temperature should be adjusted as the number of samples grows. This shifts the question from 'which temperature works best empirically' to 'what objective temperature tuning is solving.'
Because the abstract is brief, the full derivation and experimental evidence are not yet visible from the announcement. The contribution, as stated, is conceptual: connecting temperature choice to pass@k optimization and offering an explanation for why more samples call for higher temperatures.