Repetitive and degenerate output remains a known weakness in large language models, and the usual fixes—temperature, nucleus, and top-k sampling—are applied as fixed rules rather than adaptive strategies. A new arXiv paper proposes an information-theoretic foundation for online temperature sampling, aiming to address this problem more rigorously.
The authors draw on Kac's lemma and the asymptotic equipartition property (AEP) to model the decoding process as a stationary source. Their approach suggests that the time a source spends in certain states can inform how sampling parameters should be adjusted online, rather than relying on static thresholds.
Because the abstract is truncated, the full derivation and experimental results are not yet visible. Still, the framing indicates a shift from heuristic tuning toward a principled, source-aware method for controlling generation diversity.