Most decision-making frameworks focus on selecting a single optimal action. But many modern learning systems—such as entropy-regularized reinforcement learning, RLHF, and reward-guided generative models—are designed to sample a diverse set of high-quality outputs. A new arXiv paper introduces the 'tilted sampling problem' to formalize this setting.

The paper shows that two classic techniques, optimism and posterior sampling, achieve logarithmic regret, meaning the cumulative loss grows only logarithmically with the number of decisions. This is a strong theoretical guarantee for a problem that has become increasingly relevant in practice.

By providing a unified framework, the paper offers a common language for comparing algorithms across different applications. It also sets a benchmark for evaluating sampling-based learning systems, which are central to modern generative AI and alignment techniques.