In sequential decision-making, comparing several candidate policies usually requires separate data collection for each one, which can be costly. A new arXiv paper studies this problem in the setting of stochastic linear contextual bandits, where rewards depend linearly on a known feature map.
The authors propose PRISMA, an algorithm that evaluates K target policies at once using reward-free policy queries. The title, "Explore Once, Evaluate Many," suggests the central idea: a single exploration phase can support multiple policy evaluations, potentially reducing the total number of interactions needed.
The abstract is brief, so the paper's precise guarantees and implementation details are not fully described in the announcement. Nevertheless, the problem is practically relevant for applications such as personalized recommendations and clinical trials, where reusing exploration data across policies could lower cost and improve efficiency.