Contextual bandit algorithms often rely on optimism in the face of uncertainty (OFU), building uncertainty sets from statistical confidence intervals. This paper from Haonan Xu and Yingying Li instead borrows set membership estimation (SME), a tool widely used in control and system identification, and applies it to linear contextual bandits. The authors propose SME-OFU, which constructs uncertainty sets via SME and then makes decisions optimistically over those sets.
The paper provides a theoretical regret bound for SME-OFU, showing that it achieves sublinear regret under proper assumptions. This is the first step, the authors note, in connecting SME to the broader bandit literature, where such methods have received little attention. They also run simulations that demonstrate promising performance compared with existing approaches.
Because the article is based on a single source, there are no conflicting findings to reconcile. The contribution is primarily a proof-of-concept: SME, a well-established uncertainty quantification technique in control, can be adapted to contextual bandits with both theoretical guarantees and encouraging empirical results.