Standard Bayesian experimental design assumes the experimenter will use the collected data to choose an action, based on their own beliefs and goals. But in many real-world settings, data are released publicly and acted on by external parties whose decision procedures the experimenter cannot change and may not even know. Iqbal and Rainforth address this gap with a framework they call usage-aware experimental design.
The core idea is to train an adaptive design policy that minimises usage risk—the experimenter's expected loss when one or more external rules act on the data. These rules can be non-Bayesian or black-box, and when several may apply, the designer can either average over them or take the worst case. This shifts the objective from reducing the experimenter's own uncertainty to improving outcomes under someone else's fixed procedure.
A key theoretical result is that, unlike in standard BED, more informative data can be harmful. Because external rules are fixed, reducing uncertainty can lead to worse decisions under those rules. Consequently, usage risk cannot in general be expressed as an expected uncertainty reduction, which means the framework strictly generalises standard BED. The authors show that this approach can tackle problems that are difficult to address with existing methods, particularly when data will be fed into unknown or non-Bayesian decision pipelines—a common situation in public data releases and regulatory science.