Transfer Bayesian optimization (BO) reuses data from prior tasks, but published comparisons typically report a single average effect of acquisition choice across hidden regimes. Practitioners need the conditional effect for their own prior quality, budget ratio, and metric. In a paper posted on arXiv, Noel Thomas argues this mismatch is widespread: an audit of 40 transfer-BO papers from NeurIPS, ICML, ICLR, AISTATS, UAI, TMLR, JMLR, and AutoML-Conf (2022–2025) found that 98% never vary the budget-to-archive ratio B/|A| as a controlled axis.
The stakes are concrete. On the GDSC2 drug-response benchmark, changing only the budget reverses the ranking: at B=50, Greedy outperforms UCB by 0.050 Hit@1, while at B=100, UCB outperforms Greedy by 0.035. To capture this, the paper introduces the Portable Regime Score PRS=(B/|A|)(1-rho), where rho is the prior rank correlation estimated from pilot contexts. Across 79 conditions, a hierarchical model gives beta=0.50 (p=1.1e-9), and 19% of conditions fall in an equivalence zone where the advantage is below 0.01 Hit@1.
The paper also proposes RegimePlanner, which estimates rho online and switches acquisition accordingly; it wins all 16 HPO-B search spaces at B=100 and exceeds a matched per-context oracle on GDSC2 by 18%. Pre-registered predictions achieved 67.5% overall accuracy (27/40) and above 90% within EMA prior families. The author argues that unconditional rankings are unstable because when the conditional effect changes sign across regimes, the reported average effect becomes a function of the benchmark mixture. He recommends reporting B/|A|, rho, K, and metric alongside any claimed acquisition advantage.