Test-time scaling lets large language models improve by generating multiple answers and aggregating them. Weighted ensembles go further, drawing answers from several models with learned weights. A new theory paper examines the two sample sizes involved: the number of labeled questions Q and the number of generations N per model per question. The authors study the clean risk of the ensemble's infinite-generation voting rule and show that buffered empirical risk minimization (ERM) achieves excess risk of order Q^{-1/2} + N^{-β/2}, up to logarithmic factors, under a margin condition with exponent β.
To test optimality, the authors analyze a simplified submodel with two models and two possible answers per question. They characterize the minimax excess risk up to constants using a Hellinger modulus for a binomial-mixture experiment. Three regimes emerge: when questions are scarce, the Q^{-1/2} term dominates and buffered ERM is optimal; in an intermediate regime, pooling generation counts across questions beats estimating each answer distribution separately, with only logarithmic gains from more questions; and with sufficiently many questions, the error reaches an N^{-β} identification floor that persists even with infinitely many questions.
The submodel's lower bounds also apply to general ensembles, matching the buffered-ERM upper bound in the question-scarce regime and parts of the intermediate regime. The authors note that open questions remain about the exact transitions between regimes and about constructing matching pooled estimators for general ensembles. Since this is a single source, there are no conflicting findings to compare.