Multi-agent debate can push large language models toward better reasoning, but when agents talk themselves into the same wrong answer, stopping at consensus simply commits the error. A new arXiv paper proposes Conformal Social Choice, a post-hoc layer that turns debate outputs into calibrated act-versus-escalate decisions. The method aggregates verbalized probability distributions from heterogeneous agents using a linear opinion pool, then calibrates the result with split conformal prediction.
This yields a fixed-round marginal coverage guarantee: at any pre-specified round, the correct answer is included in the candidate set with probability at least 1-alpha, without assuming individual models are well calibrated. A hierarchical action policy then maps singleton candidate sets to autonomous action and larger sets to human escalation. Across eight MMLU-Pro domains and two additional benchmarks, fixed-round coverage stays close to target.
The key result is not that debate becomes more accurate, but that failures become actionable: at the final round, 81.9% of wrong-consensus cases are flagged by non-singleton sets at alpha=0.05. Separately, cases that reach a singleton before the final round achieve 90.0-96.8% accuracy on MMLU-Pro, up to 22.1 percentage points above consensus stopping, but the authors stress this is a selection effect, not a reasoning improvement. Simple probability- and entropy-threshold rules intercept comparable cases; conformal's distinct value is the finite-sample coverage guarantee plus explicit candidate sets for escalation. The paper also notes that adaptive first-singleton stopping is evaluated empirically and is not covered by the guarantee.