AI Safety via Debate proposes a way to oversee superintelligent systems: two AI models argue opposing positions while a judge—either a human or a weaker AI—decides which argument is correct. The approach is meant to make it easier to evaluate complex AI outputs without needing a more capable overseer.
According to the new arXiv study, however, the method's effectiveness depends on who is judging. Humans who watched the debates corrected their initial judgments, but weaker AI models used as judges did not show the same benefit. The title states the result plainly: humans correct their judgments through debate, while weaker AI models do not.
This finding highlights a potential gap in scalable oversight. If weaker AI judges cannot reliably improve through debate, then relying on them to supervise stronger systems may be unsafe. Human oversight might remain necessary, or alternative training and evaluation methods will need to be developed.