The paper addresses a core mismatch in AI systems: multi-agent discussions generate rich, collaborative reasoning, yet at deployment a smaller student model must answer independently. The authors propose a distillation method that transfers knowledge from these discussions into the student, but with a focus on the learner's needs rather than simply exposing it to every correct solution.
Full-graph distillation, the baseline approach, treats each correct solution as a teaching signal. The title "Overtaught, Underlearned" suggests that this exhaustive transfer can overwhelm the student, leading to poor learning. The new learner-centered distillation is designed to adapt the teaching process to what the student actually needs, potentially improving performance.
The abstract is truncated, so the specific mechanism is not fully described here. However, the motivation is clear: better alignment between how knowledge is taught during distillation and how the student will be used at inference time. This work is relevant to anyone building compact models from larger collaborative systems.