On-policy distillation (OPD) has drawn attention as a potentially cheaper route than reinforcement learning, but it still depends on a teacher model to guide a student. A new arXiv paper steps back to examine OPD from the perspective of knowledge distillation, asking a basic question: where does the teacher's advantage actually live?
The abstract suggests that OPD is not a fundamentally new paradigm but a variant of knowledge distillation that relies on teacher supervision. By revisiting the method through that lens, the authors aim to demystify why the teacher helps, rather than simply treating OPD as a black-box alternative to RL.
The significance of the question is practical. If researchers can pinpoint the source of the teacher's advantage, they may be able to design more efficient distillation procedures or decide more clearly when OPD is worth using. The paper is positioned as a conceptual clarification, though the full details of the analysis are not yet available in the abstract.