On-policy distillation (OPD) trains a student model on its own rollouts, using token-level supervision from a teacher. The method rests on a critical assumption: the teacher can reach a correct answer from any prefix the student writes. The new arXiv paper identifies this assumption and names it, though the abstract cuts off before giving the term.
The authors argue that this assumption often fails in practice, leading to unreliable supervision that degrades student training. To address this, they introduce critique-guided on-policy distillation, a technique that recovers reliable supervision by incorporating critiques into the distillation process.
The abstract does not provide experimental details or quantitative results, so the practical gains remain unspecified. Still, the proposal directly targets a known weakness in OPD, making it a relevant contribution for researchers working on distillation methods.