On-policy distillation (OPD) is a technique for transferring knowledge from a teacher model to a student model during training. A key challenge, according to the abstract of a new arXiv paper, is extracting effective supervision from teacher knowledge.

The abstract notes that two common supervision strategies have limitations. Supervision from token probabilities may underutilize the teacher's internal knowledge, while direct hidden-state matching forces the student to align with teacher representations. The abstract cuts off mid-sentence at this point, so the exact drawback of hidden-state matching is not fully stated in the available text.

The paper proposes a method called SimOPD, which the title describes as "relational knowledge transfer" for on-policy distillation. This suggests the approach captures relationships within the teacher's knowledge rather than matching individual outputs directly. However, because the abstract is truncated, the precise mechanism is not fully described in the released version.