On-policy distillation typically relies on a teacher model to guide a student model during training. But a new arXiv paper highlights a surprising failure: when the teacher takes over the student's learning process, the combined system can produce incorrect responses even though both models independently solve the same problem correctly. This suggests that the interaction between the two models introduces errors that neither exhibits in isolation.
To measure this, the authors define Collaboration Failure Rate (CFR), a metric that captures how often the teacher-student collaboration goes wrong despite individual correctness. The paper then introduces CD-OPD, a method that uses a "contextual digest" to improve on-policy distillation and reduce these collaborative failures. The abstract does not provide full experimental details, but the framing points to a previously underappreciated risk in distillation pipelines.