On-policy distillation (OPD) is a technique for transferring capabilities from a teacher model to a student model. It works by having the student generate rollouts and then using teacher feedback on those rollouts to guide learning. This approach has proven effective, but it comes with a computational burden: the student must produce complete rollouts, and the teacher must evaluate every one of them.

A new paper on arXiv, titled "Does On-Policy Distillation Need to Complete Every Rollout?," directly challenges this assumption. The abstract notes that generating complete student rollouts and computing teacher feedback for every step is a significant overhead. It then poses the central question: is this exhaustive process actually required, or can distillation succeed with a more selective approach?

The abstract is truncated, so the paper's full findings are not yet visible. However, the question itself points toward a possible optimization: if full rollouts are not strictly necessary, OPD could become substantially cheaper to run. The paper appears to be an early investigation into relaxing this requirement, though the specific method or results remain undisclosed in the available text.