On-policy distillation (OPD) is a training setup in which an agent learns from its own trajectories while being supervised by a stronger teacher. It is used in multi-turn agentic tasks, where the agent must make a sequence of decisions.
The new arXiv preprint reports a counterintuitive result: a smaller, initially weaker student can end up outperforming the teacher. The authors call this 'distillation overshoot' and argue that it deserves closer study rather than being treated as an anomaly.
The paper's contribution is to characterise this overshoot and to explore how it can be preserved. If the effect holds broadly, it would mean that distillation is not just a way to compress a strong model into a smaller one, but a route to capabilities that exceed the teacher's own.