On-policy distillation is a technique that gives a student model dense, token-level feedback while it generates its own rollouts. This supervision is meant to guide the student more closely than static, off-policy signals. But the approach has a known weakness: when the student and teacher drift apart, the feedback can become misleading or unreliable.
The new paper introduces DTDD, or Divergence-Triggered Dynamic Distillation, which directly targets this failure mode. Instead of applying distillation uniformly, DTDD monitors the divergence between student and teacher and activates dynamic adjustments when that divergence occurs. The authors identify two common causes of such divergence: a large initial gap between the policies and errors that compound over time.
By tying the distillation process to measured divergence, the method aims to keep supervision reliable even as the student evolves. The abstract does not provide experimental details, so the reported gains are not yet clear, but the conceptual shift—from fixed distillation to a divergence-aware trigger—addresses a core limitation in on-policy training. Further reading of the full paper would be needed to assess how well the mechanism works in practice.