Policy-shift distillation is a technique for transferring knowledge from a teacher policy that has been refined by reinforcement learning (RL) to a student. It works by using the difference in the teacher's log probabilities before and after RL as supervision on trajectories generated by the student.

According to the abstract, this endpoint policy shift cannot reveal something important, though the full description is not provided. The paper introduces Waypoint-OPD, a gated on-policy distillation method, as a proposed solution to this limitation. The abstract does not give further details on the mechanism or experimental results.