Both arXiv papers address the same family of methods: on-policy distillation, in which a student language model trains on its own rollouts while a frozen teacher provides dense token-level supervision. The two share a starting concern—that vanilla on-policy distillation leaves accuracy on the table—but they take different cuts at the problem.

The first paper, What Does Privileged Information Add to On-Policy Self-Distillation?, asks whether the teacher's effectiveness improves when it has access to an answer or a worked solution. The abstract notes that such extra information seems to give the student more to learn, and the paper appears to test how much of that promise is real. It focuses on the teacher's input.

The second paper, Verify Before You Distill, focuses instead on the teacher's output. It observes that vanilla on-policy distillation applies supervision uniformly across all prompts, without checking whether the teacher is reliable for each one. The authors propose prompt-level gating: verify the teacher on a prompt before allowing it to supervise the student.

So the two papers agree that naive on-policy distillation is suboptimal, but they diverge on where the fix should go. One enriches the teacher's information; the other selectively silences the teacher when it might mislead. Read together, they frame a design trade-off for distillation pipelines: whether to improve what the teacher knows or when it gets to speak.