The arXiv abstract for ReTeach introduces a self-teaching approach that lets a model improve its own reasoning through multi-round reflection and retry. The central claim is that self-distillation can work without a separately trained, stronger teacher, but only if the self-teacher manages to gain some advantage over the student it is training.
The abstract notes that this advantage is the key variable. One way to create it is by conditioning the teacher on reference answers, though the text is cut off before explaining further. The paper's title and framing suggest that iterative retry and reflection are what allow the teacher to surpass the student's baseline performance.
Because the abstract is truncated, the full mechanism and experimental results are not available from this source alone. Still, the core idea is clear: self-improvement is possible, but its success hinges on carefully designing how the teacher becomes more knowledgeable than the student.