A new arXiv paper, "Radioactive Feedback: Tracing Models Trained by Label-Only LLM Supervision," investigates a specific form of knowledge distillation. Unlike conventional distillation, which transfers capability through teacher-generated demonstrations, the label-only variant relies solely on labels produced by an LLM acting as a judge. The authors describe this approach as distinct and increasingly practical.
The paper focuses on what happens when models are trained under this label-only supervision. The central idea, hinted at by the title, is that such training leaves a detectable mark—radioactive feedback—that can be used to trace whether a model was trained using this method.
Because the abstract is truncated, the full methodology and experimental results are not available in the source. Still, the framing suggests the work is aimed at model provenance and detection, rather than just improving distillation performance. The source does not compare this approach to other tracing methods, so those details remain open.