The paper describes a post-training scenario where a language model produces several candidate solutions to the same problem, with only some being correct. Rather than discarding the incorrect ones, the authors propose using the full set of unverified trajectories as a learning signal.
The method, called shared residual distillation, appears to distill common residual patterns across these peer solutions. This could allow the model to improve by comparing its own outputs, reducing the need for labeled or externally verified data.
The abstract is brief, so the exact mechanism is not fully detailed here, but the core idea is clear: leverage unverified peers to create training signals through residual distillation.