Reinforcement learning typically benefits from verifiable rewards—signals that can be checked against ground truth. A new arXiv preprint (2610.20898) examines a cheaper alternative: using the model's own samples and taking a majority vote as the label. In many cases, this self-generated label recovers most of the performance gain of verifiable rewards, which is surprising if you look only at whether the vote is correct for each prompt.
From the usual prompt-level view, majority votes are often wrong, so it is unclear why training with them works so well. The authors argue the answer lies in gradients. Even when the majority label is incorrect at the prompt level, the gradient signal can still point in a direction that tracks what verifiable rewards would provide. That distinction—wrong votes, right gradients—explains the empirical success.
Because this is a single preprint and the abstract is partial, this summary does not compare findings with other sources. The proposed mechanism is theoretical and would need further validation.