Reinforcement learning with verifiable rewards (RLVR) has become a common approach for training math models, and the corpora used for this purpose are typically screened to remove examples that overlap with benchmark test sets. But a new preprint argues that the reliability of these decontamination policies, and the price paid in lost training data, have not been carefully studied.
The authors audit sixteen open RLVR math corpora, examining how cleaning policies perform in practice. The abstract, which is truncated, indicates a focus on both the trustworthiness of the screening methods and the cost of removing potentially useful data.
Because the full paper is not yet available, the specific findings and recommendations are not detailed in the abstract. Still, the framing points to a practical concern: ensuring benchmark integrity may come at the expense of training data volume, and the trade-off deserves closer attention.