Training-data quality gates are often evaluated by comparing models trained on gated data against those trained on random subsets. But a new arXiv paper argues that this comparison is misleading: a gate simultaneously changes which examples are kept within each region of the input space and how the total training mass is spread across those regions. These two effects are entangled in any simple gate-versus-random test.
The authors propose a counterfactual gate-support decomposition that separates the selection effect—which specific examples survive—from the redistribution effect—how training weight shifts across regions. By isolating each mechanism, the method allows researchers to ask whether a gate improves performance by removing bad examples or by reallocating data toward certain parts of the input space.
The paper is presented as a methodological contribution, and the abstract does not yet report empirical results. As the sole source, it offers a conceptual framework rather than experimental findings, so its practical impact will depend on future applications to real filtering pipelines.