Machine learning benchmarks are more than evaluation tools; they shape where research effort goes. A new arXiv paper (2610.27533) points out that benchmarks also create incentives for building training datasets that nudge scores upward.
The authors introduce BenchDoor, a family of data poisoning attacks that exploits this incentive structure. Rather than targeting a model directly, the attack appears to work by poisoning the datasets that are created in response to benchmark pressure, turning the pursuit of higher scores into a backdoor entry.
The abstract is brief and does not give technical specifics about the attack mechanism or experimental results. Still, the core idea is notable: the same incentives that drive progress can be weaponized.