A new preprint on arXiv challenges a common assumption in AI safety evaluation: that guardrail classifiers detect something inherent in text. The authors argue that what gets labeled "unsafe" is always a product of a specific policy, and that policy enters the model through its training labels. As a result, a benchmark score may say more about which policy definition a model was trained on than about its ability to recognize harmful content in general.
The paper points to an accident in an industrial deployment as a natural experiment. While the abstract is truncated, the core claim is clear: when the policy shifts, the classifier's apparent performance shifts with it. This suggests that many safety benchmarks are not measuring capability in the abstract, but rather alignment with a particular labeling scheme.
The authors do not propose a new benchmark in the available text, but their framing implies that safety evaluations need to be explicit about the policy they encode. Without that, a high score on a safety benchmark could be mistaken for robust safety behavior when it is actually just a match between training labels and test labels. This is a single-source digest; no other studies are compared here.