Representation-based safety defenses for vision-language models typically learn to separate harmful inputs from benign ones. According to a new arXiv preprint, that separation is not enough: it does not establish what a risk score actually means or when it should be trusted.

The paper introduces counterfactual neighborhood risk fields, a method designed to ground risk scores in evidence. The idea appears to be that instead of relying solely on a decision boundary, the model should consider local counterfactual neighborhoods—inputs that are similar but would change the outcome—to give a more interpretable basis for safety judgments.

Because the abstract is truncated, the full details of the method are not available from the source. The preprint is a single source, so there are no conflicting findings to compare. The significance, as stated, is in moving from simple separation toward evidence-grounded selective safety.