A new arXiv paper, Same Attack, Different Judge, presents what it calls the first controlled adversarial comparison of three approaches to safety evaluation: TypeSafe AI's Jev, LLM judges, and purpose-built safety classifiers. The abstract frames this as a direct test of how each handles adversarial input, though the full methodology and results are not included in the available text.
According to the abstract, Jev works differently from typical judges. It does not generate text or a label. Instead, the developer defines questions in advance, and Jev returns a probability for each answer. This design is meant to give developers a more structured and interpretable signal than a free-form judgment.
The abstract also mentions that the vendor admits adversarial input can... and then the text cuts off. Because the source is a single truncated abstract, there are no other sources to compare or contrast, and the details of that admission remain unclear. Readers should consult the full paper for the complete findings.