Detecting alignment failures in deployed language models typically relies on two kinds of tools: generative judges that spend a decoding pass on every criterion, and classifiers that read token probabilities, such as those based on Llama. The new arXiv paper "Just Ask Jev" proposes a different route, using reinforcement learning to make calibrated decisions about whether a model is exhibiting alignment failures.
Because the method is designed as a zero-shot detector, it aims to flag failures without being trained on specific failure criteria or requiring a separate decoding pass per check. The reinforcement-learning component is what produces the calibrated decision, according to the abstract.
The source abstract is brief and does not include experimental results or comparisons, so the practical effectiveness of "Just Ask Jev" is not yet detailed here. The contribution is the conceptual shift: framing alignment detection as a calibrated decision problem rather than a generation or classification task.