Language models are frequently asked to judge whether an answer is correct, whether a statement follows logically, or whether a response is safe. However, the way instructions are phrased can shift these judgments, making evaluations unreliable. A new arXiv paper introduces a method to isolate a model's judgment in a dedicated subspace, without relying on labeled examples.
The approach is label-free, meaning it does not require manually annotated data to separate judgment from instruction-driven variation. By identifying a subspace that captures the model's stable judgment, the method could help researchers probe what models genuinely believe versus what they merely adopt from prompt wording.
The abstract is truncated, so full methodological details and experimental results are not yet available. Still, the proposal points toward more robust ways to measure model judgment across correctness, logic, and safety tasks—areas where instruction sensitivity is a known challenge.