Large language models are now commonly used as judges to evaluate text produced by other LLMs. But the question of what such a judge actually needs to see remains unresolved, according to a new paper on arXiv.
The paper, titled "The Anatomy of a Blind Spot," treats the judge as a regulator whose reports are acted on. This framing suggests the study is concerned with the practical consequences of incomplete or misleading information given to an LLM judge.
The abstract outlines the problem but does not yet provide specific findings or conclusions in the available text. It sets up the core issue: without knowing what a judge must be shown, evaluations may be unreliable or biased.