The paper, "Explaining Chess and Go in Natural Language," addresses a transparency problem: modern chess engines play beyond human ability, and the ideas behind their moves are often difficult for people to understand. That opacity makes it hard to assess whether an explanation of an engine's choice is accurate or useful.
The authors propose an automatic metric for evaluating chess explanations, grounded in the chess engine's own evaluation. Because the metric is automatic, it could in principle be applied without human judges, though the abstract does not detail the implementation.
The title indicates the work also covers Go, but the abstract provided focuses on the chess case. No further methodology or results are specified in the abstract.