Evaluating generative image models remains a persistent challenge. According to a new arXiv paper, widely used metrics rely on fixed pretrained feature extractors that were optimized for other tasks, and these extractors do not always align with how humans perceive realism across different datasets. This mismatch can lead to evaluation results that feel out of step with human judgment.
The paper proposes saliency maps—visualizations that highlight which parts of an image a model attends to—as a possible tool for evaluation. The idea is that saliency maps may offer a more direct window into what a generative model is actually doing, potentially bridging the gap between automated metrics and human perception.
Because the abstract is truncated, the full methodology and experimental results are not available in the source. The paper's contribution appears to be conceptual at this stage, suggesting a new direction rather than presenting a fully validated metric. As with any single preprint, the claims have not yet been peer-reviewed or replicated by other groups.