According to an arXiv abstract, the authors of TasteBench argue that expert-aligned “taste” benchmarks should reward craft judgment, not the ability to game the test. The proposed benchmark is built around pairwise good-vs-bad comparisons across visual, brand, book, and tech items, all evaluated under a frozen rubric.

Because the rubric is frozen, the benchmark is intended to measure consistent alignment with expert taste rather than adaptive test-taking. The abstract does not provide details on the rubric's contents or evaluation results, so the claims are limited to the design rationale.