In a new essay, Eugene Yan pushes back against the idea that an LLM-as-judge can rescue an AI product. He argues that the real lever is the development process itself: applying the scientific method, building with evaluation-driven practices, and monitoring outputs once the system is live.
Yan's central point is that evaluation should be a continuous discipline, not a final check. Teams should define what good looks like, test changes against those expectations, and treat failures as data for the next iteration. This shifts the focus from chasing a better judge model to improving how the team learns from its own system.
The essay also emphasizes production monitoring as a core part of the process. Because a model's behavior can change after deployment, ongoing observation is necessary to keep the product aligned with user needs. The source offers a single, coherent argument rather than contrasting viewpoints.