Evaluating a fine-tuned AI coaching model typically requires expert graders, a process that is both slow and costly. An LLM judge could speed this up, but without ground truth its judgments are hard to trust. A new arXiv preprint tackles this problem directly, proposing a staged pipeline to screen candidate LLM judges and student models before committing to a full evaluation.
The pipeline is described in the context of an AI dental hygiene coach, where the authors fine-tuned a model and needed a scalable way to assess its outputs. Rather than relying on a single LLM judge, the staged approach filters candidates early, aiming to identify judges and student models that are reliable enough for downstream use. The abstract does not yet include results, so the effectiveness of the pipeline remains an open question.
Because this is a single preliminary source, there are no competing findings to compare. The main contribution at this stage is the framing: a practical, staged screening method for LLM-based evaluation when ground truth is unavailable. Future work will presumably show whether the approach holds up beyond the dental coaching domain.