A new preprint on arXiv examines typed decision models (TDMs) as a possible low-cost substitute for LLM-based judges. Unlike generative models, TDMs output a probability for each allowed answer rather than producing free-form text, which could make them faster and cheaper to run in evaluation pipelines.
The authors position TDMs as candidates for replacing LLM judges that compare two answers. But the central open question, as the title suggests, is whether these models are also biased. The abstract notes that the work asks whether TDMs share the biases of LLM judges, though the full findings are not included in the available text.
Because the abstract is truncated, the results are not yet clear. The paper's contribution, at minimum, is to frame TDMs as a promising but unproven alternative in the ongoing effort to build reliable automated evaluation systems.