A new preprint on arXiv (2610.27711) compares two fine-tuning paradigms for detecting hallucinations in large language models: classifier-head fine-tuning (CLS) and LLM-as-Judge sequence-to-sequence training (s2s). Under matched conditions, the authors report that the CLS paradigm consistently wins on binary hallucination detection tasks.
The study attributes this gap not to overall model capacity but to layer-wise representations. By tracing where and how hallucination-related information is encoded, the paper suggests that the fine-tuning objective strongly influences which layers carry useful signals for detection.
These findings highlight that the choice of training objective is not just a practical detail—it shapes the internal structure of the model and, ultimately, its reliability for hallucination detection. Further details on the exact mechanisms remain in the preprint.