A new paper on arXiv highlights position bias as a persistent and critical vulnerability in large language models. The authors describe it as a weakness that can compromise both the integrity of multiple-choice evaluations and the reliability of retrieval-augmented generation (RAG).

Position bias refers to the tendency of models to favor certain positions in input sequences, such as answer choices in a multiple-choice question or documents in a retrieved set. According to the study, this bias can lead to incorrect conclusions in evaluation settings and unreliable outputs in RAG systems.

The paper is described as a comprehensive study of the issue, though the abstract does not detail specific methods or mitigation strategies. As a single source, it adds to a growing body of evidence that position bias remains a significant challenge for LLM reliability.