Most factual knowledge benchmarks ask a model to produce one correct answer. But many real-world questions require an organized set of facts, such as a list of countries, symptoms, or historical events. The new ListQA benchmark addresses this gap by focusing specifically on list-formatted retrieval.
The paper introduces ListQA, a benchmark containing 9,045 questions designed to test whether large language models can retrieve complete and well-structured lists rather than isolated facts. By shifting the evaluation target, the authors aim to better reflect the demands of practical information-seeking tasks.
Because the source is a single arXiv abstract, no comparison with other benchmarks or reported results is available. The significance lies in the benchmark's design: it pushes factual knowledge evaluation toward a more complex and realistic format.