Object hallucination in vision-language models (VLMs) is typically measured by checking each object a caption names against the image. According to a new arXiv paper, most benchmarks and detectors alike treat every object mention as a claim that the object is present in the image.

The paper, titled "Not Every Mention Is a Claim," argues that this assumption is flawed. Mentions can occur in negations, hypotheticals, or comparisons, where the object is named but not asserted to exist. The authors suggest that such cases are often misclassified as hallucinations.

By rethinking what counts as a claim, the paper aims to push toward more accurate hallucination detection. The full methodology and experimental results are not detailed in the abstract, but the core argument is clear: current evaluation practices may be overcounting errors.