A recent arXiv paper examines a previously overlooked vulnerability in large vision-language models (LVLMs). These models are commonly used with text-only prompts or with an image accompanying the text. The researchers found that simply adding an image can unexpectedly alter the model's output, undermining the stability of its responses.

The effect is specifically observed in attribute-based descriptions—statements about properties or characteristics of objects. According to the abstract, the framing of the visual input also plays a role. The paper, titled "Visual Input and Its Framing Affect Undermine the Stability of Attribute-based Descriptions Produced by Large Vision-Language Models," suggests that this instability is a new aspect of LVLM vulnerability.

The findings imply that users cannot assume consistent behavior from LVLMs when switching between text-only and multimodal prompts. The authors do not propose a fix in the available abstract, but the work highlights a practical concern for applications relying on stable attribute descriptions.