Vision-language models (VLMs) are becoming a central target for adversarial image research, according to a new arXiv preprint. The paper, titled "Answer It or Not? Towards Prompt-Adapted Adversarial Images on Vision Language Models," highlights that VLMs differ from traditional image classifiers because they also take text prompts as input.

The authors propose that adversarial images should be prompt-adapted—crafted with the specific text prompt in mind. This aligns with the prompt-driven nature of VLMs, where the same image can lead to different outputs depending on the accompanying question or instruction. The title's "Answer It or Not?" suggests the attack may influence whether the model produces an answer at all.

The abstract starts to describe a limitation of existing VLM approaches, but the sentence is cut off in the source. Since this is a single preprint, there are no other sources to compare; the results should be treated as preliminary.