Resource-exhaustion attacks against autoregressive vision-language models (VLMs) have usually assumed a unimodal threat model: the image branch is the main optimization surface, while the user-visible prompt stays unchanged. That assumption, according to a new arXiv paper, may overlook a richer attack space where text and image are manipulated in concert.
The paper, Multimodal Resource-Exhaustion Attacks on Vision-Language Models via Joint Pixel-Prompt Optimization, describes a method that optimizes image pixels and the text prompt jointly. Rather than holding the prompt constant, the attack varies both modalities to trigger excessive computational load or other resource exhaustion. The abstract frames this as a departure from standard unimodal treatments, but does not yet include experimental results.
Because the source is an abstract, concrete numbers and benchmark outcomes are not available. The core contribution is conceptual: by treating the prompt as a variable instead of a fixed input, the attack surface widens. This points to a potential gap in how VLM threats are modeled, and suggests that defenses may need to account for coordinated adversarial manipulation across multiple input modalities.