Multimodal large language models (MLLMs) generate text based on inputs from multiple modalities, such as images and audio. A new paper on arXiv introduces "backdoor infection," a cross-modal attack framework that targets these models.
The framework demonstrates how an attacker could embed a backdoor in one modality that triggers malicious behavior when the model processes inputs from another modality. This cross-modal nature distinguishes it from traditional unimodal attacks.
The work underscores the growing security concerns around MLLMs as they become more integrated into applications. The authors propose this framework as a step toward understanding and mitigating such vulnerabilities.