Direct Preference Optimization (DPO) has become a standard approach for aligning vision-language models (VLMs) with human preferences. However, a new paper on arXiv argues that the security of the multimodal preference data used in this process has received little attention.
The paper, titled "DPOison-VL: Clean Preference Poisoning in Vision-Language Direct Preference Optimization," introduces a new attack method. The abstract, though truncated, indicates that existing preference poisoning attacks are not directly applicable to this setting, motivating the need for a novel approach.
The work underscores a potential vulnerability: if an attacker can manipulate preference pairs, they may be able to influence a model's alignment behavior. The authors call for greater scrutiny of multimodal preference data security, though full details of the method and results are not available in the abstract alone.