Protein foundation models are increasingly used to generate novel biological sequences, but their reliance on large public datasets creates a new attack surface. According to a recent arXiv preprint, adversaries could manipulate the material these models learn from, either during pre-training or at retrieval time, potentially steering the generated proteins toward unintended or dangerous outcomes.
The threat is specific to the data pipeline: because these models are trained on publicly available sequence databases, an attacker who can inject or alter entries in those corpora could poison the model's behavior. The abstract highlights that this reliance "creates opportunities for adversarial manipulation," though the full details of the attack scenarios are not yet public.
This work underscores a growing concern in AI-driven biology: as generative models become more powerful, the integrity of their training data becomes a security issue. Researchers and practitioners may need to develop robust data curation and validation methods to protect against such poisoning attacks.