Researchers have demonstrated a method for extracting training sequences from genomic language models using only the models' output. These models, trained on DNA sequences, can generate long nucleotide continuations from short prefixes. The study treats this generation capability as an attack surface, allowing an adversary to probe the model and recover fragments of its training data without any internal access.
The attack is described as output-only and black-box, meaning it relies solely on the model's generated responses rather than gradients, embeddings, or other hidden states. This makes the approach practical in real-world settings where models are accessed through APIs or other restricted interfaces. The authors present this as a direct route for studying training-sequence extraction, suggesting that the risk is not merely theoretical.
Because genomic data is highly sensitive and often linked to individuals, the ability to extract training sequences raises significant privacy concerns. The paper does not claim to have fully solved or mitigated the problem, but it underscores the need for safeguards in genomic language models, such as differential privacy or output filtering, before they are widely deployed. The source is a single arXiv preprint, so the findings have not yet undergone peer review.