Retrieval-augmented generation (RAG) systems pull relevant passages from a large corpus to ground model outputs. A common defense against poisoning is to check consistency between the retrieved passages and the generated answer. But a new arXiv paper argues that this defense rests on a fragile assumption: that an adversary can only tamper with a negligible fraction of the corpus.
The authors show that a sparse adversary can break this assumption. Even if only a tiny fraction of passages are modified, a coordinated poisoning campaign can make retrieval concentrate on exactly those passages, so the poisoned content dominates what the model sees. In that case, consistency checks may pass because the model's output aligns with the poisoned retrieved text.
To counter this, the paper proposes detecting coordinated retrieval poisoning using embedding geometry. Rather than relying on consistency alone, the method looks at the geometric structure of retrieved embeddings to identify when passages have been clustered or positioned to manipulate retrieval. The abstract does not include experimental results, so the effectiveness of the approach remains to be evaluated, but the framing highlights an overlooked attack surface in RAG pipelines.