Model merging has become a common way to improve task-specific performance in large language models without additional training. But a new paper on arXiv highlights a potential security risk: a backdoor that is triggered only after merging. The authors describe this as a "merging-conditional backdoor unalignment" attack.

The key idea is that a specialized model from a third-party platform might pass a safety audit on its own, yet carry a hidden backdoor that activates when it is merged with another model. This could allow malicious behavior to emerge only after the merge, bypassing standard safety checks.

The paper is described as a step "towards" this attack, meaning it is likely a proof-of-concept or theoretical framework. The abstract does not provide details on the method or results, but the threat is clear: model merging, while useful, may open a new avenue for compromising LLM safety.