Large language models (LLMs) continue to be susceptible to jailbreak attacks even after extensive safety alignment. According to a new paper on arXiv, the core issue is that existing defense mechanisms have not fully deleted harmful knowledge from the models, leaving pathways for attackers to exploit.

The paper proposes a method called constrained knowledge unlearning, which aims to remove harmful knowledge more completely while preserving the model's overall safety alignment. This approach differs from prior defenses that focus on blocking attacks at inference time or adding safety layers on top of the model.

By targeting the knowledge itself rather than just the model's behavior, the method could address the root cause of jailbreak vulnerabilities. The abstract suggests this is a promising direction, though full details of the technique and its evaluation are not included in the available summary.