Fine-tuning a large language model on a downstream task can rapidly erode its safety alignment, according to a new arXiv preprint. The authors describe this as a substantial risk when aligned models are adapted for specific uses, because even a small amount of malicious training data can undo months of safety work.
The paper argues that current alignment-stage defenses are limited: they depend on harmful examples during training. The proposed alternative, called joint weight perturbations, appears to make the model robust by collectively perturbing weights at the alignment stage, rather than relying on exposure to specific harmful inputs.
Because the abstract is truncated, the exact mechanics of the perturbation method are not fully described in the source. The preprint is the only source for this digest, so no independent comparison is possible. What is clear is that the authors position their work as a response to the failure of existing defenses under collective attack.