LLM watermarking has emerged as a way to detect downstream model misuse, but most work assumes a centralized training setup. A new arXiv paper, "Watermarking for Byzantine Robust LLM Training," looks at what happens when that assumption is dropped and training moves to federated learning.
The authors note that extending watermarking to federated learning presents unique challenges. In particular, federated systems often rely on mechanisms that are not present in centralized pipelines, and the paper appears to focus on Byzantine robustness—the problem of malicious or faulty clients disrupting training.
Because the abstract is truncated, the full details of the proposed approach are not available, but the core message is clear: watermarking cannot be transplanted directly from centralized to federated settings. The paper highlights the need for new methods that account for the distributed, adversarial nature of federated training.