A new arXiv paper, titled "The Code of Theseus," probes a growing mismatch in AI-assisted software development. LLM agents are increasingly used to design a system before writing any code, yet the datasets used to train and evaluate these agents are heavily weighted toward implementation and testing. Requirements and design documents—the very artifacts that guide early decisions—are frequently absent, stale, or kept private.

The authors appear to test whether agents can reconstruct software from these recovered design documents alone. The title evokes the ship of Theseus, asking whether the rebuilt code retains the identity of the original system when only its design blueprints remain. This is not just an academic exercise: if agents can work from partial or outdated design specs, they could be deployed in real-world codebases where documentation lags behind the code.

The significance lies in the direction of the field. As agents take on more design responsibility, evaluation must shift from "did the code pass tests?" to "did the agent follow the intended design?" The paper highlights a missing piece in current benchmarks and suggests a path toward more design-aware training and evaluation.