Large language models are increasingly being used to generate Verus contracts for Rust programs, along with the proofs that establish them. The standard measure of success in this area is verifier acceptance: whether the generated proof is accepted by the formal verification tool. However, as a new paper points out, this metric is limited.

The abstract of the paper, posted on arXiv, notes that verifier acceptance "shows only that the…" — suggesting that acceptance alone does not guarantee the specification is correct, complete, or meaningful. This gap motivates the need for evaluation methods that look beyond whether the verifier says yes.

To address this, the authors introduce VerusEval, a framework designed to evaluate LLM-generated Verus specifications more rigorously. While the full details are not yet available from the abstract, the goal is clear: move past a binary pass/fail signal and toward a deeper assessment of the quality and utility of AI-generated formal specifications for Rust.