Self-evolving agents are usually scored by endpoint Gain, which compares held-out success after an update with success before. The problem, according to a new arXiv paper, is that this difference can credit a frozen rescore as evolution—meaning an agent might appear to improve when it has not actually learned anything new.
To address this, the authors introduce Trace Evolution Bench (TEB), a protocol designed to measure whether agents improve from their own environment traces. The abstract indicates that TEB offers a more direct test of whether the agent's own experience drives improvement, rather than relying on a simple before-and-after score.
The significance is that evaluation methods that conflate rescoring with learning could mislead progress in self-evolving systems. By focusing on trace-based improvement, TEB aims to give researchers a clearer signal of genuine adaptation, though the full details of the protocol are not spelled out in the abstract.