Self-improving agents can be built by letting a policy model and a growing skill bank refine each other during training. This skill-policy co-evolution is seen as a promising paradigm, since the policy learns to use new skills while the skill bank is shaped by what the policy can handle.
A new preprint warns that this process can fall into a trap. The title, "Evolving Together, Failing Apart," captures the risk: the policy and skill bank may become tightly coupled in a way that looks successful under standard evaluation but actually masks brittleness.
The abstract points to a specific flaw in how such agents are assessed. Current evaluations score the co-evolved policy on the final skill bank, which may not reveal whether the co-evolution process itself was healthy. The paper suggests that evaluation protocols need to look beyond the endpoint and examine the dynamics of skill-policy co-evolution directly.