Automatic prompt generation is essential for general-purpose multi-agent systems to perform a wide range of tasks without human intervention. According to a new arXiv paper (arXiv:2505.13546), current methods for evaluating these prompts typically look only at how well the prompt performs on the immediate task at hand.
The paper, titled "Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systems," argues that this narrow focus is insufficient. It proposes that prompt stability—how consistently a prompt performs across variations or repeated use—should also be considered when evaluating and optimizing auto-generated prompts.
Because the abstract is truncated, the full details of the proposed stability metric and optimization approach are not available from this source alone. However, the core claim is clear: stability is a missing dimension in current prompt evaluation, and addressing it could improve the reliability of autonomous multi-agent systems.