LLM agents are increasingly taking actions with real consequences, but safety evaluations often boil down to a binary safe/unsafe label on the final outcome. That approach misses whether the model ever noticed the risk and how it handled it along the way.
A new arXiv preprint, AURA-Eval, proposes an evaluation framework that examines the full trajectory of an agent's actions. The goal is to assess risk awareness in the process, not just the endpoint, so that failures and successes can be understood more precisely.
Because the abstract is brief, the paper's exact methodology and metrics are not detailed in the source. Still, the framing points to a growing concern: as agents act autonomously, safety evaluation needs to account for intermediate decisions, not only final results.