When evaluators assign step-level rewards to an AI agent's actions, those rewards are often reused across different evaluation channels. That reuse raises a measurement question: does the assigned direction of the reward remain stable? A new paper from arXiv introduces Agent Step Value (ASV) as a way to audit this sensitivity.
ASV is defined as a channel-indexed before/after target-margin gain. In other words, it measures how much the margin to a target changes around a step, with the result indexed by the evaluation channel. This allows researchers to compare whether the step's contribution is consistent across channels, rather than assuming it is.
The paper's contribution is a measurement framework for auditing sensitivity in black-box settings, where internal details of the agent are not available. By making the stability of step rewards explicit, ASV offers a concrete way to check whether evaluator-derived signals can be safely reused across channels.