Researchers propose GraphACM, a method for quantifying uncertainty in large language model (LLM) agents by measuring behavioral consistency across repeated executions. The work, posted on arXiv, argues that consistency across runs provides a useful signal for reliability, but notes that current approaches usually summarize that consistency with a single aggregate value.
GraphACM instead represents agent actions as a graph, allowing the consistency measurement to account for structural relationships between steps. This graph-based action consistency measurement is designed to offer a more informative uncertainty signal than a single scalar summary.
The paper is a new announcement on arXiv and does not include experimental results in the abstract. As a single source, it presents only the proposed method and motivation; no comparisons to other techniques are described.