Large language model agents have reached the point where they can run causal discovery algorithms on a dataset and report the output. But whether they do anything more than that—such as recognising when an algorithm has failed and designing follow-up tests to fix it—remains unclear. A new arXiv preprint takes up this question directly.
The work, titled "Benchmarking and Enhancing Agentic Causal Discovery," proposes a benchmark to evaluate these capabilities and explores ways to improve them. The abstract highlights the difficulty of telling from a simple output whether an agent has truly engaged with the underlying causal problem or merely executed a routine procedure.
By focusing on the gap between running algorithms and actively debugging them, the paper points to a key limitation in current agentic systems. It also suggests that enhancing agents to detect and correct errors could be a meaningful step toward more reliable automated causal analysis.