Large language models used as conversational agents are increasingly expected to follow procedural logic—multi-step instructions, workflows, or reasoning chains. Factual hallucination, where models invent false information, has been widely studied. But a different failure mode, procedural hallucination, has received far less attention.

The arXiv preprint introduces DialFork, an evaluation approach that goes beyond factuality to assess procedural hallucination. It targets the ability of conversational agents to adhere to correct sequences and logical steps, rather than simply checking whether individual statements are true.

By drawing attention to this overlooked failure mode, DialFork suggests that evaluating conversational agents requires more than fact-checking. The work points to the need for benchmarks that capture whether models correctly execute procedures, not just whether they produce accurate facts.