Deep research agents are increasingly used for open-ended synthesis, but standard benchmarks often fail to capture how these systems behave in practice. A new arXiv preprint takes a different approach by probing self-consistency: what happens when the same agent is prompted repeatedly on the same task?
The study focuses on one commercial deep research system, examining whether the entity sets and citations it produces remain stable across runs. This is a practical concern, since users may rely on such agents to gather sources and synthesize information, and inconsistent outputs could undermine trust.
The abstract does not report specific results, but the framing highlights a gap in current evaluation methods. Closed-world QA benchmarks are poorly suited to characterizing open-ended performance, and the authors argue that consistency offers a useful lens for assessing these tools. The preprint is available on arXiv under identifier 2610.26421.