Text-to-SQL benchmarks usually hand an agent the target database, reducing the task to writing the right query. In real-world data catalogs, however, the database is not specified in advance. An agent must first determine which sources are relevant to a request, then figure out how results from different databases fit together.

DSABench is a new benchmark designed for this harder setting. It evaluates multi-database agents on three capabilities: discovering the sources needed for a request, coordinating their results, and executing the overall task. The benchmark reflects the practical problem of working with large, distributed data catalogs where choosing the sources is part of the challenge.

By focusing on source discovery and coordination, DSABench highlights a gap in current evaluation methods. It suggests that future text-to-SQL systems should be assessed not only on query accuracy but also on their ability to navigate and combine data across many potential sources.