Relational databases (RDBs) are still the main way organizations store and manage structured data. Natural-language interfaces aim to let users ask questions directly, without needing to know SQL or database schemas. But evaluating such systems is tricky, especially when queries depend on the specific task or context.
The new benchmark, RDL-Bench, is designed for this challenge. According to its abstract on arXiv, it focuses on task-aware question answering over relational databases. The name suggests it tests not just whether a model can translate a question into a query, but whether it understands the underlying task the user wants to accomplish.
The abstract is brief, so details on the benchmark's construction, datasets, and evaluation metrics are not yet available. Still, RDL-Bench appears to be a step toward more realistic assessment of NL-to-RDB systems, which are increasingly important in both enterprise and scientific settings.