Scientific research increasingly depends on curating experimental data from published literature, but the process is error-prone because papers vary in format and detail. A new preprint introduces SciCurateBench, a benchmark designed to test how well AI agents can handle this task.
The benchmark focuses on verbal-action agents—systems that can take actions through language—and requires them to verify exact experimental conditions such as specific assays, clinical outcomes, and data availability. This goes beyond simple extraction, forcing agents to confirm that the data they pull is accurate and complete.
Because this is a single preprint, there are no other sources to compare against. Still, the work offers a standardized way to measure progress in automated data curation, which could help make scientific databases more reliable.