Formulating optimization problems from raw inputs such as problem descriptions and data is time-consuming and costly. Large language models (LLMs) offer a promising way to automate this process, but evaluating their capability requires a dedicated benchmark. The new arXiv paper introduces NLData2Opt, a benchmark designed specifically for parameter derivation and prediction in this setting.

According to the abstract, NLData2Opt focuses on the step where raw inputs are converted into the parameters needed to model an optimization problem. This is a critical but often overlooked part of the pipeline, and the benchmark appears intended to measure how well LLMs handle it.

The abstract also states that existing benchmarks overestimate something, but the text cuts off at that point. As a result, the exact nature of that overestimation is not available from the source. The paper is listed as a new arXiv announcement, and further details would require the full manuscript.