A new paper on arXiv (2610.14550) addresses the challenge of evaluating Dynamic Product Advertising (DPA) retrieval systems, which are a core component of modern e-commerce advertising. The authors argue that conventional offline engagement metrics are insufficient because they are sparse and biased.

The paper proposes using large language models (LLMs) to evaluate user-product relevance directly. This approach is intended to provide a more robust signal than engagement-based metrics, which often suffer from data limitations and systematic biases.

Because only the abstract is available, specific methodological details and experimental results are not yet accessible. The work is positioned as a response to a known evaluation gap in DPA systems, but further reading of the full paper would be needed to assess its contributions.