The arXiv paper introduces RefCraft, a benchmark for visual-reference-conditioned agentic generation. The authors argue that, beyond basic tool use and coding, the ability to incorporate visual references into generated output is a crucial capability for AI agents in real-world productivity settings.
RefCraft approaches this through web development, a domain where visual fidelity matters. The benchmark is designed to test how well agents can follow visual references while building or editing web pages, rather than relying solely on textual instructions or code-level reasoning.
The paper's framing suggests that current evaluations may overemphasize tool use and coding proficiency, leaving a gap in measuring how agents handle visual context. RefCraft aims to fill that gap, though the abstract provides limited detail on the specific tasks or evaluation methodology.