Multimodal large language models have made rapid progress on visual reasoning, but a new arXiv preprint argues that many real-world tasks require evidence that is difficult to obtain through internal reasoning alone. The paper, "OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory," reports a controlled study across 22 benchmarks that points to this limitation.
The authors propose OpenVLHarness as a solution: a framework that lets visual agents use external tools and maintain multimodal memory while solving tasks. The abstract does not give specific benchmark numbers, and because there is only one source, there is no independent comparison to note. The core claim is that robust visual agentic performance depends on more than the model's internal reasoning.