Large vision-language models (VLMs) are increasingly used as the perceptual backbone for embodied agents, yet their fundamental vision-language grounding ability in embodied scenarios has remained largely untested. A new preprint introduces EPIC-Bench, a fine-grained benchmark designed to systematically evaluate this capability in real-world environments.
The benchmark comprises 6,600 carefully human-annotated tuples of image, text, and mask, spanning three core stages of the embodied interaction pipeline: target localization, navigation, and manipulation. It includes 23 fine-grained tasks and multiple settings, allowing for a detailed diagnostic assessment of model performance.
The authors evaluated over 80 leading VLMs on EPIC-Bench, providing a broad comparative analysis. The results offer a systematic view of how well current models connect visual perception with language instructions in embodied contexts, highlighting strengths and weaknesses across different task types.