A team of researchers has introduced CLARO (CLutter-Aware Robot Orchestrator), a system that lets a robot follow natural-language instructions to find and retrieve objects in messy, real-world environments. Rather than training a single monolithic policy, CLARO uses a vision-language model (VLM) as an agent that coordinates a set of modular skills: open-vocabulary object detection, segmentation, 6-DoF grasping, and non-prehensile pushing. The agent keeps a memory of its own past actions, which allows it to work toward long-horizon goals and replan as the scene changes.

The system is designed for transferability: it emits grasp poses and 3D waypoints rather than robot-specific joint commands, so the same orchestration layer can be moved to different robot arms without task-specific training. In simulation, CLARO achieved an average success rate of 89.1%, and in high-clutter scenes it outperformed the strongest competing method by a relative margin of over 16%. When deployed on a physical robot arm across diverse cluttered scenes, it maintained a success rate of 88.8%, suggesting strong sim-to-real generalization.

The paper also breaks down performance along grounding and grasping axes to show where different methods succeed and fail, offering a clearer picture of the remaining bottlenecks. The work is accepted to WACV 2027 and appears on arXiv as a single-source preprint; no independent replication or commentary is included in the source material.