Researchers evaluated GPT-6 Astra, a general-purpose multimodal model, for closed-loop robotic manipulation using a lightweight agent execution framework. They first probed three foundational embodied capabilities: depth perception, embodied in-context learning, and tactile understanding. The model demonstrated an ability to exploit depth information, acquire simple manipulation strategies from contextual demonstrations, and use tactile feedback to correct its actions.
Across five established manipulation benchmarks, GPT-6 Astra achieved 26.92% success on RoboDojo and 76.0% on RoboTwin 2.0. However, the authors report a pronounced gap between semantic reasoning and physical execution, particularly in consequence-aware action planning, precision, and complex motion coordination. To examine these limitations, they introduced LIBERO-GeoMotion, a benchmark of geometry-constrained tasks requiring precise pose alignment and coordinated translation and rotation; GPT-6 Astra succeeded only 11.1% of the time in zero-shot settings.
Real-world experiments corroborated the benchmark results. Zero-shot execution was feasible for simple tasks, but the model showed persistent deficits in motion feasibility, precise control, and physical safety, while substantial inference latency hindered practical deployment. The authors frame their findings as a characterization of emerging embodied capabilities and a delineation of the key challenges toward safe, efficient, and reliable robotic manipulation.