Robots typically rely on tools for their designed purposes, but creative use of everyday objects could greatly expand what they can do. The paper opens with a pointed question: can a robot use a plate to cut a cake if no knife is available? This scenario highlights the need for robots to go beyond fixed tool–function mappings.
The challenge lies in open-world tool use, where objects may serve roles they were never designed for. The paper introduces GROW^2, which centers on grounding two aspects: which object to use as a tool, and where to apply it. By addressing both, the approach aims to make tool use more flexible and context-aware.
While the abstract does not detail the underlying method or experimental results, its framing suggests that solving the “which” and “where” problem is a key step toward enabling robots to improvise with available objects in unfamiliar settings. This could reduce the need for specialized equipment and improve robot adaptability in real-world tasks.