Robots perceive the world through a stream of observations, but not every change in that stream matters. An object sliding across a table, for instance, alters the visual input without altering the underlying task. According to a new arXiv paper, language-conditioned manipulation systems frequently fail to make this distinction, treating any shift in observations as a reason to replan.
The paper, 'Sequential Planning via Anchored Robotic Keypoints,' proposes a method that anchors planning to keypoints tied to the task rather than to the raw observation stream. By doing so, the robot can maintain a stable representation of what it is trying to accomplish even as the environment changes around it.
The abstract does not spell out the full mechanics of the approach, but the framing suggests a focus on separating task identity from transient perceptual details. The work is available on arXiv.