Robotic manipulation policies typically need to reason about both the current scene and how it will change when the robot acts. The authors of this paper argue that existing methods either feed current geometric features directly into the policy or treat future prediction as a side task, without clarifying how predictive geometry should be represented for action generation. They propose a two-stage framework that learns two complementary representations from multi-view video clips.
First, a frozen 3D foundation model produces dense geometric features, which are compressed into predictive geometric representations that preserve scene structure and its temporal evolution. Then, action supervision is applied to the geometric transitions, yielding a separate motor representation that emphasizes action-relevant changes. During downstream training, the predictive geometry is transferred via geometry queries, while the motor representation guides the intermediate features along the action-generation pathway. Future observations are used only as privileged supervision during training and are not needed at inference time.
The authors evaluate the approach on LIBERO, LIBERO-Plus, SimplerEnv, and real-world bimanual and single-arm tasks. They report consistent improvements over downstream policies, including an absolute gain of 10.4 percentage points on the WidowX evaluation, and show transfer across multiple vision-language-action architectures and whole-arm manipulation systems. The code is planned to be released.