Generative policies have made it easier to synthesize robot behaviors, but specifying exactly where, when, and how an action should occur remains awkward. The authors of this paper argue that diverse control references—keypoints, optical flows, orientations, keyposes—are all partial views of the same temporal Cartesian action space. That observation leads to a unified interface called MAP (Masked Action Space), which frames controllable manipulation as an action-inpainting problem.

Under this interface, an Inpainting Policy reconstructs missing end-effector action dimensions. The training scheme combines progress-guided alignment with multi-mask training, allowing the policy to handle combinations of constraints it has not seen before. A training-free RePaint variant iteratively refines the action during denoising, which the authors say reduces control error.

Experiments on ManiSkill and LIBERO-10 report average task success rates of 92.67% and 64.67%, respectively. In real-world tests, the policy achieved average positional and rotational errors of 44.3 mm and 7°. The work suggests that a single masking interface can let users guide robot behavior through a wide range of reference types without retraining.