Robots that learn from past experience need to retrieve the right context: the task, the current stage of execution, and the action that should follow. Existing retrieval models often ignore one or more of these. Robota-Embed, introduced by a team of researchers, is described as the first vision-language-action (VLA) embedding model to jointly encode all three modalities in both current-state queries and recorded future continuations.
The model was trained on a multi-source corpus of 1,695,684 cleaned anchors. On the new Robota-Embed-Bench benchmark, which contains 4,000 queries and an 8,000-entry gallery, it achieves 47.33% clip-level and 81.77% episode-level recall at rank 1, outperforming action-free Gemini Embedding 2 by 22.09 and 33.47 percentage points, respectively. Controlled studies showed that removing native action information degrades retrieval more than removing vision, highlighting the value of action-aware embeddings.
The authors demonstrate two applications. Embed2ICL uses retrieved experience for policy conditioning, raising success from 47.0% to 56.3% in DROID-Sim and from 86.4% to 92.8% on the LIBERO difficult subset, while also improving task progress on real robots. Embed2Evolve uses execution feedback to grow task-local memory around a frozen GPT-6 Astra policy. Across five ManiSkill tasks, it achieves 68% success, compared with 24% for GPT-6 Astra Direct and 33% for a harness independently evolved with Omni-Embed-Nemotron-3B under the same protocol, while using approximately 79.5% fewer policy tokens per test rollout.
These results suggest that a unified, action-aware retrieval space can support both in-context learning and task-local context evolution for robots, making past experience more useful at inference time.