A new paper on the arXiv preprint server examines how the wording of reference descriptions affects the retrieval of robot task instructions. The authors focus on RLBench, a standard manipulation benchmark, where natural-language instructions are paired with robot tasks. They find that identifying the correct task from a short instruction is highly sensitive to which descriptions are available for retrieval.

The study suggests that the same instruction can lead to different retrieval outcomes depending on how the candidate task descriptions are phrased. This is not simply a matter of the instruction itself, but of the relationship between the instruction and the set of reference texts.

This sensitivity has practical implications for designing robots that follow natural-language commands. If reference descriptions are poorly chosen, even a clear instruction may fail to retrieve the intended task. The paper points to the need for careful selection or generation of task descriptions in robot learning benchmarks and real-world systems.