Vision-language-action (VLA) models are typically fine-tuned and tested on a fixed set of instruction strings, which may hide how sensitive they are to language variation. A new arXiv paper (2610.19438) examines this by rewording instructions for models fine-tuned on the LIBERO benchmark.
Using base OpenVLA, the authors find that simply rephrasing an instruction changes task success by -38.0 points. That is, the model's performance collapses when it sees a semantically equivalent but differently worded command. The paper argues that this kind of 'paraphrase consensus' is not the same as self-consistency, and that instruction reliance is a robustness problem.
Test-time prompt ensembling is mentioned as a possible mitigation, though the abstract does not give full results. The study highlights the need to evaluate VLA policies on varied instructions rather than a single fixed set.