Vision-language-action (VLA) models are known to be sensitive to visual corruptions, but the authors of a new arXiv preprint argue that aggregate sensitivity is not enough. Knowing that a kind of corruption is usually harmful does not establish whether it is the culprit in a specific episode. They frame the problem as one of failure attribution: identifying which corruption, if removed, would actually change the model's output.
The proposed method takes a counterfactual approach, evaluating what would happen to the current episode's outcome if the corruption were absent. This allows for per-instance diagnosis instead of relying on broad vulnerability patterns. The abstract also begins to address situations where several corruptions appear together, though that part of the description is truncated.
The work is presented as a preprint and does not yet include experimental details in the available abstract. Still, the core idea offers a more precise way to debug and interpret VLA models in real-world settings, where visual disturbances are often mixed and context-dependent.