Vision-language-action (VLA) policies are usually tested under one perturbation at a time, but real-world deployments often involve several shifts at once. The authors of a new preprint ask whether compound robustness can be inferred from single-axis evaluations, and introduce LIBERO-CTRL, a six-axis benchmark that pairs each initial state across single-axis conditions with a matched simultaneous condition.

The paired design reveals two opposing outcome changes that aggregate success rates cannot distinguish: emergent failures, where all single-axis rollouts succeed but the simultaneous rollout fails, and compensated successes, where at least one single-axis rollout fails but the simultaneous rollout succeeds. Because one transition decreases compound success while the other increases it, they can cancel, making aggregate compound performance appear consistent with single-axis measurements even when individual outcomes differ substantially.

Across six policies and three severity levels, outcome changes reached 34.5% in the most affected condition. Even when the difference between the two transition rates was not statistically distinguishable from zero, as many as 29.0% of matched initial states still changed outcome. The relative prevalence of the two transitions varied across policies and severities, while transition rates remained similar under independent re-evaluation of stochastic policies. The authors conclude that compound robustness cannot be characterized from aggregate single-axis success rates alone; matched per-instance evaluation is needed to reveal how joint perturbations alter behavior.