Wasserstein distributionally robust optimization (WDRO) evaluates model robustness by taking the worst-case expectation over distributions inside a Wasserstein ball around a reference distribution. The paper proves a sharp limitation: ordering hypotheses by increasing WDRO risk is equivalent to ordering them by decreasing robust accuracy only when the loss function is a positive multiple of the zero-one loss plus a label-dependent offset.
For common surrogate losses such as cross-entropy, this equivalence breaks down. The authors show that a hypothesis with perfect robust accuracy can have a higher WDRO risk than another hypothesis with near-zero robust accuracy. In other words, WDRO risk can be a misleading proxy for robust accuracy when surrogate losses are used.
To evaluate robust accuracy directly, the authors introduce Wasserstein distributional attacks (WDA). Unlike existing methods that assign each data point's full mass to a single returned attack point, WDA adds a weight generation stage after candidate support search, optimizing the allocation of empirical mass under the Wasserstein constraint. They also introduce a support-search algorithm that finds nearby misclassified inputs.
On CIFAR-10, CIFAR-100, and ImageNet, WDA using their search algorithm achieves the lowest adversarial accuracy in most tested settings. The paper's code is available alongside the preprint.