Multilingual commonsense reasoning benchmarks assume that translated premises preserve the logical relations of their English originals. If a translation flips or collapses meaning, any cross-lingual comparison becomes unreliable. The paper argues that standard automatic translation scores, while useful for ranking candidate translations, are not sufficient to certify that a given premise retains its intended relation.
The authors introduce a source-conditioned diagnostic specifically aimed at a failure mode where opposite premises end up translating identically. By conditioning on the source text, the diagnostic can flag items where the translation process erases a distinction that the original English item depends on. This goes beyond a simple acceptance/rejection threshold for translation quality.
Such a diagnostic could help benchmark builders identify problematic items before they are used in multilingual evaluation. It also highlights a broader point: translation quality scores and semantic preservation are not the same thing, and reasoning tasks need checks that are sensitive to the specific logical structure of each item.