According to a new paper on arXiv, a benign image paired with a benign caption can become harmful only in combination. This makes detection fundamentally a cross-modal reasoning problem, rather than a straightforward classification task. The authors note that existing multimodal harm benchmarks are limited because they evaluate only classification, leaving out the kind of reasoning needed to understand implicit harm.

The paper's title points to two key concerns: diagnosis and generalization. That is, how can models be diagnosed for their ability to reason about implicit harm, and can that reasoning generalize beyond the specific examples used in evaluation? By highlighting this gap, the work suggests that current safety tests may be insufficient—models could pass classification-style benchmarks while still failing to recognize harm that emerges from the interaction of image and text.