Explaining a vision-language model's prediction usually means identifying which training examples were most influential. But existing attribution methods operate at the sample level, giving a single influence score per example. That loses the detail of which parts of an image or text actually mattered.
The new paper, posted on arXiv, introduces Cross-Modal Latent Influence, a method for patch-level training sample attribution. It aims to trace influence to the multimodal evidence that mediates a prediction, rather than treating each training example as an indivisible unit.
This finer granularity could help researchers understand how VLMs combine visual and textual cues, and which specific image regions drive a given output. The work, described in an arXiv preprint, points toward more precise interpretability tools for multimodal models.