Traditional image-computable saliency models have shaped perceptual psychology and computer science, but they are limited to low-level features and miss the high-level cognitive factors that drive human attention and eye movements. This is the problem the new arXiv paper addresses.
The paper, titled "Semantic Saliency from Multi-Modal Large Language Model Scene Understanding Maps," proposes a new approach. According to the abstract, it uses multi-modal large language models to generate scene understanding maps that serve as the basis for semantic saliency. This moves beyond pixel-level features to incorporate a model of the scene's meaning.
The significance is that such semantic saliency could provide a better account of where people look, bridging the gap between low-level visual features and high-level cognition. The abstract does not yet detail experimental results, but the direction suggests a new role for LLMs in perceptual modeling.