Visual encoding models are designed to predict how the brain responds to images, and they have traditionally been optimized for one goal: accurately matching observed brain data. According to a new arXiv preprint, however, that narrow objective may leave a gap between what the models learn and what individual humans actually perceive.
The abstract argues that while these models perform well at predicting activation patterns, their internal representations may not reflect individual human perceptual similarity. In other words, two images that a person sees as alike might not be represented as close in the model's feature space, even if the model's brain-response predictions are accurate.
The authors propose that aligning encoding models with perceptual similarity judgments could be an important step, though the full details of their method and results are not available in the abstract. The work raises a broader question: should brain-encoding models be judged not only by how well they predict neural data, but also by how well they capture subjective experience?