Multimodal large language models (MLLMs) used for remote sensing scene classification typically make decisions in a single forward pass, picking from lettered class names in the prompt. A new arXiv preprint argues that this setup can produce "silent hallucinations"—confident but wrong labels that are hard to catch. The authors propose RSAJEV, a training-free abstaining prototype memory that lets the model hold off when it lacks confidence.
RSAJEV works by storing prototype representations of known classes and comparing the model's current input against them. If no prototype matches closely enough, the model abstains rather than forcing a choice. Because the method requires no retraining, it can be dropped into existing JEV-based pipelines. The authors report that this reduces silent hallucination while preserving accuracy on confident classifications.
The preprint is preliminary and has not yet been peer-reviewed. Still, the idea of adding an abstention mechanism to remote sensing MLLMs could be useful for safety-critical applications like disaster mapping or land-use monitoring, where a wrong label is worse than no label at all.