Multimodal large language models (MLLMs) are increasingly used in socially sensitive settings, but their gender bias is hard to evaluate. According to a new arXiv paper, current benchmarks mostly look at bias in isolated contexts, which may miss how bias appears in real-world social relationships.

To address this gap, the researchers propose a benchmark that assesses gender bias from the perspective of social relationships. The work is described as an early step toward ensuring the trustworthiness of MLLMs in domains where social dynamics matter.

The abstract does not provide details on dataset size or results, but it frames the benchmark as a way to move from individuals to interactions.