Generative large language models are known to exhibit harmful biases and stereotypes, but most safety work has focused on English. A new paper introduces MBBQ, a dataset built specifically for cross-lingual comparison of stereotypes in generative LLMs.
The motivation is practical: safety fine-tuning usually happens in English, if it happens at all, yet these models are used by speakers of many different languages. MBBQ is meant to help researchers see whether biased behavior shifts or persists when the same kinds of questions are asked in other languages.
Because the abstract is the only available source here, the article cannot describe specific languages, benchmark results, or methodology details. The key contribution, as stated, is a dataset that enables direct comparison of stereotype-related behavior across languages rather than assuming English-only evaluations are sufficient.