A paper posted on arXiv (2610.14879) highlights a blind spot in how social bias is measured in Chinese large language models. Most existing benchmarks use clean, template-based sentences that do not reflect the informal and often messy language found on social media.
The authors point out that internet slang, emoji, and sarcasm are largely left out of current evaluations. Because these expressions carry subtle and sometimes coded meanings, models might exhibit biased behavior in real-world online contexts that standardized benchmarks never surface.
The abstract notes that safety alignment may be impacted by this gap, though the full results are not included in the available excerpt. The paper underscores the need for benchmarks that incorporate the expressive, informal ways people actually communicate online.