Researchers have introduced CSWBench, a benchmark designed to evaluate context-sensitive code-switching in creative writing. The paper, posted on arXiv, addresses a limitation in how large language models are assessed for multilingual ability.
The abstract argues that effective code-switching requires more than multilingual proficiency. Models must decide when to switch languages, a context-sensitive skill that goes beyond simply generating text in multiple languages.
By focusing on creative writing, the benchmark targets a domain where stylistic and contextual choices are central. CSWBench could help reveal whether LLMs can make appropriate language-switching decisions in nuanced settings, rather than only demonstrating raw multilingual fluency.