Multilingual models share a fixed pool of capacity across all languages they support. As a result, adding a new language can degrade performance on languages the model already knows. This trade-off is a central challenge for scaling multilingual systems.
The paper "Scaling Multilingual Mixture-of-Experts: How Language Coverage Changes Capacity and Compute Allocation" (arXiv:2610.20247) proposes mixture-of-experts as a way to ease this tension. MoE models add capacity without proportionally increasing per-token computation, which could allow broader language coverage at lower cost.
The abstract indicates the study focuses on how language coverage alters both capacity and compute allocation. However, the source is truncated, so specific results and conclusions are not available in the provided text. This digest is based solely on the partial abstract.