Previous work on multilingual large language models has argued that language-specific features are encoded most strongly in the initial and final layers, implying that intermediate layers support language-agnostic representations. A new arXiv paper, 'Where Are Language-Specific Features Represented in Multilingual LLMs? Everywhere,' reconsiders that claim.
The title states the central finding: language-specific features are not localized to the edges of the network but are distributed across all layers. This directly challenges the earlier view and suggests that intermediate layers are not purely language-neutral.
If confirmed, the result matters for interpretability and multilingual transfer. It implies that language identity is a distributed property of the model, so interventions that aim to control or remove language-specific behavior may need to act across many layers rather than just the first and last.