A long-standing debate asks whether lexical representations in language models are abstract—stripped of context—or holistic, bound to specific usages. A new arXiv paper, titled "Everyone Was Right: Lexical Representations in Language Models Are Both Abstract and Holistic," suggests the answer is not either/or but both.

The researchers investigate by decomposing contextual word representations, aiming to see how abstract and context-specific information are organized within the model. While the abstract does not detail the decomposition method or full results, the title signals that both competing accounts capture something real: lexical representations encode general meaning and situation-specific nuance at the same time.

If confirmed, this finding matters for interpretability and for practical uses of language models. It implies that probing for "true" word meaning may need to account for both stable semantic cores and flexible contextual shifts, rather than treating them as alternatives.