A new preprint on arXiv argues that a language model's vocabulary is its native semantics: the embedding and unembedding layers fix the meaning of a state at the two ends of the residual stream. The authors propose using this vocabulary as a lexical reference frame for interpreting what happens in between.
The abstract contrasts this with existing interpretability methods, which it says either reuse a fixed lens or treat the residual stream in a way that does not align with the model's own semantics. By anchoring analysis in the model's native vocabulary, the paper suggests, intermediate states can be read more directly in terms of the model's own meaning-making.
Because only the abstract is available, the digest is limited to the conceptual framing; no experimental results or comparisons are reported here. The preprint is a single source, so there are no differing accounts to weigh, but the claims should be treated as preliminary until the full paper is examined.